This action might not be possible to undo. Are you sure you want to continue?
An Introduction to Formal Languages andAutomata
Third Edition
PeterLinz
University Californiaat Davis of
filru;;;:
IONES AND BARTLETT P,UBLISHERS
.BOSTON Stdlnry, Massnclrrsrtr TORONT'O LONDON SINGAPORE
6l;l .rf;ti.
etig*
t' o
dtry'l,tlti,tFI, hgfryfl6a
';;'ut: \ n qf I" A\ ,r'f7 lA ,obi
Workl Headquerters Iones and Bartlett Puhlishers
40 Tall PineDrive MA Sudbury, 01776 9784435000 info@jbpub.corn www.jbpub.com
Jonesand Bartlett Publishers Canada 2406 Nikanna Road Mississauga, ON L5C 2W6 CANADA
Jonesand BarJlettPublishers International Barb House,Barb Mews London W6 7PA UK
Inc. Copyright O 2001 by Jonesand Bartlett Publishers, or All rights reserved.No part of the materialprotectedby this copyrightnotice may be reproduced recording,or any infotmation including photocopying, or mechanical, utilized in any fonn, elcctronic storage retrievalsy$tem,without written permissionf'romthe copyright owner. or Library of Congress CataloginginPuhtication Data Linz, Peter. / and automata PeterLinz'3'd cd An introductionto formal languages p. cm. and Includesbi hliographicalref'erences index. G' A
26+,3 . L5 4 LooI
rsBN0763714224
l. Formal languages. 2. Machine theory. l. Title. QA267.3.Ls6 2000 5  1.3dc2l Chief ExecutiveOfficer: Clayton Jones Chief OperatingOfficer: Don W. Jones,Jr. and Publisher: Tom Manning ExecutiveVicc President V.P., ManagingEditor: Judith H. Hauck V.P.. Collese Editorial Director: Brian L. McKean V.P;, Dcsigir'and"Prodgction: \ Anne $pencer arket+rgiFauI Shefiardson V. P., Salcs anit*ffr . V. P., Man uf aeturingjandilnhrr'trrry dpntrol : ThereseBriiucr SeniorAgquisitionsEditor; Michacl $tranz f)evelopment and Product Managcr: f,lny Rose Markcting Director: Jennifer.Iacobson ect Production CoordinationI Tri{ litrm Pt'oj M anagcment Cover Design; Night & Day Design Composition:NortheastCompositors Printing and Binding: Courier Westford Cover printing: John Pow Cotnpany,Inc.
;..#F*F*.,.
00062546
Covel Imasc O Jim Wehtie This book was typesetin Texturcs2. I on a MacintoshG4. The fbnt families usedwere Computer The first printing was printed on 50 lb. Decision94 Opaque. Modern, Optima, and F'utura.
States Arnerica _. '_ of Printed the United in 04030201 lo987654321
I
L
lch4
I ,r./1,il.t!\
his book is designed for an introductory course orr forrnir,l larrguages, autornatir, txlmputability, and rclated matters. These topics form a major part of whnt is known as tht: theory of cornputation. A course on this strbitx:t rnatter is now stir,nda,rdin the comprrter science curriculurn ancl is oftrlrr ta,ught fairly early irr the prograrn. Hence, the Jrrospective audience for this book consists prirnrr,rily of sophomores and juniors rnirjrlring in computer scicntxlor computer errgirrwring. Prerequisites for the material in this book are a knowledge of sorne higherlevel prograrnrning la,nguage (cornmonly C, C++, or .Iava) and fatrrilinritv with ihe furrdarn<lnta,ls data structures and algoriihms. A colrr$e of in discretc mathematics that irx:hrcles theory, furrctions, relations, logic, set and elernerrtsof mathematical reasorringis essential. Such a corlrse is part of the standard introductory computer sciencecurriculum. The study of the theory of cornputa.tionhas several purposc$, most imprortantly (1) to fa,miliarizestuderrts with the fbundations and principles of computer sciettce,(2) to teach tnaterial that is useful in subsequerrt colrrres! rrnd (3) to strengtlrcrr utudents' ability tu t:ilrry out formal and rigorous rrrirthematical argurnerrts. The presentatiorr I ha,ve chosen for this text fa
III
lv
F RHrAciE
vors the first two purpose$r although I would rr.rguethat it a,lsoserves the give strrdcrrtsinsight into the material, thircl. To prt:sent ideas clenrly arrd 1,o rnotivation and ilhrstration of idcir.sthrough ex* tlte text stressesintuitive a,m1llcs.When there is ir choice, I prefcr arguments thtr,t a,reeasily grer,sptxl and elegant brrt rlifficult in concellt. I state clefinito thosr.'tlnt are concisr,l tiorrs ancl theorems llrecisely and givt: the tnotiva,tion ftlr proofs, brrt tlf'tt:rr k:ave out the rorrtirre and tediorrs rlctails. I believe tlrrr.tthis is desirnblc for peclagogir:nl rcasdhs, Many proofs are unexc:itirrgapplications of irrduction that are sptx:ific to particuLrr llrobletns. with diff'ererrt:es or contra,clit:tiotr, arguments in full detail is not orrly ullllecessary,lrtrt interPresenting $rrdr feres with the flow of the storv. Therefore, quite a few of the proofs are Inay consitlclr tltern lacksketchy irrrcl someone wlxl irrsists on complerttlrress in cletrr.il. I do not seq:this as a clrawback. Mathematica,l skills are uot ing else's argutttents, but comc frorn thinkthe byproduct of reading sorrreorle ing atrout the essenrxlof a problem, disrxlvtlrirrg ideassrritatllc to make the poirrt, thel carrying tltetn out in prtruistldetail. The kr,tter skill certainly arrd I lhink th.r,t the proof sketches irr this text providc has to be lea,rnerd, very appropriir,tcstartiug points fbr such a practitx:. sornetitnes vi(lw a course in the theory of StudentS irr courputer sclit1rrce computation aa urlnecessarily abstract and of little practical con$(xpelrce. 'Ib convinr:c thetn otherwi$e, t)nc treeclsto appeir.l tcl their specific irrterests and strengths, suclt a,stena,t:ity and inventivttntlss itt clealing with hardtoof solverllroblettts. Beca,user tltis, tny a,pprtlitt:h empha,sizeslea.rnirrgthrough probletn solving. I By a problemsolvitrg approa,ch, rrteattthat students learn the material prirnarily througlt problemtype illustrative examplcs that show the motiwell as their conncction to the theorcrns attd vation bohirrd the concepts, a^s tirne, the examples rrriryinvolve a nontrivial aspect, clefinitiotrs. At the sa,me a for whir:h students must dist:ovc:r solution. In such an approach, htlrnework contribute to ir,rrrajor part of the leartting procefJs.The exercises exrrrc:ises :rt the end of each sectiorr are designed to illutrftrate and ilhrstrate the malevels. Some tr:rial and call orr sttrdents' problemsolving ability a,t vtr,riotrs of the exerci$csare fairly sirnple, pickirrg up where the discussiotrin the text Ieaves ofl and asking students to carry ou for antlther step or two. Other extrrcisesare very difficult, challenging evtrrrthe best ntinds. A good rnix of such exercisest:ilrr be a very eff'ectivt:teaching tool. Ttr help instructors, I have provitled separately an instructor's guide thrr.t outlines the sohrtitlrrs of the exercise$irrrd suggeststheir pcdagogical value. Students need not trrr asked to solvc all problems bqt should be assigned those which support tlte goals of the course and the viewpoint of the instnrt:tor. Computer sr:ience currir:ulir tliffer from institrrtiorr to iilstitutiorr; while a few emphasize the theoretir:nl side, others are alrnost entirely orientt:d toward practiclnl application. I believe that this tt:xt can serve eitlNlr of these extremes, llrclvided that the exercises a,re stllected carefully witli the students' btr,c:kgroundatld intertlsts in mind. At ttle same time, the irrstructor needs to irrform tlle
PRnrecn
students aborrt the level of abstraction that is expected of tlxrm. This is particularly tnre of the prooforietrttxl exercises. When I say "llrove that" or "show that," I hrr,ve mind that the student should think about how a, in proof rnight be cclnstrur:ted ancl then produr:e a, clear argurnent. How fbrrrtal srrch a, proof should bc needs to be deterrnined by the instructor, ancl stutltlnts should be given guitlrllines on this early irr the txlrrse. Tltc content of the text, is allllropriate for a onesernestcr txrurse. Most of the nraterial can be covered, although some choice of errrpha.sis will have to be rnirde. In my classes,I gencrirlly gloss over proofs, skilr4rv as they are itr tlte tcxt. I usually give just enough coverageto make the rcsult plausible, asking strrdents to read the rest orr their own. Overall, though, little can be skippexl entirely witltout potential difficulties later on. A few uections, which are rnrlrked with an asterisk, c:rr,n omitted without loss to later be material. Most of tht: m:r,teria,l,however. is esscrrtial ancl must be covered. The first edition of this book wrr,u published in 1990,thc:stxxrnda,ppeared in 1906. The need for yet another cdition is gratifying and irrtlic;atesthat tny a1l1lrorr,ch, languagesrathcr than computations, is still viable. The via charrgcs ftrr the second edition wercl t)volutionary rather than rcvolrrtionary and addressedthe inevitable itrirct:rrra,c:ies obscurities of thtl Iirst edition. and It seertrs, however, that the second r:dition had reached a point of strrbility that requires f'ew changes,so thc tlrlk of the third editiorr is idcntical to the previous one. The major new featurtl of the third edition is the irrc:hrsion of a set of solved exercises. Initially, I felt that giving solutions to exerciseswas undesirable hecause it lirrritcd the number of problerrts thir.t r:a,nbe a,ssigned hourework. Howfor ever, over tlre years I have received so rrrany requests for assistance from students evt:rywhere that I concluded that it is time to relent. In this edition I havc irrcluded solutions to a srnall rrumber of exercises. I have also added solrro rrew exercises to keep frorn rtxhrcing the unsolved problems too much. Irr strlec:ting exercises for solutiorr, I have favored those that have signiflcant instructioner,l ver,lues. For this reasorr, I givc not onlv the answers, brrt show the reasonirrg that is the ba,sisfor the firml result. Merny exercises have thtl ser,me theme; often I choose a rupresentative case to solve, hoping that a studerrt who can follow the reasorrirrgwill be able to transfer it to a set of similar instances. I bclicrve that soluiions to a carcfirlly selected set ttf exercises can help studerrts irrr:rea"re their problernsolvirrg skills and still lcave instructors a good set of unuolved exercises. In the text, {lxercises for whir:h rr,solution or a hint is givcrrrr,rqr identified with {ffi. Also in response to suggcstitlns, I have identified sonre of ther harder exercist:s. This is not always easv, sirrt:e the exercises span a spectrrrm of diffic;ulty and because a problen that seems easy to one student rnay givr: considerable trouble to another. But thcre are some exercises that havcl posed a challcnge fbr a majority of my studcnts. These are rnarked witlr a single star (*). There are also a few exercisosthat are different from most in that they have rro r:learcut answer. They rnay crrll f'or upeculation,
vt
PRnrncp
suggest additional reading, or require some computer programming. While they,are not suitable for routine homework assignment, they can serve &s entry points for furtlter study. Such exercisesare marked with a double star (** ). Over the last ten years I have received helpful suggestions from numerous reviewers, instructors, and students. While there are too many individuals to mention by name, I am grateful to all of them. Their feedback has been in'aluable in my attempts to improve the text. Peter Linz
Chapter
1
fntroduction
to the Theory
of Computation 3
1.1 Matlrenratical Prelirrrirrrlricu ar,nd Notation Sets 3 Functions and Relations 5 Craphs and l}'ees 7 Proof Techniques I 1.2 Three Basic Concepts 15 Lirrrgrrir,ges 15 Grarnrnilrs 19 Automala 25 +1.3 Some Applications 29
Chapter 2 Finite Autornata 35 2 , 1 I)eterrrrinistit: Finite Accepters 36
I)ctc:rrnirristic Accepters and'IIrrnsitiorr Grir,phs Languir,gcsand Dfa,s 38 R.t:gulil,rL:lngrrages 42 2 . 2 Nondeterrriinistit:FiniteAccepters 47 Definilion of a Nonrleterministic Accepler 48 Whv Notxlctt:rrninism'1 52
36
vll
trrnars 89 Right.2 Elementary Qrrestionsabout Regular Languages 111 4.1 Closure Propertitrs of Regular Languages 100 Closure under Simple Set Operations 100 Closure under Otlter Operations 103 4.mma.r Expressions for Regular Languages 81 Regular Expressions for Describing Simple Patterns 85 3.4 Reduction of the Number of States in Finite Automata 62 Chapter 3 Regular Languages and Regular Grammars fl 3.3 Regular Gra.anrl LeftLinear Grammars 89 RightLinear Grammars Generate Regular Languages 91 RightLinear Grammars for Regular Languages 93 Equivalence Bctween Regular Languages and Regular Gra.l Delinition of a Regular Expression 72 Languages Associated with Regular Expressions 73 3.1 Regular Expressions 7I Forma.3 Equivalence of Deterministic and Notrdeterministic Finite Accepters 55 +2.rs 95 Chapter 99 4 Properties of Regular Languages 4.3 ContextFtcc Gramrnars and Programmirtg Ltr.2 Connection Between Regular Expressions and Regular Languages 78 Regular Expressions Denote Regular Languages 78 Regula.3 Identifving Nonregular Languages 114 llsirrg the Pigeonhole Principle 114 A Pumping Lemma 115 5 ContextFree Languages L25 Chapter 5.rrgrrages 146 .1 CorrtextFree Grammars 126 Exarrrples of ContextFlee Languages 127 Leftntost and Rightmost Dt'rivations 129 Derivation Tl'ees 130 R.elation Between Sentential Fttrms and Derivation 'llees 13? 5.2 Parsing and Ambiguity 136 Parsing and Mcnbership 136 Anlbiguity in Grarnrnars and Latrguages 141 5.vlll CoNrnNrs 2.
ngua.7 Nondeterrnirfstic Pushdown Automata 176 Definition of a Pushdown Arrtomaton tTti A LangrrageAccepted by a Pushdowrr Automaton I79 7.chines as Tlansducers 232 9.ges 210 8.1 The Standard T\rring Machine 222 Definition of a Thring Machine 222 T\rring Machines as Language Accepters 229 Tlrring Ma.3 T\rring's Thesis 244 .l Form 168 +6.2 Two Important Normal Forrns 165 Chomsky Normal Form 165 Greibach Normtr.rsfbr Deterministic CorrtextF}ct:Langua.2 Combining Tlrring Machines for Cornplicated Tasks 238 9.ges 200 Chapter 8 Properties of ContextFlee Languages 205 8.1 Two Pumping Lemmas 206 A Purnpirrg Lcrnrna fbr ContextFlee Languages 206 A Purnping Letrrnil firr Linear La.3 DerterrrinisticPushdown Autornataand Deterrrfnistir: ContextFr{lc Lirnglrri.ges 195 *7.CoNrErrts ix Chapter 6 Simplification of ContextFlee Grammars 149 6.3 A Me:mbership Algorithm for ContextF]'eeGrarnrnrr.'e Languages 218 Chapter 9 Turing Machines 221 9.\Productions 156 Removing UnitProductiorrs 158 6.72 Chapter 7 Pushdown Automata 175 7.2 Pushdown Automata and ContextFree Larrguagcs 184 Pushdown Autorrrata fbr ContextFlee Languages 184 CorrtcxtFloe Grammars for Pushdown Autorrrata 189 7.rs 1.4 Gramma.r.2 Closure Propcrtien and Decision Algorithrns for ContextFree Languages 213 Closure of ContextFree LangJuages ?13 Some Decidable Properties of ContcxtFre.1 Methods for Tfansforrrring Grammars 150 A Useful Substitution Rule 150 Removing UselessProductions 15? Removing.
rrnotBc: Solved By l\rring Machines 300 The T\ring Machine llalting Problem 301 H.4 I'he Chomskv Hierarchv 29Ir Chapter 12 Limits of Algorithrnic Cornputatiorr 299 12.2 Ma.etlucitrgOne Undecidable Problem to Another 304 12.1 Some Probletns That (ltr.yOption 251 Thring Machines with SemiInfinitc Tape 253 The OffLine Tttrirrg Mat:hine 255 'I\rring 10.nguages 318 .3 Norrtletertninistic T\rring Ma.tx:ursively A Language That Is Not R.4 [Jndccidable Problems for ContextFree Lir. Sta.t:cursively Enumerable 279 Erlrrrterable But Not A Language That Is Rer:rrrsivr:ly Recursive 28.3 ContextSensitivc (]rarnrna.rsarrd Lirnguages 289 ConterxtSrlnsitivc Languages and Litrear Bounded Aulomata 29t) Relation Betweeu Recursive and Ctlrrtt:xtSetuitive Languages 2gz 11. A Hierarchy of Formal Languages and Autornata 278 11.chirrt:s 261 10. 250 clf Ttrrirrg Machines with a.2 Uritlt:c:itlrrble I'robletns for Recursivelv llnrtmertr.(.chines 263 10.ble 278 Languages That Art: Not R.5 Liricar Bouttded Autotnata 270 Chapter Ll.l 11.3 Tlte I'osL Correspondence 12.1 Recursive and Reclrrsively Euurnerable Languages 276 Enumera.chineswith Morc Cotttplex Storage 258 Mullitape Ttrring Ma.4 A lJrriversal I\rring Machine 266 10.'onrnrlrs Chapter 10 Other Models of Turing Machines 249 10.t:hint:Therne 25t) Eqrrivalcrrt:tl Classesof Autonrata.2 Uurestricted Grarnmars 283 11.chiners 258 Mttltidimensional T[rring Mtr.ltlrr Languages 308 Ptoblem Sl2 12.1 Mirxlr Virriatiotrs on the T\rring Ma.
ewriting Systems 337 Markov Algorithms 339 LSystems 340 Chapter 14 14.2 Post Systern$ 334 13.1 Recursive Functions 325 PrimitiveRecursiveF\nctions 326 Ackermann'sF\rnction 330 13.Cot{reNrs xi Chapter L3 Other Models of Computation 323 13.2 14.3 I4.3 R.1 14.4 An Introduction to Computational Complexity Efficiencyof Computation 344 Ttrring Machinesand Complexity 346 LanguageFamiliesand Complexity Classes 350 The Complexity Classes and NP 353 P 357 343 Answers to Selected Exercises References 405 Index 4OT .
.
l topics.reessential to many of the special and complex corrstructs we crrcourrtrlr while wclrking with computers.tiorr. sinr:e without npplications there would be little interest in cornputers.ndthat a.inly true of computer science studcrrts who rrru interested rna.pra.re.rr.pplication. Thiu is certa. Tcl strrdy these basic principles.sk "why study theory?" 'Ihe first arrswer is that tlrrxrry provides concepts and principles that help us understand tlrtl gerrcral rrirturt: of the discipline. f'rom machine design to progratntrtittg. This attitude is appropriirte. The field of computer scienceincludes a wide rarrgr:of sper:irr. Tlte use of cornputtlrs irr thel rea. But in spite of this diversity.I N T R O D U C T I OO TN T H E H E O RO F T Y COMPUTATION otrprrterr science is ir. I'hose who work irr it oftcn hirve ir.inly in working on difficult applicatious from the real world.ctical discipline. Even . there are soure colrtlrlotr urrclcrlyirrg prirrt:ipltrs.l world involves a wealth of specific detail that must lre lerirrrrcxl a uuccessfirla. But givcrr ihis practical oritlrrtir.mrlrked pref'erencefbr useful and tangible problerns ovt:r theoreticrrl spt:c:ulirtion. onr: rnight well a. This ftrr makes computer science a very diverse arxl lrroarl rlis<:ipline. we construct abstract rnodels of corrrllrtcrs and comprrtation. These ruodels embody the important features tlnt are cornnron to both harrlwarc and softwtr. Tlteoretical qucstions arcrinteresting to them only if they help in finding good solutions.
Chopter I IurnorrucjrloN To rHE Tsr:enY ol' Col.ges. and the usefiilnessof a discipline is often sentials of any sc:iurrtific clf clependenton the exi$ttrrrt:c simple. the insiglrts wt: gain frotn studying them provide the foundations on which sptx:ific. wtr will forrrralize the concept of a rnechanical computation by givirrg a precise clefinition of the term algorithrn and study tlrt: kittds of problems that are (and tr. a.bout programming lir.tr. thtxlric:satrd laws.tlrttrnaticsthat will be required. prograrntning laugua. A second.1. yet powerfirl.rxlllcrhaps not so obvious answer. we look at these ba.lir. Trr rrrodel the hardware of a comprrtt:r. and can make decisions in tra. is that the ideas we will discuss have srlmt: irnrnediate and itnporta.lthoughlittle is needed beyond the definition of a. The construc:titlrrclf rnodels is one of the esdisciplitte. We cau learn a great deal ir. The srrtricc:trnatter is intellectually stimrrltr.lly. the conchrsionrr wu draw will be based on rigorous arguments.they have rnarry of the same esserrtial features. may have somtl tcrndigital computer.s. It provides ma. we will show the clo$er(xlrrrrc(:tiotr between these abstractions and irrvc:stigate the conclusions we carr tlcrive from them. An automaton is a. automata). tr. This is lenging. Fina. A formal language is arnir. In Section 1.bstractiorr of the general characteristics of prograrnming languages.spof the terminology and of the elementary results of set thtxrry.retoo simplc to be applicable immediately to realworld situations. Tlte reader will need a rea^sonably good gra. Although sorne of the formal langrrirgcs we study here are simplt:r thirrr prograurmitrg langua. This will involve sclmelrnilthernatical machinery. In this hook. In tlx. The third irlrswer is oue of which we hclpc to txlrtvittce the reader.'first chapter. although these requirementn alrelnot t:xterrsive. This tr. It :r. we revit:w thc rrrain ideas from ma. The fields of digital design. we introcluce the notion of iln automaton (plural.rrnlxrtcrscience.rxxlpts porary utorilgrl. prrzzlelikeproblems that can lead to ir()rrrc probkrrnsolvittgin its pure essence. Perhaps the rnost stringent requirement is thu rrtrility to follow proofs aud .ges. A f'ormell lirnguage is the set of all strings permitted by the rules of fi)rrnirtiorr. A ftrrmal lirrrgrrage cotrsists of a set of symbols irrrd some rules of forma.ndrrirnpilt:rs are the most obvious erxarnplcs.t possessesall Lhe indispensable f'eatrrrt:stlf a irrput. T!'eesand graph structures will be rmul f'requently.rrguirgesfrom formal languages. from opera.tirrg atrd furr. we will look at models that represcrrtfcatures at the core of all c:ornputers and their applica. ftnetions. In the cour$e of orrr stutly.ny crha.sicidea.nsformirrg the input into tlte output. produces output.nt applit:atiorrs.but there are rnanv othcrs. rlevelopment is ba*sed.lsleepless nights.sin a vcry broad way to set thtl stagc for later work.tputarlott whertr such moclelsa.renot) suitable fbr solution try srrclt trtechatrical Ineans.pproach is of course not unique to rx. While intuition will frcquently be our guide irr exploring idea.ttern rtxxrgrritiorr.tiorru.tion by whit:h thcse sytnbols can be cotnbined into crrtities called sentences.beled. anrl rclatiorrs. construr:t thir.ting systerrrs to pa. The cotrcepts we study hert: nrrr like a thread through mrrr:h of txrrrrputer sciettce.directed graph.
lphabet.Sz : 5r {z : z € Sr arxl r fr 52}.92}.sthis necessarybackground.the fbr set of irrtt:gers0.2}. in S.t rl is grea.9.sto illustrate that thesc c:tlnr:rrpts have widespread uses itt cornputcr ur:ience. A set is specified by cnr:losingsome description of ils elernentsin curly bracxrs.diction. 5 1 1 5 2 : { z : r € S r a r r rr E .tementthat r is not in S is written r f 5.1 is induded to review some of the rrririrr results that will be used arrrl to entahlish a notational colrurrorr grourrrl f'rrr subsequentdiscussion.l idea. Sectiott 1.1 MarrrnlrATrcAl PRr:r. we will make all of this rmrr:hrnoro precise.. thtl goal is to get a. 9 z } . b.but for lhe ntotttettt. To rnakc this .tion is complementation. gralrllnar$.sicopera.pplica. Later. l . ziseven} (f.ning clear. and difference (). ancl proof by c:clrrtrir. in which we write S= {i:i >0.ndrj is even. consists of a. wc givc some simple a. When the neecl arises. We read this as tt. {a.1. denotecl by F. intersection (n). defined as 51U52 : {z : r e Sr or r €." implying of course that z is irrr irrteger.6.utomata. In Section 1.ter tltatr zero.9is sc:t of irll ri. .while {2. exa.2. This includes farniliarity with the hasic proof techniques of dcrluction. .The discr. The usual set operations arc union (U). Tltus.ll elernenls not. Mothemoticol Preliminoriesond Nototion WffiWmHilW Sets A set is a collectiott rtf t:lclrno'rrts. The cotrplerntlnt tlf a set .lnrwnRrES AND Norauou atr utrderstarrdingof what constitutes proper rnathcrnirtical reasoning. 1.We will assurne that thc rcirrlrlr ha. To indicate that r is arr clcrnrrnt of the set 5.rssion these two scc:tionswill be intuitive in rather tltirrr rigororrs.l) frrr the ltr.clear picture of tire corrceptswith which we are derrling. srx:h thrr. These cortcepts oc{:rrr irr rnarry specific fbrms throughout the book. without any structure olher tharr rnr:rnhership. 2 is shown as 5 : {0.tions of tlrr:sc gerrera.mple. trrrd a..} denotes the set of all positive everr irrtcgrlrs. we write r € .J 1. Ellipses are usetl wltcncvc:rtlNl rneir. we take afirst look at thc r:entral concepts of languages..4. a. .3..9.z} slands for is all the lowercaseletters of thc Engliuh a. In Section 1. we use rrrore explicit notation. The sta. Anothttr bir.st example. induction.
Here lSl : 3 and lZtl :8.. { c } .n#S}. hut 5 rxrrrtilirrs ir. known a. Wc write this as 5rc5' If St C S. it is obvious that Sus: Sfr:5.9r is said to be a subset of 5 if every element of 5r is also atr element of S. this is denoted bV l5l.rrelernetrt not in 51 we say that Sr is a proper subset of . " } } .t the universal elements is. { a . { o } .c}. are needed orr $eivc:ral A set .4 Chopter I llqrnooucrroN To rrrn THnoRv cln Cor. then its powerset z s : { f r . thcrr set U of a'll possitrlt: F:{*:rr(I. A given set norrnally has marry sutrsets. Observe that 2s is rr.f I is If $ is thc set {a. { o . c } . fr =U. wc write tltis as $rCS. Snfr=9.3) occasions. The following useful identities. size of a finite sct is tht: rrurrtberof eletnents in it. 5::5. ffi: F l n S z : . then This is arr instirrrce a general of result. The sct witlt no elements. r : } { b .ncl5'z have IIo coilllrron elemeut. b . b } .9. set 5 is callecl the powerset of S ir. If U is specified. 5 rU S z . (t 2) (1. called the ernpty set or the null set is denoted by #. set of sets. TIte set of all subsets of a.if 5 is lrsl . If 51 a. n .sthe DeMorgants laws.b.rlsl I .t b } .tpu'rn'rtou rnerarrirrgful. tltett the sets are said to berdisjoint. that is. Exottplq l'. Sr n 5'2 = fl.nd is denoted by 2's.we need to know whir. Srl3z. { . finite. A set is said to be linite if it contains a finite nlrmbcr of elemenls. Flom the definitiorr rtf a set. 'Ihe otherwise it is infinite.
tion carr be used. 5 ) . 2 . otherwist: ./) :r€S'1.( 4 . If / dcrxrtt's a futrcLion. 5 1. n . . 1 ' r()4 .( 4 .ny applir:rrtiorrs.lltxl sct is its range.in is a. positive constant c such that for all rz f(n)tcs(n). we writer S:Sr x52:{(*.ltvttw.mples. 3 ) . The nolation is extendeclin a.f ha. we sav that .f is said Lo be a partial function. .in of / is a strtrsc:t . we say thrlt / is a total function on 5r. Itr in the heha. . . . )r: . are in the set of positive integers. and the serxrnd / : . the donaiu and rauge of the firrrt:tiottsinvolved In ma. ( 2 . $ 2 of to itrdicate thal the doma. 1 ' r . which itself is a set of orclerednairs. generally S r x 5 ' r x ' . We write this ir.then the flrst set is t:ir. € S r } . For the Ca.51atrd that the ra. Thc pair (4.tes is oftetr sullicient and a corrrrrrorr order of magnitude nota.s f ( n): o ( s( n) ) . ( 2 .IEs Norauolt In rnany of our exa. 6 ) } . . Let / (n..ngeof / is a subset of 52.sordcr at most g.nobvirlrs firshiorrto tlte Cartesian product of rnr)rt)than two sets.\ AND 1.rtcsiarrproduct of two sets. 3 . Furthermorel we il. If thcre exists a.) and .rcoften interested only very large.( 4 . 3 ) .q.q(n) be functions whose doma. ' x 5 r : { ( r 1 .but (2. Srrr:h$ets arc said to be the Cartesian product of other sets. (2 S r x 5 ' z : { ( 2 . llnir are written matters.:.unirptl cletrtetrtof the domain another set.virlrof tltese functions as their arguments btlclottte arr such c:asers urrrlerstanding of the growth rtr.4) is not.n. T 6 c ' 6 . If tht: tlornirirr of / is all of 5r. Notc that tlte order in which the elements of a. I Functions Relotions ond A function is a rukt that assignsto elements of one set a.the elements of il stlt irre ordered sequencesof elements frorn otJrer sets. 6 ) . 2 ) .2) is in 51 x 5'2. . T 2 .1 MnrHntutnrtcAL PRt:t.Ee Sz}. . subst:t of the positive itrtegers. We write of /. 2 ) .
In order of rnagnitude notatiorr. wh{:rc il.(ru)). there exist constants c1 and c2 such tlnt if cr lg(?l)l l/ (")l ! czle(rl)l. wtr ignore multiplicative constants and lower order tertns that becotnencgligibkl as n increases.)). expresseda. Finar.3 Let f (n) : 2nz + iJtt.Still.sthe first element of a pair. l/ (")l l c ls(?z)l then / has order at least g. < / and g have the same ordcr of magnitude. ea. ExompleI. . s (n) : o (ft. y r ) . } . If ttris is not satisfied.n element in the clornain of t}re furrc:tion. the . tlrrl syrnllol : should not be interpreted irs txlra. .s I(n)*o(g("))' In this order of magtritude trotittitlrr. .nalysisof algorillurs.'l'roN 'r'Hu If .renot sensibleand catr lead to irtr:clrr(lct rxrnclusions. if used properly. is a. 2 Thcn / (t) : o (s (rl)). I (rt) : "'t..nge. Ma.'l'Hnonyor. and gti is the corresportdirrgvilhrelin its ra. / (n) : o (h (r. I Some functiorls can be rtlprt:srlrrttxl by a set of pairs { ( " r . the order of magnitude argurnents tlrrrr tlr: effective.lity a.'ConrurArrorv 'r'o ChopterI IurR. u z ). I'rrr whit:h we tuJe /(") :o(g(")).lly. ( n ) : 1 o r z+ 1 o o .ncl order of magnitude expressiorrs r:annet be treated like ordirrirry cxl)r{}ir$ions.ch can 11 occur at rno$t on(:ea.nipulations such as O (rz)+ i) (n) = 20 (n) a.onuc. a*swe will see in later r:hirpturson the a. ( r z . h .For such a set to delirrc a firnt:tion.
in ha.. Clearly this is atr equivalencerelation. Wc srry that the edge e.tethat a.ctly orrclitssociated eletnent itt the ra.4 ' ' Consider the rela. Such a construct is actually ir.ph)..ransitivity.n orrtgtlirrg edge for ?ri and an incoming edge forr... Both vt'rtices atrd edgesmay be lahclctl.37) arr crpivirlcrrce relation.'tt2.tion on thc sct of rrotrttegativeiulegers defined hy 'l': l' if and only if rmocl 3 .sirrce we associate a direction (fiorn ui to u6) with each edge..i: \U.chedgttis a pair of vertices fiom V..tion tht:re miry trcl scvtlral such elernenls in the range. I Grophsond Trees A graph is a construct consistirrgof two fitilte sets. 12 = 0.l.pair (r:. Ea. then n # E. a generalization of thc is concept of equality (identity).q. :y r' dt and the transitivity nrlc If fr: A a n c ly : a. and 0 = il6. 'Ihen 2 : 5.'Dn} of vertices and the set E: {e1. the syrnrnetry rule if '': r .Ltk) is an edge from ui to tr4..rATrcAL PnnLrNrrNanrES AND No'r. irs it satisfies reflexivitv. syrntttetry. Graphs miry bc labeled. and l. A relatiori rlcrrotexl lry : i... Orre kirrd of relatiott is that of equivalence.err} of edges. we write :I:='!J.i. a*ssor:iated . directed graph (digrrr.t'l'tor'r set is called a relation.37mod3.sexir. frlr irrstance e.ll z. the set V : {tt1.e2. Relatious are Inore general thtlrr firrrt:tions: in a function each element of the doma. a label being a ntrme or other itrformation with parts of the graph. ll r r a u hvn o . is a.nge.l M.'r. I f xsnrslF f ..uml. To indica. consiclered atr equivalence if it satisfies three mlcs: the reflexivity rule r: = r: for ir. in a relir.
u3} and edges {(u1. The sequence edges(ut. (r*.ched.lly. Starting frotn tlte giverr vcrtcxr say ?ri.t)t.. The height of the tree is the Iargest level number of any vertex. r r " i.. we will refer to atr algoritlun for lindirrg all sirnpkr paths between two given vertices (or all siurplc c:yrlcs bn^sed rr.th we are rxlnstnrcting. . Fina.. ( u i . ( ' . we will have all sinrple paths of lerrgth two origirrrrtirrgat a. ( u 3 . 1 . of llrt rxit ir. we carr llsc tlrrl following obvious method. The length of a walk is the total nurrrber of rxlgcs travr:rscrl .cycle with base u4. (z1. u * .sirnple one.in going from the initial vertex to the final orre.y do not lead to arry vcrtclx alrtlirdy rrsed in the pa.. In Figure 1. For a. These are called the leaves of the tree. graph are labeled. Since there ate orrly ir finite number of vertices.id !o be simple. . If no vertices other thatt tlte base are rrlllc:itttxliri ir r:yr:le.trxltxlges is ca. ( t r . A walk fron ui to itself with rro rcpcir. we have all paths of length orrt: txlgt:s (u. If there is an edge from ua to ui. (rr.. ' u i ) . we will eventually lisi all sirnple paths beginning at rr.1 so rea. then ua is said to be the parent ()f rrj. Flom these we select those ending at the desired vertex.Chopter I Itrrn. .ur). u3) . called the root.u'r'n'r'tow Figurc 1. This definition implies that the root ha^srro irrcoming edges and that there &re some vertices without outgoing edges. . .rt) is ir cyc:le. u 3 )  i s d e p i c t e di n F i g u r e 1 . .ur) .vertex).slong as the. This label is thc scqucrrr:c r:dgo ler. .llerla.nd ui the child of u1.. If the edges of a. In Figure 1. Iist all orrtgoing (ui. The graph with vertices {u1. we a. (rr.n eclgefrom a vertex to itself is calk:d a loop. il.1 f \ \'otl t Graphs are conveniently visualized by diagrarns in which the vertirx:s are represented as circles and the edges as lines with arrows corrrrt'cting tho verti<x. u2. list all outgoing edges .. a.116). A s e q u e n c e f e d g e s( a t . startittg at u4. After we do this. (u3.u2) is a simple perthfiom ?rrto ??.tt. At this point.. .rr).'s. On several occasiotts. u r ) . We r:ontinue this until all possibilities are accounted for. on If we do not concern ourselves with efficiency.2. ) . we can talk about the label ofa walk.1 there is a loop on vertex u3. such that there is exactly one path frorrr the root to every other vertex.1. The level associated with each vertex is the nunber of edgesin the path from the root to the vertex.s s a i d t o h e t r .then it is sa. A tree is a directed graph that has no cycles. and that htust)ne rlistinct vertex.llverrticesuk1.. Tlees are a particular type of graph.bels of encorrnteredwhen the path is traversed..w a l k o ) fiom rri to urr.u1) . .onuc'l'roN'l'o tsr: Tttr:rtHv ou Cotr. . These terms are illustrated in Figure 1.rr). A wrrlk in which no eclge is repeated is said to be a pathl rr path is simple if no vertex is repeated.
.rth of a few specific instances. Pz. we want to a*ssocirr. arrd so on.1. $rrpposealso that the following holds: 1.P.nulumnn. Suppose we have a sequenceof statements Pr . Then Condition ? tells us that P611 alsrl rmrst btr tnre. In mathema. Therefclrc. Pk are true.L"'"13t I I At times.t we know that the first h * 1 statements are trlrc. Irr a pro<lf by irrclucticln.. ilnd rnilny proofs artl simllly a . imply the truth of P. we want to prove to be true. tlrt: trutlm of P1. Pz. an in srrch ciL$e$ talk aborrt ordered trees. we rrrguo as follows: Ftom Condition 1 we know that the first k statements are true. Induction is a technique by which the truth of a number of statements can be inf'erred from the trr.. we employ the accepted rules of deductive reasorring.rATrcALPn.. . The problern is such that for any z ) A. .. Furthermore..1 MarHu. 2. We can then use induction to show that everv statement in this sequenceis tnre..P2.. Thc cltairr of rcit*sorrirrg t:itrr btl cxtended to any strrtcrnerrt. .ticalarguments. Brrt now thir. Two special of proofteclufques are used so frequently that it is approprintc to rcvielwthem briefly. For some fu ) [. . .sequence such steps. we r:arr allply Contlitiorr 2 agairr to tlaim that P61z must be true. we know that Pt..2 Root Height = 3 . we More details on graphs and trees can be found irr rnost books on discrcte mathematics.1.rEs AND Norauoru Figure 1. every statement is true.te ordering with the nodes at each level.. ProofTechniques An important requirement for reading this text is the ability to follow proofs. \Me need not explicitly continue this argument because the patterrr is clcrrr. These are proof by induction and proof by contradiction.
a finite a'utomat(lrr (:arronly deal with situations in which tlrt: infortnalion to tre stored at irr arry tirrre is strictly bounded.r't: sirnple' spea gcrreral scherneiutroduced in the ltrst drapter. 35 . I'his type cia. is sptx:ifi<: state.FIN ITE AUTOMATA tho". Ttl llrogress.rrrlstirrt to clevelop . particularlv tIrc tlist:ussiotr autornata' was hrief antl irrftlrrnirl.utornittorr Exatnple1.ph. The a. we harvt. linite atnoutrt of inf'ormir.n input file cannot be rewritten. But sitrcethe number of srxrtrstir.l ur introduction in the first cha.pacityte "rcmcmbcr" things during the comprrtir.tt:s firriLe.t. provide formal defiuitions. orrly ir general understanditrg of whir.l case of thc of autclmittotr is characterized hy having no tt:rnporary storage. whit:h ir. Wt: llcgitr with linite accepters. Since a. a firritc itutornittott is severely limitcd irr its A ca.tirlrr.t irrl auAt this point.ptcrto the basic cotrceptsof comprrtaof tion. ir. we must be more prct:istt.r tomaton is and how it c:itrrbe representedby a gra. 1ti is tt.rigorous results.ninstarrt:t: clf ir firrite acceuter.tiorr carr be relained in the control rrnit tly placittg Lhe uttit itrto a.
the string is ircx:clltedif the automaton is in one of its Iirtr. Q x X .lizeand represetrt fittitr: irtrt{lrna. itrlrut syrnhol is a.tesclf some clfa M.l function called the transition function.n. ond Grophs Accepters Tronsilion Deterministic sniiit$lil]tnir tiftffif[j! f. in whir:h the vertices represerrt stattls arrtl the edges of represent transitiorrs.d. Otherwise the string is rejer:tcxl.nhm one input symbol. In dist:ussirrg arrtomata.te4u ancl the current.Tln input rnechauismcan Inove otrly frorn 'I'he tratrsitiorrs Icft to right and reads exactly ontl symbol on each step.tes.qt) la.. while the labels orr the eclgcs art: the current values of the input sytnbol. if rI (qo. internal state to another are govcrned bv the transition futrctiorr frorn one d. An edge (qn.tir. where Q i s a firtite st:t of internal states. then the graph associatcd with M will have one vertex lalrclcdq0 irnd another labeled q1. the dfa will go itrto stirtc' q1. d .if qp and qt are interlral strr.F). t J O € Q is the initial state.ssurnecl be irr thc initial state q0.a) : gr . Drrring eelchmove advancesone position ttt tlrc right. The initial .mple. clt:iu and intr"ritive we picture to work with. Rrr tlxtr.ffifrtll A deterministic finite accepter or dfa is defined by the quintulller M : (Q.Q iu el tota.qs. T C Q is a set of final states. Thc Inhels on tlre vertices are Lherrirlrr()s tlx: sttr. it is a.E.rx:epteroperates in the following tttanrrt:r. A deterministic finitt: ir. it is essential to havc a. then if tlrtl rlfa iu in sta. with its input mechanisrrrort the lefhmost symbol of the input strirrg. When the etttl of tlxr string is so each rnove colrsurrrrlri reached.) = q't. tlrtl irrlxrt meclha. Trr visutr.Chopter 2 AUToMATA ffi FiniteAccepters Deierministic The first type of automa.tes. Ibr exirtrllle. We start with a precise frrrrnir"ldefinition of deterministic accepters. At to thc initinl time.ton we study in detail are firrite accepters that are deternfnistic in their opera.tion.l stir. E i s a flnitc sct of symbols called the input alphabet. of the automaton. use transition graphs.bclcda represetrtsthe [ratuitiorr d (gs.
d. d.t sta.ttcr to converl from the (8. ) : q' n . 0111' and :.redrilwn with ir.. is More forma.nsitiorrgraph Gna has exactly lQl vcrtitr:s. are now at the end of thrl The dfa does not a.rst. { 0 . rathrlr tltatr a single symbol. finitt: ttt:t:cpter. qi) Iabeled a. if M : (Q. Stnrting irr state gs.') a cletertnitristic: then its associatedtrtr. at the same time.For every transitiott rrtle d(rJ. vertex assocria.bt:lt:r1 arry vertex.i:i.nsition firrx:tiotr d* : Q x X* .lly.plr:RS 87 Figure 2.1 I arrow ttot origiuating a. and its va. c:orrsecutive 0's.q t . :i. 1 } . The ser:orrdargutrtettLof d* is a string. while tltose labeleclwith qy € F arc tht: final vertices.l ma.we $eethat the autouraton will accept thtr strirrgs 101.double circle... whcrc d is given by This dfa accepts the strirtg [Jl. It is a trivia. not 100or 1100.tctlo.nsitiorrgraplt represeutatiou and vice veruir. q0.2.li:i ii =r.nedge (qr.lo.cccptthe string 00' Therefore. For exermplc. t:itch otte Iabeledwith a diff'erentq.i €Q. represents tlrc dfh } . 4 0 { q r } ) . the l is relirrlirntl the autotnaton goerJ strirtg and..tecl the initial vertex.1 DnrnnluNrsrrc Frurrn Accl. X.Q. Next.crt:cpted. it will be in uttr. X. d . r 1 z .# T . hrrt tlf b .tc will be identilied by a. IUI= ( { g u .tonwill llc irr after reading that string.trincoming rrnlir. the string 01 is a. irr a fltral state q1. 11001. Final sta. Looking at the edgesof thrl graph. the syrnbol 0 is rcird fi. 6 r"L+Q It is convenient to irrtrorlut:e the extendecltra.. By similar since after reading two reasoning.if d (qo.a): qi.luegivtls tlre stale the autotna.f. the 'Ihe with q6 is called graph has a.1I) tlc:firritionof a dfa to its tra.tesa. we see that the arrtrlmatott remains intcl state 4r' We in state qs.
e F} .l) . vious: the language is the set of aJl the strings accepted by the automaton. lsfi. we use (2.F) is the set of all strings on X accepted by M. we a. we can define d* recrrrsively by d.A) : q.38 Chopler 2 FIt'rIrp AuToMATA and 6 ( q t . First. longuoges ond Dfq's Having made a precise definition of an accepter. But d" (q0. d.we get !'!'' d*(qo. b ): q z . : d(o:. let us apply thesedefinitions to the simple caseabove. r The language accepted hy a dfa M : (8. q) d (go'a) (2.E. The associatiorr is ob.3) : Substitutingthis into (2. ab): qr. then d* (qn. In formal notation. (2l) 1' r)\ &t+!qF Ir u € Ej. Formally.3). b) ab] a).!: oz ab) as expected. (9. go.ffl$ifii .(qo. s EA To seewhy this is appropriate. .0) : d (d.2) to get U_!no' :_d (d*(qs. L (M): ur) {ru e E* : d* (q6.renow ready to define formally what we mean by an associated language.
1) arrcl rMe (2. Sut:h rnultiply labelerdcdges are shorthand for two or mor(r distittct trirrrsitiols: the trelnsition is taken whenr:vcr the input syrrrbol nratches any of the edge labels.sedon graphs are emvalid as those that use the fbrrrtrl properties 6f d' Thel following preliminary result gives us this assura.r.'tt.llNIS'tICiFtNtre Accr:p'tuns 39 and trlrrstltluenlly d*.cr:cptedby the a'rrttlrlatotr is f.nt:tl. until the The automaton irr Figure 2. All other irrput strings are rejtN:tt:d. A dfa will pror:tlsstlver. so t'hat we irr'(l justifiecl in ca.t e$(rirpe.''b:rt.rlyfiorn tlte graph tha.ry number of c.which ptlssible.ccolltit.2 wt: allowecl the use of two labels on a sitrgle edge. a urrique tnove is dtlfirrcd. Whilc it is possible to hase all arguments strictly on thc properties ofthc transition functiorr attcl its extensiorrtlrrough (2.te q2 is ir trap state.te frotn which it can never 'I'ire sla.gea. then the q2. If this is also the la"rt syrnbol of the input. so ther.cr:ttlttsall slrings clclnsistirrgof au arbitrtr.narrtornatott deterministit:.tiorr. thg autorlatotr a.chstep. In our discrrssiorr. Note that we rt:cpire that 15. Nonaccepta.t L(M): ( ) { r ue X * : t 5 * q o . followed by a single b. string is accepted. Consider the dfir. Figtrre 2. I These exarnples sltow htlw t:ottvenient transitiott graphs artl for workiug with finite irutotnata. In set nota.renttt rtfsled bV the rtlpresentation antl thal arguments ba. wc tttust of cotrrse irave sonre are more intrritive. ttre langua. as far i.lling such a.nt:tt in a nrlrtlirral state.2 In drawirtg Figure 2.2). the clfa goes into trtir. bc trltal functions. At ea.teq11 first b is etrcountered. To do so. Tf nqt.2 .irr Figure 2.1 Dnrurl.cc:cpt or not a. * lu.2 reurtrins in its initia'l strr.2. use gra. Wtr see clea.rrd rnealls that thtr tlfa stops it either ir.phs. the results are hard to fbllow. w f F I .ystritrg in X* ir.r assllrirnce that we a.> 0l .
irrrtl k:t Gna be its a.sa sequence of "if" statements. Agir.l. t\ implies that there is a walk in Gy from qi to qj laheled u.. arrtl . Corrsirlr:r then any .ur): qi.in.nsition graphs are very corrvcrrient for the kinds of argurnents we warrt to make here. 'l'herr frrr every q?.3 is equivalent to Figure 2. straiglrtforward way to show that the existence of such a path irrrplit:s (2.. yet typiur.X. so wr) rr$e them in rnost of our discussions. Whilt: graphs are convcrrir.mination of srrr:h simple carrcs Ilxample 2.nt fbr visualizirrg irutomata.. But the .et M: (Q. Here the row Iabel is thc r:urrent state.ssociatedtrarrsition graph.mplesand proofs more transpa"rent than they worrld be if we used thc properties of d'*.F) be a deterrninisticfinite accepter.€ X*. The table in Figure 2.n proved rigorously using an induction on the ir$ be Icrrgth of ur.w) : qi if antl only if there is in G.w) : qt (.1. then M rmrst have a tranuition d(qr. so that by construction Gy has arr cdge (Sn.irly obvious frorrr irn exa. Thus there is a walk in Gnr labeled ua: u between {a and qi. Since the result is ohviously true for n: l.urof lr:rrgth n * 1 and write it ns . whilcl the column label representsthe currerrt input symbol.4). Since lul: n. as a simple tablelookup or a.qi) with label a. to qi. Tra. In construc:ting automata ftrr la. The errrtrv in the table tlcfines the next state. the result of tlx: theorem is so itfrtititfr' obvious thrrt a formal proof seems unrlecessary. Proof: This clairn is fa.qo. $o sta.mple. The first is that it is a simple.rr. ilire secorrdis that the result will be rrsed over and over. It is apparent f'rom this exarnplc that a dfa can easily be implemented a^s ir {xrmputer prograrn. Supposenow that 6* (qi.40 Chopter 2 Frr.l example of an inductive proof in connection with automata.ll strings u with lrl < n. we can clainr by induction that. But if d* (r71.(qi.ri. we errrllloy reasoning sirnilirr to that for prograrnming in higherlevel languages. This rnakes orrr exa.u :.ting and proving it as a theorerrr lets us argue quitc confidently usittg graphs. there rnust be a walk in Gy labeled u from qi t..ei € Q. 6* (rqi. other representations art) also useful. Fbr exa. thus completing the 1rr{lof. 6. we can represent the functiorr d as a table..DQ.1air walk with label zu frorn q.2.u): q6. fclr example.ru: Aurovarn I. The best implernentation or representation depends on tlte specific applicatiorr. Assurne thnt the claim is truc frrr a.() qk. I lJt_lr. The argurnerrt r:irn be turned around in a.a) : {i. It r:a. f'or every 'ur€ I+. We went through the details for two rt:a^sons.nguagesdefinerrl informally.
an initial"state.".I DETERMINISTIC FINTTU ACCEPTERS 4l Figure 2. The only issue here is the first two symbols in the stringl after they have beerr read. b} starting with the prefix ab. If the first symbol is an a. and the second is a b. I Figure 2. where it will stay since the rest of the input does not matter. . Iz {t 9z !t ?z programming of a dfa is tediou$ and sometimes conceptually complicated bv the fact that such an automaton has few powerful fbatures. no further decisions need to be made.2. and one nonfinal trap state. Itwo statesrfor recognizing ab ending in a final trap state. We can therefore solve the problem with an automaton that has four states. b.4. the automaton goes to the final trap state. the automaton enters the nonfirlal trap state' The simple solution is shown in Figure 2. On the other hand.$ b' tl b !o ?o ?t q.J Find a deterministic finite accepter that recognizes the set of all strings on X : {a.4 . if the first symbol i$ not an a or the $econd one is not f.
We are only interestcd in the la. Like variable narrr()sin ir prograrnming languager. hecause Iater synrbcllsdo not matter.5 liol*r# riilIr. In this case.0 0 . Thc strrrcture and properties . We can keep track of this hy pr. The family of languages that is accepted lty detcrministic finite acceptersis quite limited.l state. This state mtrst be a trap stelte. tlx: sta.rtting the autornaton into specific slates arrrl la.rrr I 1 ' ( o t lc ) o . Fcrr cxrnvenience.Chopier 2 }'rllrn Aurolr. we part of string to the left. . it is vr:ry ea.this nonfi. If we labcl the states with the relevarrt symbols. but we still rnust add provisions for the srrbstring 001 occurring in the middle of the input. For exantple.syto see what the trrrnsitions must be.5. whether or not need to know sclrnel the two previous syrnbols were 00.heling them accordilgly.tra q Figure 2.whett a synrLrol is read. In decidirrg whether the substritrg 001 ha. We must define Q and d so thnt whatever we need to rnirkn the correct decision is rernerrbered by the autornaton. d ( 0 0 . becausc this situation arises only if there are three consecutive 0s. For example. but wt: also need to remember whethrlr or not it has been preceded by one or two 0s. f i ( fixotrtpl* [.to see that the solution is irrdt+ed correct. We will call finite autornatal. A complete solution is shown in Figure 2.ted such a set of Ianguages a family.state names are arbitrary arrtl r:an be chosen for ttttternorrit: rea^rons. for example.ssocia.cceptsall the strings on {0.4 Find a dfa thtit tr. a fact we remember by keeping the dfa in the state 00. This gives us the ha"sicstructure of thc solrrtion. All other stateri are acceptirrg state$. 0 ).te in which two 0s were the immediately prrrcding symbols can be labcled simplv 00.te is labeled 001. If we consider all possihle with thertr. 1}.st two. I Regulor [onguoges Every finite arrtomaton accepts sorne larrguage. Tbar:ea fbw strings. tht:n it must be rejected. we get a set of larrguagcs ir. except those corrtaining the substring 001. This implies that therc must be a path labeled 001 lrom the initial state to a rronfina.soccurred. If tht: string starts with 001. We see frorrr this exarnplc how useful mnemorricrlabels on the states are for kceping track of things.rral sttr. such as 100100 and 1010100. we need to krrow not only the currcnt input symbol.
Thc: cornplete solution is shown in Figure 2. and ends with an a. wltat is between is immateritr. it will take the dfa out of the final state. nanely.l irr value).. What this dfn.LL)1.L is called regular if and only if there exists sornedetertnitristic flnite accepter M such that L: L(M).r to Exrrrnpkr 2. by definition.ngualgcis rcgular.ccepts strirrg if atttl otrly if it begins rr.m tially tlrc sartreforttt (but noi necessa. This clifficulty is ever(xrrnc try sirrtply prrtting thrl rlfir irrto a final state whenever the second a is enrxlrrrtcrtxl.ngua.gc rr:gular by corrstructing a. all we have to clo is find a. e {a.s 2.mple 2. but a little more complicated. lf this is rxrt tirc end of the string. caclt a taking the autorna.* 1".o. R. tlrc lir.2 regular. L.3.l.11t2 b}. and another b is ftlrrrd. it will be obvious tha.tt: 'u e {o. Aga.nexplicit exprussirlnf<tr L?.Lr. trace a few exa.tnples see why this works. 7.nytil.r. Sca.2 : {rz.5.dfa for it. b}* } is regular. " F A language .ckto its finir.in. } . Since we have txlrrstnrc:tc:drr dfa for the lauguage.rily .1 Dulnnl.rrgrrirgc regular. diagra.tza i . filr ttxl trorn(lrrt wtl will sirnply attach a rrarrreto this farnilv. Thc txrrrstructiorrof a dfa for this language is similn.qftfr f.!. Aga'in we is show tha.nningcontinrresin this wiry..t the dfh. Exolmple f . Aftcr orre or two to tests.6.gt: 1. We carr writc a. To show that this or any other la. we is can claim that. a.lstiltc.AccpprpR. Tlrl sohrtiorr is cotnplicated by the fact that there is no explir:it way of tr:stirrg tlrtl errd of the string.wcl rrt:cxlrr dfa that recognizestwo conseclttivc strings of essen'Ihe identica. : {utL.trNrsTrc F'tt'tt'l'Fl 43 in of the la.t the langrra.ges this family will bet:ome clearer irs our study proceeds.tonba. Slxrw that 1.5 Show that the lirngrrtr. I is Let tr be the language in flxa. Therrcftlrc. must do is check whethcr a striug begins and ends with au a.fi ffi $$lliRtl$ri tffi uil. dfa fbr it.
44
Chopter 2 FtuIrn Aurolvlare
Figure 2.6
\=.
tl/
I t Jt bJL r i f+_!4 " * ' A ' * t ' r A (fitl,r aa YJ aa 9F{ r?,F j * _.v  t
in Figure 2.6 can be used as a starting point, but the vertex gs has to be modified. This state cafl no longer be final since, at this point, we must start to look for a secondsubstring of the form awa. To recognize secthe ond substring, we replicate the states of the first part (with new names), with q3 as the beginningof the secondpart. Sincethe completestring can be broken into its constituent parts whereveraa occurs,we let the first occurrenceof two consecutive be the trigger that gets the automaton into a's its secondpart. We can do this by making d(gs,a) : qa. The complete solution is in Figure 2.7. This dfa accepts,L2,which is thereforeregular.a
l
v
tnlr
, ,q ^ w
,T.u.rp"o u r t f t  i l
={ tu f)J^
4111 61aujLft
ri,,
Figure 2.7
t?rfl'b,'
t L,ilat"
t
t r ,,1,86 4."
r\t
?.1 DETERMTNTSTTC Frrrrrn Accnprnns
46
The last examplesuggests conjecturethat if a language is regular, the "L so are .L2.,L3. We will seelater that this is indeedcorrect, ....
1. Which of the strings 0001,01001,0000110 are accepted by the dfa in Figure
2.r?
f)t* E = {a,b}, constructdfa's that acceptthe setsconsisting of
(a) all strings with exactly one a, (b) all strings with at least one a, (c) all strings with no more than three a's, ffi
(d) all strings with at least one a and exactly two b's. (e) all the strings with exactly two a's and more than two b's.
3 . Show that if we change Figure 2.6, making qs a nonfinal state and making
qo1qt1 qz final states, the resulting dfa accepts Z.
4 . Generalize the observation in the previous exercise, Specifically show that if nI
"(fr)
A* G"ive
(a) I=
M : (Q,8, d, go, F) and
= (Q,8, d, qo,Q  F) are two dfa's then T(6
:
dfa's for the languages
{abswba:tue{a,b}} ffi
(b) fI : {wpbwz:
tor € {a, bI* ,wz € {a,b}}
Give a set notation description of the language accepted by the automaton depicted in the following diagram, Can you think of a simple verbal characterization 6f the language?
b
I
(T
g
( z) fi"a
dfa's for the following languages 61 5 = {a, b}.
(a) I=
{ta: ltalmodS=0} ffi
(b) Z, = {tr : ltrl rnodSI 0} (c) I = {w : no(trr)mod3 1} > (d) 3 {w : n* (tu)moct > n6 (w)mod 3} W
46
Chopter 2
FtmIrn Aurouare
(e) r,:
nu(tu))mod:} >0} {ur I (n*(ur) (f) , : {u : ln* (w)  nr, ('u)l rnotl:l < 2}
Nole
tl^t
* 7 rna:/ :t" !
q$
e run in a string is a silbstrirrg of length at least two, as long as possible anrJ conrristing entirely of the satre symbol, For instance, the string a,bbhaa,lt contains a rul of b's of length three and a nrn of n's of lertgth two. Find dla's for the following languages on {4, h}. (u) t : {tr : ru cxrntairlrr IIo rtllls of length less than four} (b) .L : {'ur : every rtrrr rtf a's }ras lengt,ir either two or three} (c) I : {'ur : t}rere are at tnost two runs of a's of length three} (.1) f : {tl : there are exactlv two ruus of a's of length 3}
t.}tg
Consider the set of striflgs on {0, 1} delined hy thc rcquircments below. For each construct an acccptirrg dfa. (a) Every 00 is fbllowed irnrnediatel.y by a 1, For cxample, the strings 101, 0010, 0010011001 are in thc languagc, but 0001 and 0010t) are rrot. ffi (lr) all strirrgs containitrg 00 but not 000, (c) The lef'tmost' symbol diffcrs frorn thc rightrnost one. (d) Every substring of four symbols has at most two 0's. Ftrr exarnple, but 10010 is not since one 001110 arrd 011001 are in the latrg;uage, of its substrings, 0010, contains three zeros. {il (e) AII strings of length five or rnore in which the fourth syrrrbol lrom the right erxl is tliffererrt frorrr the leftrnost sytrbol' (f) All strings in which the leftrnost two syrnbols a,rrd the righttnost two syrnbols are iderrtit:al.
*10.
Corrstruct a clla that accepts strings on {0, 1} if and only if thc value of the string, intcrprctcd as a binary representation ofan integer, is zero morlulo five' lbr example,0101 and 1L11, representing the integers 5 and 15, respectivel.y, are to be acceptcd' 11. Show that the language 7 : {uwu : 'u,w E {o, b}* , lrl : 2} is rcgular. L2. Show that tr : {.a," : ",> 4} is regular. 13. Show that the language L: It^: rt.) 0'n I Sf]o* that the langua,geL : {a" : n,: i, I ula,r. ) is rcgular. ffi (1,1,2, "'} is regih,i,,k fixerl, j :
[iQ ."
15. Show that the set of all rcal numbcrs in C is a regular lauguage. L6. Show that if L is regular, so is Z  {I}. Qf Use (2.1) and (2,2) to show that
d " ( s ,t u u ): d " ( d " ( q , w ), u )
frrr all tr,u € E'
?.2 NorurErEFMrNlsTrc Flrulrl
Accnpr'trrs
47
1 8 . Let L be thc language ar:cepted lry thc autorrraton in Figure ?.2. Find a dfa that at:cepts L2.
1 9 , I,et .L be the langrrage acceptcd by t,he automaton in "Figure 2.2. Firxl a dfa
for tlre larrguage Lz  L,
2 0 . Let I, be the language in Example 2,5, Show that .L* is regular, 2L. Let G,r.rhe the transition graph for some dfa M, Prove the following,
(a) If f (M) is infirrite, then G,y must have at least one cycle lbr which there is a path fronl thc initial vertex to some vertcx in the cyclc and a path frorn some vertex in the cyr:le to some final vertex. (b) If , (M) is finite, then no such cycle exists. ffi
2'
Let rrs define an opcration trun,t:a,te, which removes the rightmost symbol lrorrr arry utring. l,br example, trurtt:a,te(aaliba) is aaab. The operation can be extenderl to languages bv :ate(L): {truncate (tu) : trr € I}.
Show how, given a dfa for any regular langrrage 1,, onc can construct a dfa for truntate (tr). Flonr this, prove that if L is a regular language not containing ,\, then truncaLe (.L) is also regular. L e t r : a o e , r , , ' a m t a: b o b r . . . b n r z : c o c l ' , , c " , b c b i n a r y n u r r l b e r s s a defined in Exarnple 1.17. Show that the set of strings of triplets
(il)(li) (r)
where the di, lt.i, c; are such that ;r * U : z is a regular languagc, 24. Wltile the language accepted by a given dfa is unique, there are normally man.y clfa's tirat accept a language. Find a dfa with exactly six states that accepts the sarne larigrrage as the dfa in Figure 2.4. m
N o n d e t e r m i n i s tFc n i t eA c c e p t e r s ii
Finite act:rlpters rnorecomplicatedif we allow them to act rrondetermina,re istically.Nondeterrninisnr a powerful,but at firs{Er.Sh is unusualidea..$,3 gormally thirrk of computersas contplettilydeterministic,and the elernt:rrt of chijiEedeeihs out of piritie. Nevertheless, rioridctermintsmis a useful notion, as we shall seeas wc prot:eed.
48
Chopter 2 .l'l.rrtr: Aurounre
Accepter of Definition o Nondeterministic
Norrdeterminism mea,nsa choice of moves for atr automaton. Ratlter than prescribing a,uniqrrt: rnttve itt each situtr.titlrr, we allow a set of possible tnoves. Formally, we a,chievcthis by delining the trarrsition function so thirt its ratrge is a set of possible states.
onfilWin
A nondeterministic finite accepter or nfa is definedbv thc quitrtuple F) M : (e,X, rl,r/0, , but fitrite accepters, as whereQ, E, {0, F are deiinecl for deterministic d : Q x ( E u { A } )  2 Q .
Note that there are three major difli:rerrces between this definitiott ancl the definitiorr of a dfa. In a, nondctcrrninistic accepter, the rarrge of d is in the powerset 2Q, so that its vtr,hreis ttot a single element of Q, but a subset of it. This subset defines the set of all possible states that can be reached try the transition. If, firr irrstance, the current stattr is q1, the symbol a is read, atrd ,5(qr, *) : {qo, qz} , therr either 8o or 8z could be the uext state of thtl rrfa' AIso, we allow .\ as the second irrgutnetrt of d. This mcirns that the nfa can mtr.kt: a transition without corrsutningau input symbol. Although we still assurnethat the input rrrechanismcan only travel to the right, it is possible that it is statiorrary on some Inove$. Finrr,lly, in an nfa, the set ,) (ql, *) ttray be empty, mearrirrg that there is no trarrsition defined for this specific situation' Like dfa's, nondeterrnirristic accepters can btl representedby transititlrr graphs. Ihe vertices are rlt:tertnined by Q, while arr edge (q1,qr) with labcl a is in the graph if arrd otily if d(qi,a) contairrs {i. Note that sintxr n rllay be the empty string, there can be some edges labeled .\' A string is accepted lry au ltfa if thcrc is sone sequenceof possitrle rrroves that will put tlte machiue in a,firral state at the end of thc string. A string is rejcctecl (that is, not accepted) only if there is no possible sequenceof moves by which a firral state c:arrbe reached. Nondetcrrnitrism can therefortl be viewed as involving "irrtuitive" insight by which the best move ctrn be chosetr at every sta,te (assurning that the nfa warrts to accept everv strirrg)'
2.2 NoNnnTERMINIS'r'tc t'rrurlr:AccnrrnRs
49
Figure 2.E
i.WWMorrsidertItetrarrsititl'gpr'inFigure2.8.Itdescribesarrtrrrrltttttrministic accepter sirrcc tlx)rc irre two transitionslabeleda out of gg. I
WtrorrcIeterrrrirristit:a,rrtornatonisshowninFigure2'9.Itisrrorrd(lterministic not otily bcc:au$c several edgeswith the same label origirratr: from one vertex, but also bct:iurser has a Atransition. Some trarrsitiorr, suc:has,i(q2,0) are it unspecified irr tIrc graph. This is to be interpreted as a transition to the empty set, that is, d(g2,0) : g. The automaton acceptsstritrgs .\, 1010, and 101010,but not 110 and 10100. Notc tha,t for 1.0there are two alternative walks, one leading to qe, tlte other to q2. Even though q2 is not a final sta,te, the string is acccpted hs:ause one walk leads to a final stattl. Again, the trarsiticlrr firrrr:tion can he extended so its second argurnrlnt is a string. We require of tlrt: t:xtended transition function d* that if d. (q1, : Qr, tu) then Qi is the set of rrll possible sta,testhe automaton may be in, having uttr.rtedin state {1 and }ravittg rearl rir. A recursive definition of d*, analogous to (2.1) a,nd (2.2), is possible, but rrot particularly enlightening. A rnore easily appreciated defirrition t:an be made through transition graphs. I
Figure 2.9
'r^Y
0
Y,_
1'=l*^7
l.
\
0,r
Iz
50
Chopter 2
FINITE Aurolvrrrrn
Definilion
?.5
Fbr an nfa, tlre extendctl tratrsitiou function is dt:firred so that d. (qr,r) contains 4i if rr,nrlorrly if there is a walk in thc: transition graph fiom q, to e qy lir,bclt:d'u. This irolcls for all r7n,t1, Q and ru e E*.
Figure 2. l0 reprtlserrts att nfa. It has severtr,lAtrarrsitiotrs aud some undeflned tra,nsitirlrrssuch as d (q2,a). 'fhere is a walk labelod wc Sr,4rpost: wattt to find d* (qr, a) nrxl d* (,1r,.\). n, involvirrg two .\transitions from q1 to itself. tsy using some of the Atrdgtls arrd 92. twicur, wr: see that there are also walks irrvolvitrg .\transitions to q11 Thus , d * ( , 1 t a ) : { g o 'q r i q z } Sincethere is rr .\edgebetween {2 alrd q0rwe have irnrnediately that d. (q2,A) contains gs. Also, since any state can be reat:hetlfrom itself by making no move, a,ndrxlrrscquentlyusing no input symtrcll,tl* (q2,.\) also contains q2. There,ftlrtl 6* (qr, A) : {So,q'i} . Using as mir,ny .\tratrsitions as needed, you (rirrl also check that d * ( q r ,a a ) : { q o ,Q t , Q z I
I
The definition of d* through labeled wirlks is somewhat informal, so it is rrscful to look at it a little more t:loscly. Definition 2.5 is proper, sincc' between any vertices ui and r.r; there is either a walk labeled tu or there is not, inclicating that d* is cottrpletely defined. What is perltaps a little harder to scc is that this definition urn irlways be used to flnd d. (gi,ru). Itr Section 1,1, we descrihcd art algorithm for finding all sirnple paths bctween two vertices. We crr,nrrotuse this algorithm directly flirrce, as Exilrnple 2.9 shows, a labeled walk is not always a simple path. We catt ttrodify the sirnple path algorithrn, rernovirrg the restriction that no vettex or edge
Figurc ?.10
2.2 NoNDHTERMrNrsrrc FrrrrrnAccEprERs
51
can be repeated. The new algorithrn will now generate successivelyall walks of length one, length two, length tlrree, arrd so on. There is still a difliculty. Given a ru, how lorrg can a walk labeled trr be? This is not immediately obvious. In Exa"rrrple 2.9, the walk labeled a between {1 antl q2 has length four. Ihe problem i$ caused by the .\transitions, which lengthen the walk but do not corrtritnrte to the label. The situation is saved by this obscrvation: If between two vertices r.',.and ui thercl is rr.ny wa,lk labeled ,ar, t,herr thr:rc mrrst be sorne walk labeled u.' of lerrgth rro more than A + (1+ A) lrl, whcrc A is the number of .\edgcs in tlte grir.ph. The a,rgurnent for this is: While Aedges may be repeated, there is always a walk in which every repeatexl Aedge is separated by arr edge labeled with rr nonempty symbol. Otherwise, thc walk contains a cycle labeled .\, whir:h can be replaced by a sirnple path without changing the Iabel of the walk. We leave a fbrmal proof of tlfs clairn ir,$an exercise. With this observation, we have a,urethod for computing d* (q,;,ru). We elvalua,teall walks of lerrgth at trxrst A + (1 + A) ltul originating at tr,;. We select fiom them those that are labeled zr. The terminating vertices of the selectedwalks are the elements of the set 6* (qi,,ut). As we have rt:rnarked, it is possible to define d* irr ir, rercursivefashion as was done for tlte tleterministic case. The result is urrfcrrturrtr,telynot very transparent, arrd arguments with the extended transitiorr fun<:tion defined this way are hard to follow. We prefer to use the rnore irrtuitivc and more manageablealternative irr Definition ?.5. As for dfa's, the larrguagc acr:cpted by an nfa is definecl forrnally by the cxtended transition function.
lrR,Ffin',tllllellrii ,ti
The language L accepted by a,rrnfa, M : (Q,X,d,qo,F) is defined as the set of all strings accepted in the abovtl scrrsel.Fcrrma,lly, L ( M ) : { r i r € X * : d * ( q o , w )n I ' I n } . Irr words, the language consists of all strirrgs ur fbr which there is a walk labeled 'u I'rom the initia,l vertex of the transitiorr graph to some final vertex.
,
i
t
;
Exumple 2.10
What is the latrguage accepttxl by the a,utomaton in Figure 2.9? It is car,sy to see from the graph that tlrtl orrly way the nfa can stop in a final state is if the input is either a repetition of the string 10 or the empty strirrg. Theref'orethe automaton accepts tlr: larrgrrer,ge = {(10)"'; n > 0}. I
62
Chopter 2 FINITEAurouil,rt
What happens wherr this automaton is presented with the string tu 110? After reading the prefix 11, the automaton linds itself in state q2, with the transition d (q2,0) undefined. We call such a,situatiotr a dead configuration, and we r:an visualize it as the automaton sirnply stopping without further action. But we must always keep in mind that such visualizations are imprecise and carry with them some darrger of misinterpretation. What wc carr say precisely is that ) d * ( q 0 , 1 1 0= o . Thus, no final state can be rttached by processingu,  110, arxl hence the string is not accepted.
I
Why Nondeterminism?
we In reasorritrg about nondeterministic mat:hirres, should be quite cautious in using irrtuitive notions. Intuition c:an tlasily lead us astray, and we mrrst be able to give precise arguments to substarrtiatc our conclusions. Nondeterrrrinisrn is a difficult concept. Digital conrputers are completely deterministic; tlteir state at anv time is uniqucly predictable from the input and the initial state, Thus it is natural to ask why we study nondeterttftristic rnachifies at all. We are trying to rrrodel real systems, so why includc such nonrnechartical features as choice? We ca,n an$wer this question in various wiiys. Many deterministic algorithrns rcrluire that one make a choice at $omc) stage. A typical example is a gameplarying progrant. Flequently, the best move is not known, but ca,nbe f'rrrrndusirrg arr exhaustive search with backtracking. When several tr,ltt:rnatives are possible, we choose one arrd follow it until it becomes clear whcther or not it was best. If not, w€l retreat to the last decision point and explore the othtrr <:hoices.A notrdeterministic algorithur that can tnake the best choice would bc able to solve the problern withoul backtra,cking,brit a,dtltcrrtritristicolre can simulate nondeltcrmirristtt calt serve with some extra wrlrk. F<rrthis reason. noncleterministir:rnat:hirrrls rrr morlt:ls of searchandbacktracka,lgorithms. Nondctcrrnirristn is sometimes helpfirl in solvirrg probletns easily. Look at the rrfir itr F'igure 2.8. It is clear that thertt is a cltoice to be made. The first alternaLive leads to the acceptarr:t:of tlte string ail, while the seconrl accepts all strings with an even mrmhnr of a's. The Ianguage accepted bv the nfa is {a3} u {ot" : n > 1}. While it is possible to find a dfa fcrt this Ianguage, the nondeterminisrn is quite natural. The language is the urriotr of two quite difftrrcrrt sets, and the uondeterminism lets us decide at the olrtset whir:h case we want. The deterministic sohrtion is not as obviouslv
2.2 NounptrrRMtNrsTrc FrnrrrnAccnlrnns
5S
rclated to the definition. As we go orr, we will seeother and more convincing exarnplt:sof the rmefulness nondeterrnirrisrn. of In the sarrrcvein, nondeterminism is an effective rrrcr:hani$mfor describing some cornplicated ltr,ngua,ges concisely. Notice that tlrc definition of a gralrlrrlar invtllves a nondeterministic element. Irr ,9 a,9bl.\
we can at any point chooseeither tlrc first or the second production. This It:tu rrs specify many different strittgs usirrg only two rules. Firrally, therreis a technica,lreason for irrtroducirrg rrondctcrminism. As we will see, tltlrtirirr results a,re more easily established for rrfats thtr,n for dfats. Our rrext maior resrilt indica,testhat there is rro essential diffcrcnce betweetr tlrt:sc two types of automata. Consequently, allowing rron(lcterminism ofterr sirrrplifies f'rrrmrr,l arguments without affecting the gerreralitv of the conc:lusiorr.
l.
Prove in detail the claim made in the previous section that il in a trarrsitiorr graph there is a walk labelerl rl, there must be some walk labeled tu of length rro rrrore tharr A + (1 + A) l,rrrl. Fitrd a dfa that at:r:eptsthe langua,gedefined by thc nfa'in Figure 2.8. I n F i g u r e 2 . 9 , f i n d d * ( q 6 ,1 0 1 1 )a n d d * ( g 1 , 0 1 ) . ffi
, 3.
4 . In Figure 2.10, Iincl d (qo,a) and d* (r;r,l)
5 . Fbr the nfa,in Figurc ?.9, find d (qo,1010) and d* (t71,00).
6
O
Design at nfa with rto rnore than five states for thc sct {abab" ; rr.} 0} U {a,ha'o:rr,>0}. C.rrr"t.,rct an nfa with three statcs that accepts the language {tr,b,abc}. W
8 . Do yorr think Exercise 7 can be solvccl with fewer than three states'l ffi 9 . (a) Firrrl an nfa with three states that acccpts the language L : {a" : rz > 1} u {I,*aA' : rrr } fi, fr;> n ) ' t
(b) Do you think the larrgrragein pa,rt (a) can bc a,cccptcd lry an nfa with fcwcr than three states'/ \lpt' @ l,'ind an nfa with lbur states lbr L : {a" : rr > 0} U {h"u.: n } I}. Wtli"tr of thc strings 00, 01001, 10010, 000, 0000 are arceptetl by the following rrfa?
54
Chopter ? Flt{trr Aurolrere
0
/^\f \?o/
\__.\
12. What is the complement of the language accepted by the nfa in Figure 2,10? 13. Let .L be the language accepted by the nfa in Figure 2.8. Find an nfa that accepts I u {a5}. 14. Give a simple description of the Ianguage in Exercise 1.2. (rs.)fina an nfa that accepts {a}* and is such that if in its transition graph a single edge is removed (without any other changes), the resulting automaton accepts i") W 16. Can Exercise 15 be solved using a dfa? If so, give the solution; if not, give convincing arguments for your conclusion. 17. Consider the following modification of Definition 2,6. An nfa with multiple initial states is defined by the quintuple M :(Q,E,d,Qo,F),
where 8o C Q is a set of possible initial states. The language accepted by such an automaton is defined as L (M) : {tr.' : d* (q6,trr) contains gy, for any q0 € Qo,St € F} . Show that for every nfa with multiple initial states there exists an nfa with a single initial state that accepts the same language. Nft 18. Suppose that in Exercise 17 we made the restriction Qo fl F : fi. Would this affect the conclusion? ( rg./Use Definition 2.5 to show that for any nfa
d (q,uu) :
U
pE6* (tt,u)
d* (p,r) ,
for all q € Q and all trr,u € D*, 2o. An nfa in which (a) there are no trtrartsitions, and (b) for all g e I and all a e E, d (q, a) contains at most one element, is sometimes called an incomplete dfa. This is reasorrable sirrce the conditions make it such that there is never arry choice of moves.'
2.3 EQUIvaIENCE DptpRturt'ttsrrcANDNoNnnrnnN.rrNrsrrc oF FInrIre AccnRrnns
DD
For E : {a, b}, convert the incomplete dfa below into a standard dfa.
W
E q u i v o l e n co f D e t e r m i n i s t ocn d e i N o n d e t e r m i n i s t iF i n i t eA c c e p t e r s c
We now come to a fundamental question. In what sense are dfats and nfa's differerrt? Obviously, there is a difference irr their definition, but this does not imply that there is any essential distinction between them. To explore this question, we introduce the notion of equivalence between automata.
lIM
Two finite accepters.Ml and M2 are said to be equivalentif L(Mr): L(M2,, that is, if they both acceptthe samelanguage.
As mentioned, there are generally many accepters for a given language, so eny dfa or nfa has many equivalent accepters.
.11 is equivalent to the nfa in Figure 2.9 since they both accept the language {(10)" : n, } 0}.
Figttre 2.11
Convert the nfa in Figure 2. after reading ru. Therefore the corresponding dfa must have a state labeled {gr. therefore d ({sn}. Conseqrrently. it is clear that any language that is accepted by a dfa is also accepted by some nfa.. By rnore powerful we mean that an automaton of one kind can a<:hicvesomething that cannot be done by any rrutomaton of the other kintl.nite number of states. this state in the dfa mrrst be a nonfinal trap state.Qjr. After arr rrfa has read a string ?u. so it is at least corrceivable that there is ir larrguage accepted by some nfa for which we cannot find a dfa. Since for a set of lQl states there are exa. the corresponding dfa will have a fi. This me&ns that we can actually give a way of corrverting any nfa into an equivalent dfa. the nfa c:an be itt state q1 or. .qz}. But the converse is not so obvious. Since a dfa is in essencea restricted kind of nfa. will be constructive. A state labeled g represents an impossible move ftrr the nfa and... We htrve added nondeterminism... The argument.QnI be in orre state of a set of possibltr dfa after reading the same string rrrust be in some definite state. by making a . saY {qz.56 Chopter 2 Frrurrp Auronara When we colrlpare different classesof automata.t:tly2lQl subsets. But it turrrs out that this is not so. in state q2.Ui.. The corrstruction is not hard to understand. The rationale for the construction is the following. tlre equivalentdfa will be in a sitrglestate labeled {Uo.\transition..12 to arr equivalent dfa. therefore. Before getting to the formal description of this. the question invariably ariscs whether one class is more powerful than the other. the tfa has no specified transitiort wltetr the input is b. The nfa starts in state gs.An equivalent states. Most of the work in this suggested construction Iies in the analysis of the nfa to get the correspondence betrveen possible states and inputs. qz} and a transition d ( { s n }.inly lms to be demonstrated. like most argurnents in this book. so the initial state of the dfa will be labeled {qs}. Let us look at this cluestiott for finite accepters. The classesof dfa's and nfa's are equally powerful: For every language accepted by sorrrerfa there is a dfa that accepts the same language. a ) : {qt. but we can say that it must . How can we make these two situations correspond? The answer is a nice trick: Iabel the states of the dfa with a set of states in such a way that. we may not know exactly what state it will be irr. After reading an a.b) : @. Iet us illustrate it with a simple example. This result is ttot obvious and certa.qa}. once the principle is clear it becomes the starting point for a rigorous argument. In state q0. means nonacceptance of the string.
1S .q r } .12 accepts any string for which d* (q0. every state has all transitions defined. it can go to q1. from 91 the nfa can make a . d ( { q t .13. I Figure 2.3 Eeurvar.uNrsrrc oF ANDNoFrnErrgRMINrsTIc FrNrrn AccBnrens 57 Figure 2. For the corresponding dfa to accept every such w7 araystate whose label includes 91 must be made a final state. a ) : Similarly.ru) contains 91. qz}. is a dfa. At this point.12 We have now introduced into the dfa the state {h. The nfa in Figure 2.2. q z }. shown in Figure 2. for the same input. equivalent to the nfa with which we started. then there is no specified transition. If. { g r . Remember that this state of the dfa corresponds to two possible states of the nfa. so we must refer back to the nfa. If the nfa is in state Q1and reads an a. F\rthermore. The result. so we need to find the transitions out of this state. q z }. b ) : {qo} . Therefore d ( { q t . the nfa is in state q2.\transition to q2.ENCE DnrnRrr.
Q*. To understand the constructitln? relllefiIber that Gp has to have certain properties. Then there exists a deterministic firrite accepter Mn : (8o. Fp) such that L: L (Mp). Thus the inductive iutsumption . dry. . so that the loop eventually stops.tIp. Therr form the uniotr of aII these d*. {t} of Gn that has no outgoing edge fbr sorne a€8.. C o m p u t e d * ( g 1 . Bui by construction.4t} to tqt.{'} with a. By the irrductive assurrrption.Fry)... we use the procedurc nf atotlfa below to construct the trarsition graph Gp for X. in Gp there will be a walk labeleclu from {qo} to Qa.. '. Identify this vertex as the initial vertex. Each pass through the lXl Ioop in Step 2 adds arredge to Gn' But Gp has at mo$ 218'^/l edgcs. each labeled with a different element of E. E.{'} exist. create a graph Gn with vertex {qs}. we argue by induction on the length of the input strirrg' Assume that for every u of length less than or equal to n."'.. There must then be a walk labeled u frorn qo to Q. Repeat the following steps until IIo lrlore edges are missing' Take any vertex qo. some of the edgesmay be missing.q*. Add to Gp alr edge from {qr. there will be an edge from Qa to sone state whose label contains ql. During procedUre continues construction. 4. .l.'.qj.L be the language accepted by a nondeterministic finite accepter M1" : (8r.'}.Consider rrow any u : ua and look at a walk in G1r labeled r... . a ). d. a ) .q*.qo. o . yielding the set IU. To show that the constru<:tiorr also gives the correct aIISwer. {qo} .. ."'. If MN accepts ). d * ( q * .gi. the presence in G1y of a walk labeled u from qo to qi implies that in Gn there is a walk labeled tr from tsol to a state 8l : {..Chopter 2 FIruIrn Aurovete Let . Every state of GIr whose label containu a. E. but the until they are all there. and label it 3.iand an edge (or a sequence of edges) labeled a liom qt to qt. 2.rrfrom qo to q..ny qy € Flv is identified as a final vertex.. Every vertex must have exactly the lxl outgoing edges. Proof: Given M1y.. procedure: nfato dfa L..QnI' if it does not already Create a vertex for Gn labeled {qt. the vertex {So} i" Gsr is also made a final vertex' It is clear that this procedure always terminates. d * ( q j .qi.
q. (sr. : I .lthough correct.0) {qo. Since there a.14 . dfu 1)u (ti/(q'. we introduce the state {qo. .2. T l r c r r .trr) contains a firml state 91.g'. tr.s r } .14intoatlequivalentdetennitristictrraclrine. t) : {qt} gives us Lhe trew state {q1} arrtl an txlgc labclud 1 between it and {(ro}.16. Lnt uu rlrl rr.l : I.tu) contains g/.(q0. (so.2. We will fbllow this practice in the rest of the book.we continue until we obtain the complete sohrtion in Figure 2. it is true for all n.rr.HN(rE DnrnnMlr'usrrc AND Nolotrpn. ri : 0. e r .sr}. In the $irrnewity. so tl(xruthe label of di(gs.'{qo. The construction in the above proof is tedious but irnportarrt.tt). To complete the proof.qt}.n . I Figure 2. so rrrust dh (qo. : Sittcc t)1. 0 ) : { { 0 .qr} in Gt and add an edge labeled 0 between {q6} and {. we have the partially constructed autornaton shown in Figure 2. are admittedly somewhat terse.ncltherexa. r Ihe argumerrts irr this proof. k = 2..re now a number of missing edges. emphasizing the basic ideas irt ir proof arrd omitting minor deta. we compute di' (Sn'0) U div (qr' 0) : {.qz}.q. sr] makes it necessaryto introduce yet another state {q1.r r s i n ga . J : 1. 2. rnnsidering dlu (qo.mple to make sure we understand all the steps.ll EeuIv.t. The result then is that whenever dir(So.ez} and th. Q z I. i : 0. using the (:orrutructionof Theorem ?. which you may want to fill in yourself. showing only tlte ntajor steps.lo.restill some missing edges.It.ils. so we continue. With o : 0. At t}ris point.rrNrsTr(l !rNrrp AflfletrnRs oF 59 lnlds fbr all strings of length rz * 1 As it is otrviously trrrt: for z : 1. This givus us thc new state.1)u dfu 1): tsr.ru). There a.etransition d n ( { q o . we reverse the argrrtrent to show that if the label of di(qs.qzI .15.
15 (lrl r./ . Autottl'rn Figrue ?./ { . Figure 2.16 @ v'l\ lrrJ) @ 't\ V. . ___./') \.Chopter 2 Frutre.
tXZ/\ p. Carefully cornplete the arguments in the proof of Theorcm 2.. Section 2. / ..l.\transitions and with a single final state that accepts I.o"* ttrat for every nfa with an arbitrary numbcr of final rtfates there is an eqrrivalent nfa with only one final state. then dfr (So. but with fcwer statreB. Use this to find another dfa.ur)rr(Q .2 into an cquivalent rlfa.F) # s)'l If . Give a sirnplc verbal descriptiot of thc la.2. . II not' give a equal to thc set {u e X* : d* (qo. . Show irr tletail that if the label of di. Can you see a sinrpler an$wer rrrore directly? 2. 1.u) nF: J counterexample. the complement of I (M) is F) ft? fr it trr. L0.2.d. Define a dta with multiple irritial states in an analogorrsway to thc corresponding nfa in Flxercise 17.(q6..{o. d. f)oes there always exist au equivalerrt clfa with a single initial state'l Provc that all finite languagcs are regrtlar.3 EeUIVALIINCEoF DErnRuIuIsuc AND Nounu'rpRltlNISTIC FINITE AccEP'I'ERS 61 One important conclusiolr we can draw from Theorem 2'2 is that every Ianguage accepted by an nfa is regular. erpivaletrt to thc given one. Convert the nla in Exercise 11.. prove it.d" (qo.')also corrtairrs q1' it true that ftrr arqynfa M : (Q.tr.x. \0 4.F) the cornplcment of I(M) is 4 (DIs * @}? If Bo. Sectiorr 2.is regular' so is IE' 13. r7o. if not' give a r:outtterexantple. ffi /ftor'rve.t the ftrllowing nfa into an equivalerrt tlfa.nguagear:r:eptedby thc dfa in Figure 2.2.so.re that ftrr every nfa M * f eQual to the set {tr e E. ffi Show that if /. ffi 9J Let Z be a regular langrrage that does not contain. Use the construction of 1lheorem 2. Can we make a siurilar claim for clfh's'l ffi {a tl t i" + @) j #) Firrd an nfa without .2 to convert thc nfa in Figrrre 2. > 1}.16. '? : (Q. Show that thcre exists arr rrfa without .ur) contairrs 97.prove it.10 to a dfa. X.\trarrsitirrns and with a single firral state that accepts thc sct {a} u {b"' : n.
wlanguage chopL (l) 'efrmosr svmbors :. The secorrdeurtomaton cotnbines these two options. The states rcachable subsequent to the Iirst move d(S0... Frtrm a Ianguage . ffi R e d u c t i o n f t h e N u m b e ro f $ t o t e si n o Finite utomofo* A Any dfa defines a uniqrte la. there are rnany dfa's that accept it. the first tlrttomatotr has sollre redlrndant parts. then e .L be any langrrage. But even aftcr the removal of q5.nguage.1).17(a) and 2. The two dfa's depictcrl in Figure 2. 15. Corresponding to this.. all solutiorrs arc equally satisfactory. f)efine euen (trr) as the string obtained by extracting from tu the letters irt evenmrmbe. Such a statt: is inaccessible.and it carr btr removed (along with tr.'Ti . :TjTl'... .L is regular then cft.L we create a rre.arm {i+\  f.re to be applicd in a practical settitrg.red positions. We notice sottre obviottsly unnecessary fcatrrres of Figure 2.17(a). In terms of the questions we have corrsideredso far.. tht:re may be reasolls fcrr preferring one over antlther."t . The state q5 plays absolrrtelv no role in the autornrrton since it can nevet tlt: retr. we can definc a language eaen(L): l e u e r t( u .0) rrrirror those reachable frotn a first move d (So. but if the results a.but the colrvortteis not true. t t e n\ w ) : tt'2a4. (tr) Show that if .a2o"3a 4..ll transitions relalilg to it) without affec:tingthe lrrrrguageaccepted by the irrrtomaton. that is. if w : at. so is euen (I). ffiL.op2 is also regulat.Chopter 2 FIuIrn Aurov.. Thcre may be a considerable diflerence in the numlrer of states of such equivalertt inrtomata.. ): ' r r re I ) . ffi by renxrvirtg the two Provc that if Z is regular.chedfrom the initial state q0. I .17(b) are equivalent' a$ ir fcw test strings will ctuickly reveal. For a givcrr language.
rii0it iltlLfi i if Two states p and q of a dfa are called indistinguishable .tes as far as possible.4 RpnucuoN oF rnn NuunnR on Smrus IN FINITEAurouare 63 Figure 2. If.r) € F and 6* (q.u.1? @ From a strictly theoretica. ro) € f'. the second alternative is clearly preferable.2. We now describe an algorithm that accomplishes this. by a .5*(p. lbr all tu € E*.u) f F. on the othcr harrd. r) € F implies d. there is little reason for preferrirrg the automaton in Figure 2. For storage efficiency.l point of view. However. there exists some string u e E* such that d* (p.ut)f F. Representation of an automatorr for the purpose of computation requires space proportional to the number of states. then the states p and g are said to be distinguishable slrrng ?{). and d* (p. or vice vcrsa. (9.') f f' irnplies 6* (rt.17(b) over that in Figure ?.17(a). in terms of simplicitv. it is desirable to reduce the number of stir.
lWe usethis first to showthat at the completionof the nth passthrough the loop in step 3.Chopter 2 Furtre Auroneta Clearly. all states distinguishableby strings of length rz or lesshave beenmarked. mark the pair (p. 6. We now a. The only claim that requires elaboration is that the procedure fiuds all distinguishable pairs.6) for somea € X. Remove all inaccessible states. and all three states are indistinguishable.\. the procedure terminates. This can be done by enumerating all simple paths of the graph of the dfa starting at the initial state' Any state not part of sonte path is inaccessible. E.q0. terminates and determines all pairs of distinguishable states. c o m p u t e5 ( : p . with q6 and g1distinguishableby a string of length n .eo) is nmrked as distinguishable. then fro are p and r. since there are only a finite number of pairs that can be marked. q ) a n d a l l a e X . two states are either indistinguishable or distinguishable.o) : qu (2. F o r a l l p a i r s ( p .q) as distinguishable. Proof: Obviously. Repeat the following step until no previously unma. if and only if there are transitions : 6 (qr. a ) : eo.and qj are distinguishable with a string of length ru. applied to any dfa M : (8. so we have that the claim is trrte a basiswith rz : 0 for an induction.a) qn and 6 (qi. q) as distinguishable. procedure: mark 1.ssume . We claim that this procedure constitutes an algorithm for marking all disiinguishable pairs.rked pairs are marked. It is also easy to see that the states of any pair so marked are distinguishable.5) (2.. 3. One method for reducing the states of a dfa is ba^sedon finding and combining indistinguishable states. If the pair (po.q). In step 2. mark (p. o ) : p o a n d 6 ( q .. If p e F and q fr F or vice versa. 2. we mark all pairs indistinguishableby . The procedure nlurlr.F). Consider all pairs of states (p. Note first that states q. Indistinguishability has the propertie$ of an equivalence relations: if p and q are indistinguishable and if q and r are also indistirrguishable. We first describe a mothod for finding pairs of distinguishable states.
. we can claim that..j ..qj. we construct a reduceddfa as follows.. But if there are no states distinguishable onlv by strings of length n.E.add to d a rule TU. {qr. and that any two elements from different subsets are distinguishable. Qj. By this inductive assumption. 1. 3. find the setsto which q' Ttd qo belong..Qnl1..at the completion of the nth pass..5) and (2.. at the beginning of the nth pass through the loop. To show that this procedure marks all distinguishable states.'.a):Im..q^.qr} and € Qp {qt. as describedabove' 2 . and so on.a) : qe. h for M. assume that the loop terminates afber rz pas$e$. there cannot be any states distingrrishable orrly by strings of length n * 1.. it then follows that there cannot be any states distirrguishable by a string of Iength rz. As a consequence. op Srarns ru Frlrrrn Aurotrlnrn oF 65 for all i : 0. {qr. Flom (2. enl of such irrdistinguishable states.. for any TL.6) above.i'"k. create a state labeled i.rQn}.Q*.6).Qr}.4 RbDUCT'roN rsp Nulrnnn.'.. that elements in each subset are indistinguishable.'..whett the loop terminates. etc'..F). Using the results sketched in Exercise 11 at the end of this section. say {ft. such that any (t E I occurs in exactly one of these subsets.. This means that during the nth pass no new states were marked. all states distinguishable by strings of length up to n will be marked.d..1n .q*r. Qj. . Use procedure mark to lind all pairs of distinguishable states. all distinguishable ptrirs have been marked. For each set {qa.. it can be shown that such a partitionirrg can always be found.q0.. all states distinguishable by strings of length up to rl .n. r After the marking algorithm has been executed. at the end of this pass.... find the sets of all indistinguishable states. we use the results to partition the state set Q of the dfa into disjoint subsets {qn.tQn}.'. .1 have been marked.1. Then from this. all pairs distinguishable by strings of length rz or less have been marked.. 1 . For each transition rule of M of ihe form 6 (q.ei. .. If q' € {U.8'}. procedurer Given reduce fr: (8.5) and (2. .. By induction then. but not distinguishable by any shorter string. Becauseof (2.2.. Flom these subsets we construct the minimal automaton by the next procedure.
qr) is also marked.Therefore.gt).1 ) : g r . F ir th* set of all the states whose label contains i such that ga € F. and (g3.19 .l (gr. Consider the automaton depicted in Figure 2. the procedure computes . the marking algorithm eventually marks the pairs (qo. the states q1. kh. Since (qs. (h. qz.q+). the procedrre mark will iclentify distinguishable pairs (qo. and all ofthe states have been partitioned into the sets {So}. (qo. 1) : q+ and d ( q u . (qo.18 Figure 2.18. o.g+). the pair (qo.qs} ancl {ga}.19.qz). (qr. qz).ga)' In some pass tlrrough the step 3 loop.(qr.q3) and (qr.qn) and (qs.qa)as distinguishable.g+) is a distinguishable pair. {qr. Applying steps 2 and 3 of the procedure reduce therr yields the dfa in Figure 2.qn).gt).66 Chopter 2 Fmrrn Aurorr.{2. leaving the indistinguishable pairs (q1.n3 &r€ all indistinguishable.m). I Figure 2. (qz. The initial state f11is that state of M whose label includes the 0.q+). (qo. In step 2. Continuing this way.rnra 4 .q+).
This contradiction proves that M' cannot exist.wk) . This is relatively easy and we can use inductive argument$ similar to those used in establishing the equivalence of dfa's and nfa's. Assume that there is art equivalent dfa M1.wn srtch that . . We will leave this as an exercise. and d* (qn.. there must be some strin&z such that d.. ) .such that and d i ( q o .. .ut). . there must be distinct strings 'trr. withps the initial state.. I . Sincepl andp. Proof: There are two parts.w n ) : d i ( s 0 .z) is a final state..r) : .. 1 . i : 1 .pz. . Thus. trrr) .r*) : d* (pt'z) is a nonfinal state (or vice versa).pt.li (qo. d " ( p o w t ): P t . Suppose M has states {ps.. equivalelt to M..l. (g. In other words. with transition function d1 and initial state gn. is harder.tupn): di (di (qo. The second part. M1 either acceg$ both tlrz and ur.Fm... rurr is accepted by M and rurr is not.Auroraarn 2.ur) is of the form. application of the procedure reduce yields another dfa fr suchthat L(M):"(fr) F\rthermore. M is minimal in the sense that there is no other dfa with a smaller number of states which also accepts L(M).. .4 RpnucuoN oF rHn NuNaenR Smrns IN FINTTE on 67 Given any dfa M. But since Mr has fewerstates than fr. But note that di (so. The first is to show that the dfa created by reduce is equivalent to the original dfa.Tr... are distinguishable.. (po. .r or rejects both. to show that M is minimal. upr) : 6* (pn. contradicting the assumption that M and M1 are equivalent. sa./?116 ur1.'trz. All we have to do is to show that d..): qi if arrd only if the label of F(ga. 2 . but with fewer states.j. there must be at least two of these strings. n) * dl (di (qo. Since there are no inaccessible states in M. m .
ngua. Prove or disprove the following conjecture.. . ffi and q. Show the explicit steps of thc suggested proof of the first part of Theorem 2. mrtst be distinguishablc.* that inrlistinguishability is an equivaletrcerelation but that rlistinguishabilitv is not.t. Write a cornputer pr()grarn xhat produce.E. is re. ** g.. q6. namely. (*) fI:{o"b"''n}Z.Z) Sl. for Z. If M : (Q. 8. Prove the fbllowirrg: If the states g* and qa are indistinguishable. X. arc distinguishable.l (d ltrrq below.du. .L has length at lea^strz.f') is a minima. that M is equivalent to the original dfa.d. then any dfa accepting L rnust have at least n. r l l 3 } ffi {a^:nt'2 andn+41. ffi f.such that any r. ffi f1/ Strow that if . therr r71. 6. In eachcaseprove that the rcsult minimal dfa's for the larrguages rs mlnlmal.. F) is a' minimal dfa for a regrrlar la.> 0 . Showthat the automatongenerated prot:etlure fr) Virri*i"e the states in thc dfa depicteclitr the ftrllowing diagram. / . f 1 states.16.tn'r'e of 1.l dfa.ga rninimal dfh for any given clfa..gc then M : (Q. by 3. Minirnizethe rturrrber statesin the dfa in Figure 2. 10.Q .ce deterministic.Chopter 2 FIullp Aulor. and if q..4.l in .L. arxl g.mlll (b) r : {a"h: n> 0} U {b'a : n } 1} (c) I: (d)r: : { o " o n .L is a norrernPty langrrage .go.
Then take another state. Then formalize this suggestion to make it an algorithm. not in the precedirrg equivalence set. and do the sarne thing' Repeat until there are no more states aw.' rxn Nuvel. to be done after the corrrpletion of the procedure rnar.4 RnnucrloN oI. Put all states rrot marked distinguishable from ge irrto an equivalerrce set with qcl.2. saY. Start with some state. {0. Consider the following process.R oF STATESIN F'INITE AUToMATA 69 11. and prove that this algorithm does indeed partition the original state set into equivalence sets. .b.ilable.
.
parentheses. and the operators *. we need more (roncise ways of desr:ribitrg regular languages. c}.. .pter for it. Slightly rrrore complicated is the language {a. b. 71 .R * g u l q rL q n g u q g e s ond Regulqr Grtt m mq rs ccording to our definition. Therefore. which will be denotetl by the regular expressiona. we Iook at other ways of representinpJregular languages. The simplest case is the language {a}. But in many instances. In this chapter. if wr: vrant to show the logic by which we decide if a given string is in a certairt language. for exarnple. a rnatter that is touched on in some of the examples and exercises. These representatiorrs have important practical applications. Such a description can be very useful. Expressions ffiM Regulor One way of describing regJular languages is via the notation of regular expressions. a Ianguage is regular if there exists a finite acce. and +. This notation involves a combination of strings of symbols from some alphabet E. for which. every regular language can be described by some dfa or some rrfa.
(c+o) is a regular expression.72 nruDReculAR GRAMMARS Chopter 3 Reculan LANGUAcE$ usirrgthe * to denote uttiott.rrdsfrrr the starclosure of {a} U {bc}. A string is a regular expression if and only if it can be derived from the primitive regular expressions by a firritc mrmber of applications of the rules in (2). I . This is similar to the way we con$truct f amiliar arithnteticerxpressions.\.nd12 are regular expressions.aa. of Expression Formol Definition o Regulor from prirnitive constituentsby repeatedly We construct regulrrrexpressions applying certain recursiverules. since there is no way it carr be constructed from the primitive rcgrrlar expressions.ii. For example. if we take 11 = c and rz : fr. {. (a+b'c)'. it$..abc. (a + b+) is not a regular expression. the language hco.rtrz.ril Let E he a. c)* sta. we hirve the regular expressiona+b+o We use ' for cont:eltenation aud + for starclosttre in a. and n € X are all regular expresrtions. #. Then l .These are called prirnitive regular expression$. Repeating this. If 11 a. ubr:. similar way. that is.1b.\. we eventually generate the whole string. The expression (o. a.firril.lr. and (r1). ri. hcbc.q.. 3 . since it is constructed by application of the above rules. bc. are rt*rz.rfltltil . }. On the other hand..g.. aaa. we find that c * fi and (c * o) a"re also regular expressions. so 2 ..given alphabet. a.
will let . b } . ( a + b ) ): L ( a * )L ( a + b ) : (r (a)). l. fi is a regular expression denoting the empty set. } . 6. $*qmpfq 5. (a + b)) in set notation.3.{ o .a a br . a a . . a a r o .rexpression.1 R. L (r1 r r2) .(a)u r. a is a regularexpre$$ion denoting{a}.b .L (rr) u L (rr). the first three are the termination conditions for this recursion. (o)) (r : { t r .If r is can a regula. e a r . a ) . 5 .a b . . To seewhat languagea given expression denotes.L (r) denote the languageassociated we with r.f. I((r1)): tr(rr). e . r r ) : L ( r 1 )L ( r 2 ) .we apply theserules repeatedly. a . then 4. L(rfi: (I(rr)). Exhibit the language (a* . . I . . This languageis definedas follows: The languagetr (r) denoted by any regular expre$sion is defined by the r following rules. The last four rules of this definition are usedto reduceI (r) to simpler componentsrecursively. L L ( o * .} : { a .\ is a regularexpre$sion denoting{A}. for everya € E. If 11 and rz are regulaxexpressions. . . L ( r r . e . 2. 3.nculan ExpRpssrorls 7I Longuoges Associoted Regulor with Expressions Regular expressions be used to describesomesimple languages. 7.
I . we can see quickly what language a particular regular expression denotes. that is L(r) : {a2"b2*+t : n } o. (a * b)*. B r r t t h e r e is nothing in Definition 3.b}. Instead. (a * bb) represents either an a or & double b.IaRS to*bK\"^"{ There is otte problem with rules (a) to (7) in Definition 3.I\.b denotes the set of aII strings with an even number of a's followed by arr odd number of b's. . : {ab.c\.ac}.3 For E: {a.Chopter 3 Ruculnn Laucuecns aNo RBctiLA. I n t h i s c a $ ew e f i n d t r ( a ' b + c ) : { a b . To overcomethis.} . L (r) is the set of all strings on {a.bbb. We now get a different result. GR. c } .aa. The first part. r L : & ' b a n d r z : c . * > 0}. The second part. so we can write r1r2 for rL . terminated by either an a or a bb.rz. With a little practi<re. but there may be some amhri.ba. Exomple3. Goirrg from an informal description or set notation to a regular expression tends to be a little harder.. L(ab*c) we could require that all expressiolrs be fully parenthesized.2. Consequently.2 to stop u$ from taking rt : a and 12 : b * c. Also. Wo can consider this as being made up of . but this gives * crrmbersome results.bb. stands for any strirrg of a's and b's. the expression r=(&+b)(a+bb) is regular. Consider. the symbol for concatenation may be omitted. the regular expression a ' b + c. we use a convention familiar from mathematics artd programrning languages.abb. It denotes the Ianguage L (r) : {a.Euity in breaking a complicated expression into parts.. They define a Ianguage precisely if 11 and r? are given.b}. We can seethis by consideringthe various parts of r. I The expression y : (aa)" (bb).n. We establish a set of precedence rules for evaluation in which starclosure precedes concatenation and concatenation precedes union. for example.
5. D r H*Wnrple 3.st One catt arrive at alt arrswer by relirsclrrirrg sorntlthing likc this: Evcry utring in I (r) must conlaitr 00 somewhere.cur. as they denote the sarne language.ge.ccollntedf'or. Notc that this la.1 Rl.Although tlrc two erxpressions krok rliflilront. After takirrg care of these special caseswe arrivc at thc arrswcr s a H_Li. I The last example introduces the notion of equivalence of regula.r cxllrcssiorrs ar(l rrot verv sirnililr irrrrl do ttot suggest clearly l. Even though this looks similar to Exanple 3.rnlimited nunrber:of regula.rexpression(IilI I+)".ge.n derive a variety of rules fbr simplifvirrg rtrgrrlirr .r expression suchthat r L(r): one pair of consequtive zeros}.t thr: irrrswcr irrvcllvr:stlrrl repetition of strings of the form l .'guage 1 : {tu € {0. : rl }rirsrro llair of rxrnser:rrtivo zeros} . I r the la.ly by atr arbitrary nutrrber of 1's. However. we arrive at the solution r .he close relationship between the languages.nn FlxpREsSroN$ . An arbitrary strirrg orr {0.rexlrressionsfor any given langua. that is. Howtlvtlr. the a. Putting these observations together.ngua.^ uctr subsrrirffi )7 : ( r * 0 1 1 *( 0+ A )+ 1 ( 0+ A ) .nswer still incornplete.. If we reasorr sliglrtly differently. One helpful observation is that whenever a 0 occurs. 1}. the shorter expression ?':(1+01)(0+I) rniglrt llu rua. it rnust be lg[g*gg"gi.ts. Generally.5. 101 .rrswcrs are corl'ect. {rrr e X* :ur ha.re una. givea. ' 1.t lea.: ( 0+ l ) .t:Ircrl. . we rniglrt corrrc up with arrother arrswer.rrguagcis tlrc txlrnlllcmt:nt of thc languagc irr Exarnple 3. 0 0( 0+ l ) .S For X = {0. 1} carr bc dr:rrotrxl by (0 + 1). 1}.' .regula. the answer is harder to construct. tlrc rc:gtrlir. One ca. This suggcsts thrr..We say the two regular expressionsare equivalent if they denote lhe same la. If we see .r expressions.sa.L as the repetition of the strings 1 arrd 01. is sirrcrrthe utringu ending in 0 or c:onsistingof all l's a. the language denoted by the regula. both ir. lhere are an r. but what rxlrnc:sllc:frrrr:irnrl what gorrs after is completely arbitrary.
) 8.rrrt. fO.I.4) Firxl a regular expressiotr fbr the set {a'b"' : ('rr*'rn) is even}.) tnir'ra regula. but since we this. ( c l all strings that contain at least ollc occurreltce of each symbol in r.m.ncuLeR GR..5'l ffi Show that r : (1 +01 ). L s . Firxl a regular expressiorrfor f : Iu E {0. L1.rr} 4. ancla# clcnote? do Wfrot languages thc cxprcssions Give / \ q a I sirlple / \ + \ verbal ). w e {a. strirrgs ..itritrg IIL) Illore than three a's. of @ 7.s ( d ) all str:ings that contain no run of a's of length greater than two. ffi (#.r. I} : ur has exactly one pair of consecutive zeros} .t:n. * (") all strings in whir:h all nrrrs of n's have lerrgths that are rrrultiples of three. ffi 12. \(n.rrl { i1}. ( " ) all exactly one a.} i}. < (r:) The corrrplelnent Ir.uacns aNn R.b}.. where . a. t'ind a.'ntg 3}. have littkr rreed for such rnanipulations we will not pursr"te 1 .lculan. ((t) Does the expressiorr + 1) (0 + 1)).D Ctu" rcgular expressiotrsfor the following languages on { o .ll strings in I ((a + t?)b (a * ab)) of lerrgth less tirart ftrur. Find a regular exptessiorrfrrr the cornpletnent of the language in Example 3. : {u.\}ts expresuirlrrs(st:t: Exclrt:isc 18 irr the following exercisesection). $ f fanb^:rtlL.lol : ?}. Find a regular expressioufor L: 13. jl.n'rn> (9i Cit* a rcgularcxprcssiotrorL: 3].). O (b) . L".b}+}. (0 11. .c } .+ LL \aLL ) Dil. Give regular e. .00 (0 + 1).) also denotes the langrragein Exarnple i1'6.'"b"' : Tr.xpressionsfor the following langrrages.N'lN{.denotc thc languagein F)xarnple 3.rexpressiol for I: {o.4' {uut : t.rirrirrg (l') all strings corrtir.L is the larrguage in Exert:ise 1. Give a regtrlar expression fo. (a) Ir : {1t"ll". of tlre larrguage L((au)r b(aa).76 Chopter 3 F. Lauc. 4.h"t. (.q. of (rt) The trrrnlrlernetrt L2. b .tu e {a. Find two other equivalent expressions.
In Exercise ?2. alrd (ruldr')'. S 22. b. give a set of necessary arrl sufficient couditions that r rrrust satisfy if L (r) is to be infinite. Formal languages carr be used to dcscribe a variety of twodimcnsional figures.). and (c) of Exercise 16. (a) (ri).: riri.EGULAR EXPRESSIONS Tf . L9.6 do indeed rlenote the specified larrguag. S (rl) all strings having at least two ot. (c) (rr *rr).IJ. 000 corrtains two such oct:trrrences). Draw pictures of the ligurcs denoted try the expressions (r'd). (e) all strirrgs with at most two occurrences of the substring 00. The svmbol : stands for e<luivalence of regular expressiotts itr the scuse that both expressitlttsdenotc the same language. * (f) all strings not corrtainitrg the substring 101. l}. Givc a general method by wirich auy regular expressiortr can be changed into frsuch that (I("))" : L(F). with 5 : {4. Write (a) all strings cnding in 01.: ri.. d. wirich stands for the squarc with sides of urrit length. arrd left. (c) all strings txrrttaining an even number of 0's. rcspectively. Repeat parts (a). 1}.\ or @.:curreltces of the substring 00 (Note that witlr the rrsrtal interpretation of a substring. (u) /:: {ru : lurlrrrodS= 0} {ffi (b) f : {tn:no(to)mod3:0} (c) L = {w : no(ur)mod5 > 0} 17. 16J Find regular expressions for the following larrguageson {4. (urrLdru. 20. Determine whether or not the following clairns are true for all regular expressions rr arxl rz. w}rere t}rese symbols starxl for unitlcngth straiglrt lines irr the dircctions up. 21. Chaincode languages are dcfinccl on the alphabet X = {u. r. = (rirfi).6}. (b) ri (rr * r:)* : (" + r'r)*. (b) all strings not entlirrg irr 01. what are sullicicnt conditions orr the exprcssion so that the picture is a r:losed cotrtour iu the sensethat the beginning and erxling point are the sarne? Are thcsc conditions also rrecessarv'l S . (b).e. 23. tlrrwn. For the case of a regular expression r that doelt rrot itrvolve . 18. Prove rigorously that the cxpressions in Exarnple 3.'1i) regular expressionsfor the ftrllowing languagcs on {0. S (d) (rrrr). right. c}. Arr exarnple o1 this notation is 'urdl.1 R.
Bet:arrseof the equivalence of nfats ir.1 way. M(rr) and M (rz) tha.ngua.78 Chopter 3 Rnculen Laucuncns . We rrow show that if we have arry rcgular expressiorrrr 'we (:irn construct an nfa that arx:eptsI (r). and fl. rcsptx:tively. a regular expression that rlenotes all bit strings whose value.ted . In Exercise 7.3 to 3.r. L Proof: We begirr with automata thal.'nted this in .t accept larrguagcsrlenoted by regular expressions 11 and 12.f(r1) and M (rr) rt:prcst. The constructiorr for this relies orr the recursive definition for I (r). Assrrme now that we havc irutomata. and (3) of Definition 3.nguage there is a regular exprcssion. a language is also regulirr if it is rr. and (7).cceptedby some nfa. Tlten thrlre exists some nondeterrrrirristir: finite accepter tlnt ir.r expressiorrso.nddfa's.rexpression. Our definition says that ir.a. (2).3 we claimed that frrr every nf'a there is an cqrrivalent one with a sirrgkl fina. the graph as vertex at the lcft representsthe initial state. is greater than or equal to 40. We will slmw this in two parts. Srxrtion 2.+run Rr:crJr. (5). Find a regular expression for all bit strings.2. then . then show how they can Lrecorrrbirx:dto implement the rnore complir:a. r1t s. Regulor Expressions Denole Regulor [onguoges We first show that if r is a relgular expression.m sr:henra. ConsequentlS I (r) is a regular language. With Jl. The constructions are showrr in Figures 3. In this schema.. and rf.lstate.\.5.ler.tedparts (4). C o n n e c t i o n e t w e e n e g u l o r x p r e s s i o nq n d B R E s Regulor onguoges L As the terrninologv srrggests.the connection betwct:rr r{rgrrlar languages and regular expressions is ir close one. TIte two concepts artl e$sentia. with values not bctween 10 and 30. We first construct simple automata for parts (1).. (b). Let r' bc ir regrtla.{rcepts (r). and (c). accept thc ltr. As irrrlit:a.L (r) is a rcgular la.ssuming that tlxrrr: is only one final statc.ngua.ge.(a+b)). in Figurtt3. with leading bit 1.nR GRAMMAR$ 6+)i"a an nfa that accepts the larrguage L(o. and fbr every regular expression there: is rr regular language. for every regular ler.lly the satne. so we lose rrothirrg in rr.ngua. thc one on the right the firral state. . We neecl not explicitly r:onstruct these autornata. when inCE)l'ina terpreted as a binary integer. respectivcly. € E. but may represerrttlx.1(a). interpreted as a binary irrteger.tically. ffi 26.2 on page 73.ges the sirnple for regula. wo then construct autorrrata frlr the regular expressiorrs  12. I'hese are shown in Figure 3.ge regular if it is accc:ptrrrl some is hy dfa.
It should be clear from the interpretation of the graphs in Figures 3. (c) nfa accepts {a}.2 Schematic representation of an nfa accepting Z (r). By stringing together several such steps. .\_. (e) 6) (c) Figure 3./. . we call build automata for arbitrary complex regular expres$ions.3 to 3. M(r) '/_\.5 that this cotrstruction works. where (ba. To argue more rigorously. then prove by induction on the number of operators that the construction yields an automaton that accepts the language dettoted by any particula.2 CoNNEcttot'r BnrwpnN RncuLaR ExpREssIONs Arrrn RuculaR Latqcuecns 79 Figure 3. t ) in the drawings.L (r).3 Automaton for L(rt * rz).1 (a) rrfa accepts g. (b) nfa accepts {tr}. I . .r regular expression' We will not belabor this point. we can give a formal method for constructing the states and traruitions of the combined machine from the states and transitions of the Jrarts. as it is reasonably obvious that the results are always correct.3. the initial and final states of tlte constituent machines lose their status and are replaced try rrew initial and final states. : (o+ bb) Figure 3.+ A).
(ba.n Lerrcuacns ar'ro Rnculan Gnq. .7 I Figure S.rrr.t ((a * bb). constructeddirectly from first principles.7 Automaton accepts .0 (a) Mr accepts L (a + bb). are given in Figure 3..l. we get the solution in Figure 3.rV (a) 1J 6) r /\ Figute 3. {. (b) Mz accePts f.80 Chopter 3 Rncul. ! b M z r\t fln . d M . Putting thesetogether using the construction in Theorem 3.5 Automaton for 't (ri).rr.]\ *\*/*\.+ I)).raRs Figure 3.tr).6. Automata for (a * bb) and (ba* * A).c. (ba.4 Automaton for L (rrr2)' Figure 3.
Tltere are several ways to do this.. that is.rarily.nculeR LANGUAGE$ 81 for Longuoges Regulor Expressions Regulor of It is intrritivt:ly reasonable that the rxtrrvurst: Theoretn 3.l state. one of the tnore intrdtivtt approaches requires a sidel trip into what are called generalized transition graphs.we will deal with it informally. This creates a bookkeeping probklrn that rnusl be hanclled carrcfully..rrywrrlk from the initial sta.nguagc:ltas an a"ssociatednfa a.bels of all thc walks from q0 to anY fintr..t for clvtlry regular language. it represents L(o. thero should exist a corresponding regular exprussiort. We could have labelad this edge a* without changing the language accepted by the graph. I The graph of any nondeterministic finite accepter (:an be considered a generalized transition graph if the edge labels are interpreted properly' An edge labeled with a singlc symbol a is interpreted a^san edge labeled vritlr the expression a. This dot:s not Iook too clifficr. it follows that for everv regula'r latrguage...nd hence a transitiotr graph.te to ir lirral state is sition graph.). Sitrce any regultr. The strings denoted by such regular exprtlssiotrs are a subset of the language accepted by the generalized transition graph.hlcof getrerating the la. as should be clear ffom an inspet:tiorr the go) Iabeled n is a cycle that can genertrtc any trumber graph.2 CollruncrroN Bnrwnnrr Rncur. there exists a Figure 3.3. itr any order.b.8 . languagc being the union of all srr<:h Figure 3. with the full generated subsets.r lir. The language acrxlpted of by it is L(o* + o* (o+b) c*). and thrr. and hent:tl itself a regular I expression. all we neecl to rlcl is to lind a regula'r expresuitlrr captr. is interpreted as an edge labeled with the expr()ssioll a f b * .L should hokl. Since this idtlir is in used h<":re a limited way and plirys rro role in our further dist:trssiorr. Flom this observation. A getreralizedtransition graph is a transition graph whosc edges aret lub*l 'Ihe label of a.8 represerrts a generalized transitiorr graph. of a's. tfie concatenation of several regular expressions. The edge (qo. while atr edge labeled with mrrltiple syrnbols a.rltbrrt it is corttplicated by the existerx:c of cycles thaL catr oftetr be travt:rsc:tlarbil..AR ExpRpssIols arul R.
r language.stand for general expressions' tai a. wtr ornit the corresponding edge in (b).is. we will not pursue them here. Section 4.9 for all pairs (qt. .{q} before removing q.. there are some subtleties in the argument. We interpret the graph of.L. Accepting this. The complete process requires that this be done for all pairs (qi.qi). every language of iccepted by a generalizecltransitiolr graph.9 e ) a. To remove a vertex labeled q. Proof: Let M be arr nfa that accepts .9 shows which edges have to be introduced so that the language of the generalized transition graph does not change when we remove q and all its incoming and outgoiilg edges.. However.1.il4 as a generalized transition graph and apply the above construction to it.(l: Irr ca^reswhere an edge is missitrg in (a).T " \ J (a) generalized transition graph that accepts it.b. Conversely. where the state q is to be removed a"rrd the edge labels &..aJ 1 th@ ntosfi'e"eiili" tlie sense' rhe casiaefiiEteilETfie to all three Yg$!c9! qiL+j. The corntructiorr that achieves this is illustrated in Figure 3.reqular.3 for details.pctll.9. we are ready to show how any nfa can be associated with a regular expression.aNo R. . Then there exists a regular expreti$ion r such that .f. this appears to be an immediate consequenceof Theorenr 3. but refer the reader itrstead to Exercise 16.After all the new edgeshave been added.en LANGUAGES Figure 3. The construction in Figure 3. Siriffiiabel Tvery wat[In a genetralized transition graph is a regular expressi6n. we use the scheme in Figure 3.L: L(r).. Let tr be a regula. want to create an eqiiiVEittint'f6neralized transitiorr graph with one less state by removing g. Although we will not formally prove this.. L j t .cR GRAMMARS Chopter 3 Rncur. We can assullle without any loss of generality that M has only one flnal state and that q0 f F'.1.Si) in I . it can be shown that the construction yields an equivalent generalized transition graph. We can d0 this if we do not change the language denoted by the set of labels that can be generated as we go from q0 to q.
n b n b t a.10.c.Ncuncps 83 Figure 3.1) denotes the generalized L . ){'L!.b" rz : o. removing one vertex after the other.f\{ / lt.11(a). r HxsmplC fl"S generalized transition the Consider nfa in Figure3. T4.3. we arrive at the a..The corresponding graph after removalof state q1 is shown in Figure 3. A regular expression that denotes the language accepted by this graph is r : rlr2 (r+ + rsrirz) (31) of Since the $equence generalizedtransition gra.TB* H.phsa^reall equivalent to the initial one. Making the : identificatio[ ln1 b + a. and alwavsworks. The construction involved in Theorem 3.1l.1{) r4 }. lve can prove by an induction on the number of states in in transition graph that the regular expression (3.rr '*) q with all its incident edge$ can be removed.h A l b+abta a+b { r' }11 r'. I Figure 3.'='/ \+J/ n rr rl \ / (rn (t(rq +rr r.bb.af b. l"t ab*b A . regular expression (a r : (b a ab* ab*b * b)* a)* for the original automaton.11(b).".r' *\.2 CoruuncrroNBnrwporu Rnculnn ExpRpsslotrs AND REGULAnL. until we reach the situation shown in Figure 3.2 is but it is completelyroutine tedious and tends to give very lengthy answers.xcr )) . We continue this process.1 \'l \+J/ (b) 0 F.11 a9'__J9"\? o .
ven number of rr.b}..{* e {rr.14' Finally.(ur) is even and n6(ttr) is odd}. Next.:.Lto detrote an odd this we eaJilil * ott]With . we rerllove tlrc vertex labeled O0' This gives Figure 3.13 .n*t*ne"Iffia*nd get the solFion in Figffe. We label the vertices with EE to dcnotc zrrl e.1) with rr: aa+ ab(bb). First. we applv (3.rFumberof a's anran ..nculaR GRAMMARS trnq'tmplut.'s and b'E with O.t n for Firrd a regular expression the language L. d\Itl) /"\+ \ r+: 0. \ rc: I b*a(bb)*ba. An attempt ttt construct a regular expression directly from this description lcacls to all kinds of difficulties. finding a nfa for it is easy as long ir. giving the generalized trarrsition graph in Figure 3.Chopter 3 Rnculan Laucuaces amo R.swe rr$evertex labeling effectively.ba..re. : ru. . On the other hand.12 Figure S. !+' Figure 3. v/e rcmove the state labeled oE.3'fZ' We can now apply the conversion to a regular expression in a nre<:hanictrl way. 'rz:b+ab(bb)a.13.
.such as identiflers. dernonstration of how tho irluas we ha.. mclst rNlitors c:xtt:rrd this to penrtit searching for patterns.we explored the t:orrncx:ti6rr bt'twenn finite accepters and sorne of the simpler constituents rif prr. ir.A}. but rrcvertheless instructive.15 and in Exercise ls.tccl ancl exterrsive area of computer scienut to which we calt only briefly allude.d* .lkNl attout stl far have been found useful in pattern mtr. Tlxl rclation bctweerr finite autorna[a and regular expressions mean$ thilt wc can also use regular expressions irfJrr wiry of describirrg these features. 'Ihis is eas." a terrn that refers to a set of objects having $ome common propcrtics.: i .. rrir"tt:lrirrgoccurSintextecliting.J.14 + ab(bb)rba b + a(blt). Section 2.yto see.tlrrl eclitor ed in the UNIX operating systenrreqg$ll?fl jlq jgl[t4.tchirrg.Alltexteclitors allow files to ht: st:arrrrcdfor the occurrerlce of a given strin65. 2 colnpcjrrohr Bp'rwpEN RnculaR Expn. g{tT.anLeNcuecns eur 85 Figure 3. firr examplc.veta.ba EE b + ab(bh)*a e9 The final expression is lorrg irrxl corrrplicated. The example below is a simplifitrd. thc kcy to successfulpattern nratching Lsfincling an effective way to describe the pattertis.tf a1q5brrg ref'ers ttr gories.ESSroNS R.I* iry" Ll"t'x "t) . I Regulor Expressions Describing for Simple Pofierns Irr Exarnple 1.rgratrrrtirrg larrgrrages. This is a cornplicir. For extr. thc sct of all acceptable Pascal integers is clefined by the regular expression f')r'q' stl. but the way to get it is rela. with possible values from {t. Often.mple. tivcly straiglrtfbrward.rxl r/ stands frlr the dicits 0 to L Ptrsr:al irrtcgers are a sirnple case of what is sometimes clallccl a "pat$crtt.s staruls fcrr tlte sign. wherc.ncur. or integers and real numher$.
tecl.4 is helpful.an GRAuneRs the file for the flrst occurrence of the string ab. I bln.rch followed by arr arbitritry ntrmbtlr tlf tz'tl. (a) r(aa.ti.tterns."+aba.1. If the pa. If memory space is a problem. the state to yielcl autorrrirta with reductiorr rrrethotl rlt:st:ribedin Section 2.tching a.86 nNu Chopter3 Rr. Fiflcl aII rrfa t]rat accepts the complement of the language in Exercise 1. 'l'heorem 3. is eft'ectively tltr pl. Searching rr file for occurrellces of a givcn string is a very simple progranming exercise. lsing the constructiorr in Theorem 3. ideils from the recognition pt:ocess arttoma'ta' theorv a. but lterc tlie situ.m must be considered also. but created at mrr tirnc:. as eln instruction to sea.rtion is more complica.1 tettds of finite autorrrataI'rornrtlgrrlar tlxpression$ rntrny states. in the fomr of a trtrnsition tatrltt. (a * aa)) (c) I ((obab)" * (aaa.b(o + bb)")' ffi Find clfa's that accept the Ibllowing larrguages. so must be flexible. Tlte constructiorr using Theorems 2.1. / T l llse the conrrtrrrction in L (ub'aa.to find an nfa that arcepts the language 2.s llecuI.r that gives the general framework for using the table' Irr this way we carl autotnatically }tarrtlltl il lilrge number of patterns that are defined at run titne. Tlxr pa. furthertnore.rilv rrlmplicatefl pa. AII the progranllllel has to do is to proviclt) a rlrive.ttern is specified by a regular expression.ttern description is part of the input.). the pattertrs are not fixed beforeharrd. To solvc this probltrm. @ [1) Ciu* an nfa that accepts the larrguageL((a+ b).1 and 3. This clfa.h).) ffi (b) t (ab (a * ab). Tlxl lffit:iunr:y of the progra. the patterrr recxrgrrition prqgram can take tltis description and cotlert it into arr ccrivalerrt nfh.reoften used. We see from this example tltat the UNIX editor can recogtrize regular expressions (although it uses a sornewhat different convention for specifying regular expressions than the one used here)' A challenging task in suclt ar} applicatioil is to writc an effici<rrrtprogram for rec:ognizingstring patterns. I'heoretn 2'2 rrray therr be ust:tl ttl recluce this to a dfa.tternma.lgorithrn. We have to deal with an unlitnited rrurrrber of rrrbitrtr. f'ollowed hy a c. b)) .) @) r (((aa.*a.GI.cur.enLANGUA. f b).
Give explicit rules for the construction suggested in Figure 3.'* qLlwhat langr.ba).nculnn Expnnssrorrs ehrr R. Find an nfa for Exercise 15(f). !'ind dfa's that accept the following languages.ncuLaRLANGUAGES 87 5. (a) L : L (. (a) Find an "^equivalerrt gerreralized transition states. a*) (b) L : L (ab*a*) n r ((ab).tu)./Consider the following generalized transition graph. t . use this to crerive a regular etpression for that language. 6 ..rageis accepted by the following generalized transition graph? a+b a+b' aab+t Q0r)Find regular expressions for the languages accepted by the following automata. f$.g when various edges in 3. section 8.3. ffi . ffi graph with only two (b) What is the languageacceptedby this graph? W ". .9(a) are missing.1.2 cor'rNncrtor'rBnrwnnrq R.ab* U I ((ab).
.$u) .9 generate equivalent generalized transition graphs.) .') is even} 1 3 . ('u)) mod it : 1} ) (..6(rrL) are trot}r everr} (b) . 1 5 . only a1 approxirnate otre. : {tt : (n.10.: { t n : ( n o ( t n ) n ' 1(.nt.Rs LANGLTAGES (") 0 11. Find a regular expression for Pa^. Prove that the constructions suggested by Figrrre 3. : {ur : 2n* (u) + :3n6(u.this tirrre elinrinating the state OO first' (@ f itta a regular expressiorrfor the following languages on {a'b}' (u) I : {rrt : no (ru) and rr. Write a regrrlar expression for the tret of all Pa^scalreal rrumbers' lrets whose elementsrare integer mrmbers' 1 6 .rcal ileed t T . Section 2'l' r4. In sorrre applicatiOns. !'ind a regular expression that generates the set of all strings of triplets defining correct binary addition as in Exercise 23. we Inay rtot of t[e pattern.T a )r n o d 3 l 0 } (d) 1. Rework Exarrrple i3.88 Chopter 3 Rlculan AND RECULAR Gnauuil. srrdr asPrograrrfr that cheCk spelliilg. Once the notion arr exact rrratch .
Rightqnd LeftLineor Grqmmqrs A grirrrtrnar G : are of the fbnn (y. using a.n.t (tr).ser.L.reof tlu: lirlrl A I1t:. utt € L} .3. corrsi<lerall words that can be forrnetl frorn 1.sinprrt a regular expression for .rnrnilris said Lo be leftlinear prloduct. lbnnally rlefirre this operation rlrop for lang.alrlr.S.nd l. Whcrrrlver we define a. a.. * 18. ln effer:t. shr:w how orre can construct an nfa frrr rirr.rlariguagesis hy means clf certain simple grer.uagcs. by dropping a single syurtrol tif the string.rB. a spuriorrn^syrrrbol anywhere in a lvrrrtl. Aner'logousto the previous cxcrcise. E 7'+ A gra. First.mrniirs. Is the result corrsistent with the defirrition of thesc la.serl (. A + it:.q. wlrere A.L). given an nIa for . * (a.P) is said to be rightlinear if a. regular larrguage on }J and define 'i.nguages? RegulorGrqmmqrs tffiffiffi A third wa. As arr i llustratiorr of this.L.Grarnmars a. consider patterrrs derivcd from the original orres by inscrtion of orre syrnbol.3 Rrcur.reof'tr:il ail alternative way of specifying languages. Usc the conrrtnt:tion in Theorem 3. Let 1.T.Gir. ltrrrguage family through irrr automaton or in sorncl other way.nrr. we look at grarrrmarsthat gurxlrate regular Iangua. Ctustnr<:t arr nla lbr droTt(L). autornata theory can be applied to corrstruct approximate patterrr rnatchers.ions a.1 to find nfa's for L(aa) arxl L(O*). if all . B E V. i'rtsg'L(L) contains all the words crcated from tr by inserting.y of describing regrrla.ges.) CJiverran nfa for 1. ffi **(b) LJiscusshow you might use this to write a patternrecognitiorr program for i. we err()irrterested in knowing what kind of grammar we can associatewith tlrrl farnily.ns 89 of an approximate matt:h has been madc precise.rtsert(L) * {uau: a E X.ll prorluctions A . bc a.
). a. Sz} . Although everyproduction is either in rightlinearor leftis not rcgrrlrr. we tlirn $eethat L (G2) is the regular language I' (aab (ob). is tine tha. The granunar G2 : ({S.u.rgramma.Stab..{o. S.90 Chopter 3 Reculln GRAMMARS LeNcuacrs ann R.t nlost one varia[le appt)ilfs tl1 the tllsistent'lv right sicle of any productiorr.rGr:({s}. Flom this single instance it is ea"sy conjecture that . thet Erlablgjlxsl or bir eitherthe rightrnost leftmostsy"tbg]gi$Effi tion. S'.)' I .rs.P) with producti''s '5. . {o.i. In a sinilar way. Both Gr and G2 are regula. Sr r Srnbl5zo '52 + ft'' is lcfblinear. fz).R or AA regular gramfirar tt.5r. {o.lti$[$ .h9+ahab9+rtbaba to is a clerivatiorr with Gr. and .t} .L (G1) is thc lirrrguageclenotedby the regular exprtrssionr =.t is either righlliilear Note that in a regular Hrirrnma.4."egramma.r.trl.q.l.rc.withP1givenas SrtbSla is rightlinear. with producticlrrs S . f'\rrtheilnore. . (ab). BAh.r. b} .a.. or lcftlirreirr.Pr).Bl. Iinear fbrrn.S.mmaritself is neitlrt:r rightlinear nor leftlinear.. the gra. A .$N$iN$Ns[liEihi. BI . The stltpttntle: S+a.}.
To do so. then btx:inrstrof the lorm of the proclu:tiorrs in G. cleir.In this scheme.l ftrrm. The gramnlirr is *rr . Proof: We assumethat V = {V0.x*r'ple of a linear A litrear granrmar is a grammilr irr which at most.n imitatc this step lry going from statc D to state E w]:err a.ncul. Note that ther scntcrrtial forrns of a rightlinea. a. 1 tt'ttJ.or V.bte.tcdliy ri rightlinear gra.rnmar is tr.ges.}.t)t.ys rtlgulil..chof thtlstl u's irt turn. regular grammars a.r.fcsular qratunalh_alwavslilear.he rightmost symbol..lwir.regula. one iEliiElEorrr (xr(:rrr on tne rlght $Idetrtf arry ]1lp. RightLineor Grommors Generote Regutqr longuoges First. whilc the part of the string a. 4I)11)2.twr: have of Jrrodrrctiorrs the form Vs i r)tvi1V .nnfir that ruirnics the clerivations of a.t regulrrr grarrrrnars are associa. tlrt rrot all linear gl'aloqrar$ arreregqtal_ I (Jrrr rrcxt goal will be to show tha..r grirrnrrriir lul'e the special fonn in whic:h thcrc is exactly one variable a. c D+ a . b ' . 1 .lreacly pr'cessecl is identical to the terminal prclix of the sentential form.rea..t. t]re derivation must have thc fbrrrr V o1 u t V .S.ncl t}ra.il.r grirnrrrrirr.tc of the autornaton will .rriveri tr. LeL G : (V.. ' r d E ..UpV71 4 1 1 1 1 ) 2 '"' u k l J l = ' I t J .t try usirrg a production D + rIE. This sirnple icleais the basis for the fbllewirrs theorem.lJ Fl. Thus. The corresponding nftr <:rr..?) rhe automaton to he t:orrstnrcted will reprocluce the rklrivation by .. If .IrueRs 9l tlrt:refore is not regular.T.n GnAr. .y1.the stir. Srrl4xlsenow Lhat we have a.r Iirrrgua.nothcr wrry of talking about regu)a..rly.q.tV. svmbol d is r:nrxlurrtered.ivation a b . a. * __ (3.rois a stritrg in tr (G).step in tr rlr:r. +':**::*:* ttris varia.trlof the automaton corre$poncls te tcl variable in the sententia. we coustruct tr.. a.consuming" ea..uzVj. 'Ihe initial stir.nclit rir:<:1rs as l.4tletjoLwithout restriction orr tht: llositiorr of *+"#. that s: Vo. P) ht: * riglrtlirrear grarnmar.tecl with rcgulirr larrguagesand that fbr every regulirr language there is a.riglrtlirrear grammar. we show tlmi a language generir. Then tr (CJ) is a regular ranguage..
) F \ . Thc mrmplete automaton is a.. ir*uuume M wa.+a 1 a 2 " ' a * V i .a2. to y/. and for each variable Va there will be a nonfinal state labeled Vl. Supposenow that w e L (G) so that (3. Reprcscnts fl=6 \7 \J V. The intermediate states that are needed to do this are of no concern and can be given arbitrary labels. ( Y o& r a z ' ' ' a * ) : V i ' For each prodrrction V'ataz"'ftrntt the corresponding transition of the automaton will be 6 * ( U . a^V. innd $o on. of rr rmrst have the form 'ut = ltt\1'2"'LlnUt (\_/ A n . that trr is accepted by M.. The general scheme from such is shown in Figure 3. lr$ing paths labeled u1.15.lrun RpcuLRR GRAMMARS btl labeled Vs.Vr.i and Vi that is.+ Represents arar. by construction.ssetnbled individual parts.92 Chopter 3 Rncuren Lencuacns ./ V+ arar. Therefore.sconstructed.'. so that clearly V1 € d* (H6. Because of the way in Corrversely. to Vi labeled u2.. where V7 is a final statc..th from V.2) is satisfied' In the nfa there is. a path from V6 to V.. e pa. For ea<'hproduction f i . to accept ur the automaton has to pass through a which seqqence states Vo. d will be defined so that . d* ..r) . and tl is ar:cepted by M. r ) : V y . labeled u1.. the automaton will have transitions to connect V.. d ..'u.. & t & 2" ' g .
rguage L ((aab). o. ab).ph.3. Fbr the se<. we start fronr the dfa fbr thc language ilnd reverse thc construction showrr in Theorem 3.. . then there exists rr rightlinear g r a m r n a rG : ( V . t . and the symbols causing the tra.with vertices Vo. The sta.nd accepted hy the automaton is the regular la.reled betwecrr tr/sand trzr.lphabetE.ond a rule. t ) 7 is possible. ' .L is a regular language on the a. We start the transition gra.3. P ) s r r c ht h a t L : L ( G ) . 5 .u n V n4 u 1 u 2 . If . .t there is a path Iabeled ob between lt and Fo. we neecl to rrdd an edge labeled b between Vr and Vy.rrsitions hetxlrne the terminals in tlte productionu. giving the autornaton shown irr l'igure 3. The first pro* drrctiott rule creates an edge la.get:irrr be generattxl by some riglrtlinea.Vt.16.tcsof the dfa now become the variahles of the grarnmar.3 RnculnR GnauueRs 93 Figure3. Hence ur is in I (G). I Rightlineqr Grqmmqr$for Regulorlonguoges To show that every regular langua. Finally.r ir. we need to introdurxl att additionnl vertex so tha. irrrd the theorem is proved. and VJ.r grammar.bVolb.16 and the derivation V " + u t V 4 u 1 u 2 V i3 u p 2 . I to'that grammar V(t V  accepts the languagr: generated by the aVt. The language gerrerated bv the gramma.l. t ) 7 y . .
' ' ( I k & r . Therefore we cirn make tltc derivatiotr "'(LnQt.Qt.v can gelrerate every strirtg in . if 96 is in F. . .nsition il (qt.at):qy e F' By txrnstructiorr.ataj ' ' ' apal) : Ql t complcting the proof.'. r . t q n }a n d E : { a t ."'. . tht:n its derivation must havc the form (3'5). and u E L(G).'ia k a r q l 1 a i a i . C o n u t r u c t t h e r i g l r t .1. . For &/ to acccpt this string it must make moves via d(qo'at): qo. But this implies tliat ii* (qo.a n ) : q t ...1f u e L (G).Q..oi) : qn of M. 1 a m } . we put irr P the produr:tion qi + a. 6 (q.ep aiai4r I aiai 1 u . (3. a z 1 .g4 GnaltltaRs Chopter 3 Recul. S. P) with v = {qo. . 6(qr.5) with the granrlrlar (J.qo. Consider u € L of the fortrt w : (]. In addition.5 : gu. For each tra.L' We assumethat Proof: Let M: : { q o . 4 QoI ut.\. d ( q " . . (3'4) (3. Conversely. q r 1 .+n LANculcEs ANDREGULAn (Q.3) We first sltow that G dcfined in this wir. wc add to P thc productiott {r + .F) bc a.d. ai) : e.i. the gralnllrar will have orttt productiort ftrr each of tlxrse dts.l i n e a r Q grarunar G : (V.t.aj"'anilt.r} and .jqh. . a.dfa that ac:r:epts. E.
aQ2I aahq2 =+ aabaqs + aaba. tlfa is uot essential to the proof of Theorem 3. Proof.4. I Equivolence Between Regulor Longuoges ond Regulor Grommors The previous two thrlclrerns esta. With minor modification.) = {92} E(q. b) = {qt} E(qyo) = {s} lo d(t {tEtt 9zbg. A lanSirragetr is regrrlar if and only if there exists a leftlinear ggammar G s u c ht h a t L : L ( G ) . Construct a rightlinear gr&mmirr fot L (aab*a. it is usefhl to note that the restrictiorr that M be a. The transition firnction for att ttfa'.17. H$nuwtplo I $ $'. bc clerivedwith the constructed grarrrrnar can bv Qo 1 (rh ) 0.rs. A+rj. The string aabcr. lt aQJ ?. the sarneconstnr(:tion can be used if M is an nf'a.17 o) s(so.bliuli the connectit:rr between regular languagcs and rightlincirr grammars.). One can make a simila. is given in Figure 3.4. The result was obtained by sirnply following the construction irr Thtxrrern 3. We orrly outline the main idea.. Given any leftIirrear grammar with productiorrs of the fbrm A +Bu.rEF 9t L For therpurpose of constructing a grammar.3 It.[]cuLAnGRnnrnraRs 95 I'igure 3.r connection between regular langrragesand leftlinttar gramma. ={sJ E(qt. thereby showirrg the complete txlrivalence of rcgular gr&mmirrs and regular languages. .3. togcther with thc corresponding gramrrrar prodrrctions.
we useExercise Section verseof any regular larrgrageis also regular. ./. 3 3.uive a.whichtells us that the rc12. A Ianguage L is regular if and orrly if there exists a regrrlar grammar G suc:h that L: L(G). wc a.L(G)' is regular. We now have several ways of describirrg regular languages: dfa's.cilg every suc:h prodrrction of G with A uRB.mmars' While in sorne irrstarxle one or the other of these may be most suitable. as sltowrr in Figure 3.Chopter 3 Recrrlan Lnuclul. regular expressions.ke it clear quicklv that . Next. they are all cqually powerful' They all give a complete and unambiguous definition of a regular langnage' TIle connection between all tlrcse concepts is established by the four theorems il this c:hapter.cus ANtl RH(IuLAR GnRunlns we r:onstruct from it a rightlinear grammar G try repla.2 Theorem dfa or nfa Theorem3 .t the equivalence of regular larrguages and regular grarnmars. respectively. Figure 3. 1 3. (")) 2. nfa's. / /^\\R A few examples will ma.t&.4 and 3.L((d))R.18.3. r Prrtting Theorems 3.18 Regular er<pressions Theorem3 . Sinced is rightlirrcrrt.L (G) : (. 1{ _+ t.4 Theorem Regular Brammars . (d) But thenso are.5 together. and regqlar gra.'nd.
Find a leftlirrear grammar f<rr thc language in Exereise 5. Construct a dfa that accepts the language generated by the grammar S abA.n. F'ind a regular gramrnar that generates the language (ur) is even} .Albb.b}.ru6(. ' (O f itta a regular grarrrrnar that generates the language on E : {r.rrunRs 97 1.5.. (e) : (f (G))8. t Lj. 1 @ linrl a regular grailrrrrar for the larrguagc L .rn > B}.r. A + baB.{a"lt"' tn I mis even}. L = {w € {rr.1 1 6t r ) l i s o d d } . S 11. (zrr) arc both even} ffi ' (b) .).rrr)) mod:l I 0} (d) . Construct a riglrtlinear grailrrrrar lbr the langrrage L((aah. : {w : (no (w) . ( (l$y'Show that for every regular language not containing.{w : (n* (ur) .3 Rncure. '. (e) L : {ut : no (u) and nr. 2.t n" (w) * 3rr. 9.} 2.ab). eVardaeT B . W 1 G Suggesta construction by whicJra leftlinear grammar can be obtairreclfrom an nfa directly.provethat l.6 (t G Find regular grammals for the following languages on {a.no (to)) mod 3 : 1} (c) I . Constnrct a leftlinear grarnrnar for the larrguage in Exercise 1. fi + u. whereA. 7. ffi 5.\ there exists a rightv linear grammar whose productions are restricted to the forms AaB A+tL.antl leftlincar grarrlrrrars for the larrguagc L : {atth"' :rr.) Corrstruct right.. Find a regular Blarnmar that generates the language L(aa* (ab + a)). In Theorem3. consisting b} of all strings wittr no more than three a's. b}.: \\ {tn: lno (ur) .GRerr. S.ll.
X .98 Chopter 3 Rnculen Leucuac+psaFIIrRnculan Gn. Find a regrrlar gramlnar that generates the set of all Pa"scal real numbers.cn. ( r c .s {r+} \'l Sho* that any regular grarrrlrrar G for which L (G) + g must havc at least one procluctiorr of the form 4+r. 5 r .l i n e a ra n d G r = ( V r .t'. grammar.P). E . P ? ) b e a l e f t V that Vr and l/z are disjoint. ) f e t G 1 = ( V r . whereA e V and r ET*. Show that L (G) is regular. where. atttl a^ssume ({S}UyrUyz.9. S r . $l# {5 .9is not in VtJVz and P: g r a r n m a rG : * SrlSz} u Pr u Pz. P r ) b e r i g h t . 15. Consitler the linear lin*u.q'r'tM.
As we wiII scc shortl.tion. Closure propertios.studied $om(]wirys irr which they c:arrtre represented.a. Is the resulting langua.gc: frrrnilies deals with our ability to tltx:ideon certain properties. we rnust inqrrirc rnrlrtl cleeplyinto the nature of rcgulirr larrguagesancl see what propt:rtics the whole fa.ngrralgcs arc. wcr such as concatena. tlltlxrtrgh tnostly of theoretica.l.milieswe will crrt:ounter. albeit velry txlrnplex.t ha.Jrpr:ns when we perforrn operations on regular latrguages.stion we raise is wha.sir.The opera.ting hetween thc various language fa. But to understarxl why this is so.1. Thc fir'st que.heatrswerto this r:rirrjrx:ture cleliniLely is no.lintorcst.Properties f o R e g u l q r[ q n g u q g e s e have defincd rergularlanguages.rnilv ha. finile llct by automaton. A srx:orrtlset of questions a.tiriris corrsiderare sirnple set opuratiorrs.n acx:eptecl some. We now raise the qrrcstirlrr of how general regula.r la.ns wc:ll as operations in whir:h etr.s. Could it be thnt cvt:rv forrral language is regula.closure question. St:c:tiorr 2.r'/ Rrrhaps irrry set we can specify c:a. and have $eenir f'cw rlxirrnples of their usefillness.ch string of a language is chrrngrxl.sfrrr inslance in ltlxercise22.bout ltrngrra. cari we lell whether a lirrguage 99 . Frrr r:xirnple.y.ge still rtrgular? We refer to this a. ireltrlus in discrirnila.
we need another line of attack. brrt are not as easily Finallv we consider the important question: How can we tell whether given language is regular or not? If the language is irr fa<:t regrrlar.Li. But if it is not.r is to study the geueral properties of rcgrrlirr Ianguagtrs.Lr and L2 tne regular.r expressions rr a. Ianguages. similarities and differences in these propcrtics will allow us to contrast the various language families. We catr ask similar qucstion$ about other types of operationu on languages. Closure properties of variorrs language farnilies utrder differcrrt operations are of considerable theoretical interrest. a'ttd starclosure.lsignificancethese propcrtieu have. L1 tL2. L1L2. rl *?'2' ?"1 fl'Ild .We will see ittstancesof this (Theorem 4. is instances. more prac:tic. we look at a variety of propt:rtieu of regular languages' These properticu tell rrs a.Il regular languages.ter in this chapter. We say that Lhe farrfly of regular languagesis closed ullder ulri()rl.ill and Example 4. attd if we can show that the candidate languagc rloes not have it. If we know of some such property. Closure under$impleSetOperotions We begin by looking at ther closrtre of regular lattguages urrder thc (:ommon set operations. the auswer may be obviotrs.nuwered regular answered for other larrgua. If L1 and.gefamilies.Ll : L(rt) and tr2 : L(rz)'By defilititlrl. when we look at the same quttstions fbr other language families. By giving us insight iulo the general nirture of language families.nd ?'2' 12 such that. Is it true for all rt:gular but here we trr and Lz7 It turns out that the arlswur is yes. but many results are useful.such qttestions are easily tr. such as ttnion and irrtersection. concatenatioll. then so are L1U L2.great deal about what regular languages can and canrrot do. regular expressittn.f3) lir. Irr this chapter. it may rrot bc clear what practica.Chopter 4 Pnorenrrns on RrculAR LANGUAGES fbr is finitc or not? As we will see.Lz are regular liulgtrages. Proof: If . then we catr tell that thc larlguage is not regular. cornplcmentation. Ont: waY to show a language is not rcgultr. clostrrc properties help us answer qrrestions.fi arrd. we a can always show it by giving some dfa. sotne of Lhenr havr: vt:ry little. Admittedly. then tltere exist regula. their union also regular? In spec:ifit: want to addrcss thc problem in general.7 other. that is. Later. At first sight. irrtersection. or regular grammar for it. this leads us to tlte study of the closure properties of larrguagesin general. a fact we express by saying that the family of regular langtragcs is closed under union. o P C l o s u r e r o p e r t i e s f R e g u l o rL o n g u o g e s Oonsider the following questiorr: Given two regular languages L1 atrd L2. characteristics that are sharcd bv a.
lgtlrithms.i. Let (Q. closure unrler trrrioll.rrr) is deliled fbr all tr € E*.. Noter tltat in the defirritiotr of a' dftr. ..) whenever M1 is in sta'te qi a..'f . ta. in which caseru € tr.F ) accepls Zr. f . d .. as itr tnany c:ases.pti).Then L1) L2: Ltl Lz . k:t.pt). ( g o ..Fr) and Mz : : L ( M 1 ) a n c lL z : L ( M 2 ) . o r t ) * ( q 6 . Thrrs. X.t ur E tr1 f'l . M : ( Q .)' 96.1. : {r a) a.king the complcrnetrt of hoth sicles.dz. by M.2if a. thcy are ula. but it also shows explicitly how to construct ir.x'.f. fnis is rather strariglrtforwa. For tlosure uuder irrtrlrsection. dfa that accepts tr1.rrguages.): P' such thirt Qr e Ft a. and Lit and starclttsrrreis respectively. tomatrrrr e x P consists / \ of lrairs (q. Consequently.nrlM2 a combintltlau(P. Not orrly does it t:stablish the dcsircd result. To show c:losureunder rcrnlllerneutation.te. 4rr: sfiorter but nttrrt:ottstructive(tlr al least nrtt so obviously r:onstructive) argurrtettts.ssrrrnecl to be n total function. Thcrr as F is clefitrecl the set of nll (.F2)are clfir. L1L2. irnpurtirtrt because tlx:y give 1rs iniriglrt into the rrlsrtlts and ofttlrt serve a"s there thtr strrrting point fix practica..tena.X.finile acceptt.3). so that d* (qu. I The proof of t:losureunder irrtt:rsectionis a gtlod exampltl tlf a coustruc:tive proof.nd dz (Pl n. F ) .we havc already slggcstcrd Lhe result irr Exercise 4 in Scrtion 2.Ir.Po.1.u) is a final state.'s' We constnrr:t w h o s es t a t e s e t 8 : L 4 : ( A .i) be a 'Ihen ther tlfrr. Erpation (1. inmedia. conca.l tr.rrdpi € Fz. Hert:.Jo.trr fl 12 is regrrlirr.pr) ) : (tr*. Q .w.atrd whosc trarrsiliou functiorr d is such thnt M is itr state (q.( 0 .tiotr. p n ) .rrdM2 is iu stir.1 CLosuRE orr PRoPIIR:I'IEs Rncur. M : (8. d* we a. X .we start witlr DeMorgan's la.tcPi' Ihis is:rt:hieved by taking f ((qo. w h t r r cM t : Lt from Ml a..nRLANGUAcEs 101 ri are regular exprtlssiottsdenoting thc latrguagestr1 U L2.r la. Cotrsequentlyeitlrt:r d* (qo.{o.F a n d w e L . Dcrnonstrating cllosureunder intel$t:ction takes a little tnore work. sitrple matter to show Lha.nd orrly if it is arx:cptecl it isr.:lrj).'rfor lhe intelrscctiouof two rc:gConsJtructivepr:oofstlt:cur throughout this book.a whencvcr d1(q1.r'd.1. ' u )e Q .
we know that tr1 rf 12 is regular. Using closure under union. so are Z1 and L2. We then construct arr nfa with a single final state for it.3. Section 2. Then. Here are the details. then by closure under complementation. the modified nfa accepts .L2 is necessarily regular also.identity is immediately obvious from the definition of a set difference. It is a fairly straightforward matter to show that the modified nfa accepts ruE if and only if the original nfa accepts u. we wallt to show that if . we see that ffit: is regular. The family of regula.3. Using closure under complementation once more. we next get that Z1 UL is regular. and the argument is complete. if tr1 and L2 are regular. Lttt Lz The following example is a variation orr the same idea. namely LrLz:Lrf\Lz. Now.L2 are regular. this is always possible. WowthatthefamilyofreguIarlanguagesisclosedunderdifference'In other words. Suppose that .L1 and .Lz is regular implies that Ia is also regular. then L1 . Proof: The proof of this theorem wa^ssuggested as an exercise in Section 2.Chopier 4 Pnopnnrrns ol R. t . By Exercise 7.L is a regular language. The fact that .Bc(rrAR Larqcuecps for any languages Lr and tr2. The needed set. the final vertex the initial vertex. Therefore. In the transition graph for this nfa we make tha initial vertex a final vertex. and reverse the directiorr on all the edges. proving closure under reversaJ.r languages is closed under reversal.Lft. I A variety of other closure properties can be derived directly by elemen* tary arguments. because of the closure of ragular languages rrnder intersection.
.tcuu.):uLaz'. tlrcrt its homomorphic h(L): {h(r) twEL}.nguarges..1 Let X: h {o.lt'l't:"ll#1 1i'.(ut) : h.(n.(a1) h (az) . substitution is tr.ny srrr:h results. we select only two typical ones.u Sxsnple 4.geof L : {aa. Others are explored in the exrlrc:ises at tht: crrd of this sectitxr.6.osuRr. Tlte homomorpltic ima.(h): bbc. Tlrc dourain of tlrtr function h is c:xtendedto strings in a.R LAN(+rrAGEs 103 or underOtherOperotions Closure In acldition to the standarcl operations on la.. a hornomrlrphism is a. . Supposc X and I a. thcrr h. orre ca. Tlren h. la.1 CI. tlten a.nobvious fashion. in which a single lettcr is repla.b.b. Thr:re are ma.: Pn.rl dclirre other operatiorrs atrd investigatc closure propertics for thenr.ced with a string.function h:}jf* In words.llcld a homomorphism. aba} is the Ianguagc h (L) : laha.h. lf tr is a.b}and f : {a.nguageon X.c} arrddefine by h'(u):1.4. image is defined a. . h.Qro.) . Then a. h (a. if 1r.. ' lil$finirt.realphabels.opnnrrns R. I If we have a regular exprt:ssion r for n latrguage I.abbbcabl. regular exprrrssion lctr h(L) carr Lreobtnirrcd by simply applyirrg the homomrirphism to each E syrrrbol of r.ba): abbhca.
I liM on Let L1a. If L is a regular la.104 Chopter 4 Pnornnuns op R. we claitn that h (tr) is regular.1.'. af'tcr removal of this sullix.akcir.ARLAN6SAGES W. All we need to do is to show that for ()very ru e L (r').t have a sufrx bclonging to tr2.rrrt h {b.d}. .uctrt. .y }tottrorrurrphisms. It r:an be shown directly by an appeal to the definition of a regula'r expressiorr that the result is rr regular expression.r latrguagesis therefore closed rrrrrlt:ratbitrar.L. It is equally ea^syto see that thc rcsrilting expressiorrdenotes h (r).1.bcctlbt:c)* rlenotes the regulirr lernguageh (L). I The generrrl result on the closure of regular languirges under any horrxlmorphistn folklws f'rom this exarnplc in an obvious lrlanner.) is also rtlgular. The fanrily ttf regula.c.(a) for each sytnbol o € E of r.trt: a. denotedby If f.L1 tha. irna.ke t: E: {a. trelongsto L1f L2. regular languag<l denoted by sonre regular expressioll r. Then the right quotient of tr1 with trz is cleflled rrs € .b} '.llthe strings in. Proof: Let ..bet. is the regular latrgttage r = (e * b.w€ t. then 11: (tlbtc + (bdc)) (d.nd L2 be latrgurrgcs the sarnealphir.gcft.(")) tlterc is it ttt itt . tlten its homomorphic Let /r. such that u : h(w).nguage. . We fincl h(r) by substituting h. L 1f o r s o m e y € t r 2 } Lt/Lz {r:ry (41) To form thr: right quotietrl of L1with /12.holrorrtorllhism. thc txrrresponding h (rr') is in L (h (r)) arrd converselythat for every u in L (h.Leaving the details as an exert:ise.) (aa).Evcry such striug.be a. Defirxr hv h(a):46""' h (b) : 6.(I.
rrrl. Figure 4. ?n.L1 tha. We start with a.nguagesis also regular.F) irr l'igure 4.1 Clostlns Fn.1 . The strings itt .opnRTrns ol' R. } 1} .rn > 0} U {ba. Lp.t tr:rrrrinate with at least one l) as a sulfix.rnple.nyprefix of strings in. rz } I. We will prove this itr lhe next theorem by a.L2 consist of orre or rrlore b's. and Lt/Lz ale all reE.X.qu. we will trv to rrtodify M1 s{r that it accepts z if there is any y satisfying (4. we arrive at the answer try rcrnoving one or rtrore b's from those strings in . Since iln inrtotttaton for L1f L2 urust accept ir. tlfa for Lt/Lz.l:cjulan LANCUAGFT$ 105  s'4 Hr+r{rtrplo If L1 : {a"b"' : n } 1.L1.t hcre L1.L2 altd constnrcts from them a. lct us see how it a.1).4.pplies to this exer. Note tha.Bt:ftrre we describc the r:onstnrction in firll.> 0} .6.1.ular. {l!I : m. dfir for l1i sa/ thr: automaton Mt : (Q. This suggestothat the right quotient of any two regular la.} and L2 : thcln l. lllhrlrt:fore.t I L2 : {att6"' . constnr:tion that takes the dfa's for L1 ir.
whether there is a.te. Q1.8.cps Chopter 4 PRopeRuEsor. We modify the automaton a.F.li. Qt.F) there exists a y €.te labeled u sur:h tha.2. Figure 4.Lnrlcue. : q1 € F. We constnrct u. The resulting automaton for L1f L2 is shown in Figure 4..X. Q3. eo. we To apply this to our present ca^se. The arrtomaton Mi is M with the irritial state q0 replaced by gi. The idea is gerreralized in the next theorem I If L1 and L2 axeregular lauguages.2) : q will be in L1f L2.{i.) is a dfa. tfu:n L1f L2 is also regular. Proof: Let Lt : L (M). Qz.nother rfia fr : (A. f<rrca<:hq € 8. we determille. 92.d. check each state qq. follows. Check it to see that the construction works. walk to a final sta. If this is $o.t u € Lz. anY r such that d(q6. or 94. We say that the family of regular languages is closed under right quotient with a regular la.X. This carr be done bv looking at dfa's M.nguage.5.94 do not. € Q. We now determirre whether .Bcur.2 The dififrculty comes in finding whettrur there is some g such that ty E L1 nnd 37€ L2. For each et. = (Q. determine if ir.q0. We see that otrly 91 and 92 qualify.where M : (8. L2 such that y) d* (qo.nn.ccordingly to make t7 a finnl sta.F). Q+. qb to see whether there is a walk labeled bh* to any of the q1. To solvc it.'R.qo.
This is easy.1 s. To lrrove that "L (fr) : Lr/Lz. lf there is any path between its initirrl verttrx arrrl any final vertex. we can use the construction fbr the interscctiorr of tvro regular languages given in Theorem 4.c. we have d * ( q u . E Q. The examplc is sitttple etrough so that we can skip the formalities of the r:onstrrrction. then . LttLz.1.(fr): and fiom this that LtlLz is regular.L2 a L (M) is not empty.1 Closunr: Pnopr:t{rlr:sor RucuLA.t trt:cepts tr1.ting this f'or every et. . t Lt: Lz: L(a*batt*). Rut again by rxrnstructiotr.3. and a solution is given in Figure 4.3 it is quite evident that L(Mo))L2:s. for Converscly. so that there rnust be some q € Q such that d. we determitre F and thereby construct M. Repea.'a)e F.z) is in by F.g) e F'. This implies that d* (qo. acld qi to F. q d* Therefore. r:)q e F .Itcuacns 4. L (Mz). any r acceptedby M.R 107 there exists a E in L(Mt) that is a. Frrr this.L. Flom the graph in Figure 4.this implies that there exists a 37€ tr2 suclt tha. L (Mt).r) : q alnd d* (g. Therefore is in L1. finding the transition graph for L2tL (M1). € F. and r is in LtlLz. Lz : {o}. L ( M : t )i L 2 : s . g € L2 such that rg e trr. L ( a b * )' We first find a dfa tha. Lz: {o} # a. construction.ry) e F.We therefore r3t corrclude that . let r be arry elemetrt of L1f L2. Then '' ' \ / there must be a. In tlrat case.irtoin L2. and fraccepts u because (go. (gu.td* (q.
It accepts the language denoted by the regular exL1f pression of a*b* a.4. the automaton acceptin1 LtlLz is determined.*batt*. Thh:us L2: L(a"ba").Chopter 4 PRopnnrrEs oF REGuLAR LANGUAGES Figure 4. The result iq shown in Figure 4. I .4 \s' Therefore.which can be simplified to a*ba*.3 Figule 4.
". The symmetric difference n d:{ 1..e . a...) n L (a*b"a).r languages is closed under the nor operation. b /l r Show that the family of regular languages is closed under the cor operation.reregular. . lr. ffi (b) L (ab.) n. show that the family of regular languagesis closed under finite union and intersection. In Example 4.n} Le of two sets Sr and Sz is defined a.. show that the family of regular languages is closed under symmetdc difference. 2.r is not in both .L2."::. show that the family of regula.then Lu= and . s.1. Fill in the details of the constructive proof of closure under intersection in Theorem4. .. . J".. Use the construction in Theorem 4.L2)= h(Lr) nh(Lz) (c) h(LrL2): h(L1)h(L) .5r and Se}. that is. . but. of the following are true for all regular languages and all homomor (g/Whi"tt phisms? (a) h(hu r .2. .$ Sr $. . 7.. 6. In the proof of Theorem 4.Sr or fr € 52. L1 and w f L2} .rexpression. : 1 1 .1.1 Cr.r ls.osuRE PRopERTTEs Recur.T':. show that h (r) is a regula..L (baa.1 to find nfa's that accept (a) r ((a + b) a. but the proof was nonconstructive.)..Sz = {n t n E ... Then show that h (r) denotesh (I)..eR LANcuAcEs or l0g 1. z: h ( L r ) u h ( L { ) *fv* . Provide a constructive argument for this result. The nor of two languages is nor (L7. ffi 8' Denne rhe compremenrffjTl. if L1.4.L.a. F.1 we showed closure under difference for regular languages. 4.3. L2) : {w : w {. ) n U Lt Lt: are a"lsoregular.*r (b) h (21 rr .
t t ( .' 4il lI L. "If 'Lt is regular n'rrtl trr utrz is also regular' then 1 6 .r'l ffi 13. thefl all languages would be regular. h.r t r 7 .\ fto)t"t .110 Lat'Icuacns oF 4 Chopter PRopuwrttrs REouLaR . a t a z ' " t t n . if the statement ^Lr must be regular" were true for all . Lt: L(o...lul : Z} is also regular' 14.r languages is closed under this operation. Defirte an operation thi.} .2 is definecl as LzlLl : {g : r € L7. E L.} . Show that regrrlaritv is preserved utder the. and.'r11F.:.L is regula.'R \ : FirxlL1fL2.. Forasrring(rrtrz. . Wffi L 7 .ange(f) : {u : a : exchange (Tr. that is. ffi 1 9 .. gc ( . we can define the operation on a language as shift(L): {u:'u : sli.*haa*) : LrLzlL:r is not true ftrr all languages and Lz' I't \O " fojjEhow that L t .u E Zr} is also regular.nt.L}..tl(L) . ' ')' : a s a a ' ' ' with the appropriate extension of this defiflitiorr to languages.ft(ru) for some trr € . Flom this..^. W t\ The left quotient of a language Lr with respect to 1.eo. closure of the family of regular larrguages under this operatiotr.(rq.Lz is regula. If tr is a regular language.Lr and . If I/ is a regrrlar language' prove that L1 : {ua : u...:::. for some 'u e L} ' exch.'7. we SuDDose krrow that . Show that.r. 1 8 .Lr U ..il of a laflgrrage is defined as thc set of all suftixes of its strings.:TT:Trutr.i. The head.rand that L' is finite. so is toal (l). t 7 z ' 'f'tn .L1].shilt \\ c900"' aI]cI e n c h a n . Show that if . that is tu. F) show that the farnily of regular languages is closed under the left quoticut with a regular language.t(L) : {y : ny € L for some r € t.n//. Theto. of a language is the set of all prcfixes of its strirrgs.L2. prove that the langrtage {uu : u E L.:. Show that the family of regula./.{n : ny e L for solne gr€ t.Lr L(abo'). Prove the 20.) Sbow that the family of regrrlar langualges is clcisetl under euchan'ge.1 I t ' ' ) : Q r y t .rd on strings arrtl languages as t h i r d ( a n z o " t a 4 a ! . Can wc " {fi\ \\ U "on"tuae frotn this that Lz is regula. operation.
thorrgh.rg1r'rr:rrts.given a language. The question of the existence arrclnature of rrrerrtbership algorithms will be of great concern in later cliscussiorrsl it is a..L is defined as rni.wm € Lt.+\39Let  Show thal. can we determine whethel ol not .r an(l Zz is rlefinetl as . (Ga) ffi ffi glrammars tbr the larrguages \a..ar is an elemcnt of . shuffie(L1.L and a string to.L? This is the membership q*estiorr and a method fbr anuwrlrirrg it is called a membership algoritlmr. s u c h t h a t u r : . .l (b) ((rJ L (Gr) E l e m e n t o rQ u e s t i o n o b o u t y s Reguloronguoges [ wc rrow conre to a very fundamenttrl issue: Given a language . * 24.ul € X*) € show that operation.v QrrusrroNsABour Rncur. The m.t cxactly we rnearl when wet say . Gr and Gz be two regular grammars. u E E + . we have used several ways of describing regular languages: informal verhal .ny a..lr}6?}n1 wtwz.in of a language .. the farnily of regular languages is crosed under the sh'ffie * 28' Define an operati.L as the set of all strings of tr with the fifth symbol frorn the left rcmoved (strings of length less than fivc are left unr:harrged).rlmi'nuss on a language . Is the family of regular languages closed under this oper. The shulfle of tu'o languages . L2) : {w1u1utzuz. for all ru. We first consider whir.it is irnportant that this be unambiguous.4.tte .eR LANLITJA#E$ 111 * ??.ation? 25. . r o } . the family of regular larrguages is closed under the rnriruoperation.L bv on Ieftsi.ruruq e L} .2 Elnvnrvrnrr. i u1u2. {?./_\ 2.l. it is an easy lrratter. very little can be done with languages for which we carrrrot {ind elficierrt niembers}rip algorithms.1)m Lz.. Show how()rre derive can regular L(Gt)r L(Gz) ffi L @r) r.n(L): {tu e l: t } r e r ei s n o u E L ..r issue that is often difficult.. Definc the operation leftsi." In ma.. trtrr regular languages.de(t): .. Show that the family of regrrlar larrguages is closed under the rruinusSoperation.
qc+l:s rlcsr:riJltiqns. Often several defilitions of a programming language exist.reon a path from au initial is to a final vertex.ngrtage ttot enrpty. a regular expressiotr.t a regnlar language is given in a standard representation if arrd orrly if it iiJ dQst:rihedby a flnite automaton. Orrly tlxr la.r larrguages thel ilrgument is not obvious.lgorithm fbr determining whether a reglrlar lirrrgrtir. Iitrite.L1 irntl L2werdefine the la'ngua. finite altomata. lf there is ir. 'ur ttl stNrif it is Proof: We represent the lalguage by some dfa.ge t4: ( L 1 . attd whether orrc langrttr. given there exists an a. . It is 1ot possible to arguc on il serrtencebysentencecomparison. there t:xists an algoritlrm to determiue whether or not Lt: Lz. For rtlgular langrtagJes least.nR Lerucu. I The question of the eqrrality of two languages is also an importailt practica.. 1 6 )  ( t ' t n t ' z ). Otherwise. standard representation of any regular larrguagc tr on x and any u € X*. Given standard representatiols of two regular lalgtrirgils L1 iilIrl tr2. or clfa's.is etnpty. Given a. wc thcrcfcrrc sa. there exists an algorithm for detertninittg wltether or nttt ru is in I. or infirritc.ge a subset of at another.rcl Proof: Tlrg itrrswt:r is a.ton. Proof: Usirig .Ianguage is infinitc. to Nor iu it ea$y see the answer by looking at tlte regular expression$' gramma. An elegant solution uses tfue alreatly estahlished closure propr)rties. these questions are easily arrswcrtld. I Other irnportant questiorrsarc whether a language is finite or itrfinite' is whether two languages are the same.y tha. This is gelerally a difficult problertt.qititln graph of a dta.set nqtation. If irlY of these a. it is finite. irr sta. then the la.gc irrfirritc. the lattgtttr.ge. sirtce thiS works only fbr finite languages. find aII the vertices that are the base of sorlrecyclg. representa'tion. in spite of their dificrent irppearances.l issue.st three are sufficiently well defined for use in theorerns. or a rt:gtrlir'r grilrnfiIirr.ndir.and regrrlar grarrrrrrars.rs. and we need to know whether.sinrpkr pnth fiom the initial vertex to any firLal is velrtex. tltetr test accepted by this automtr.he latrguage as ir trirrr. cvtrn ftrr regula. regular expressions. To dt:termine whethel or ttol a. they specity the same language.ppa.rentif we represent t.tt2 Chopter 4 Pnoprnuns or Rpcul.
t. de g.'r. ? € X*.tion.L2. Find an algorithrn to dctermine if "L coltains any tu such that fi is a substring of it.Lr arlcl /.2 Er.r{s 118 Bv closure. assume tha. Show that there is an algorithrn to detemrine if 1.b to 4.Ac.1. Show that for any regular Zr and. in spite of br:irg obvious ancl unsurprising.3.r.rlar larrguages. tlrat is.. 10. there is an algoritlrru {.Ls. Section 1. for arry given to and any regr..ys case when we deal with larger the farnilies of langrrages.lwir.' X" } . 7.L contains any string rrr such that tuF € 1. but this is not ir. € .l.: tr/i.nfincl a dfa M tha. tr.1. itr stanclard representtr.we will encounter cluestionslikc these on several occasions la. : shuf f Ie (tr. tletenninerr whcther or rrot .BculeR LnFrL+r. ffi 6.L. arlcl we ca. wc will see that the answers ber:gr1(: irrcrcasirrgly rnore difficult. given any three regular languages.1. Show that there exists ari algorithm for determirrirrg if tr1 C 1. But from Ex(n:is(: 8. g.yif Lt : Lz. L.tLz.2 r:an bc zrnswcredeasily. giverr any regular language .ter orr. t3 is rcgular. 11. Once wtl httvc: M we can theu use the a"lgoritlrrn in I'heorern 4. ffi 3. Exhibit an algorithm for tleterrnining whcther or not a regular language . Iittle. A larrguageis said lobe apa.J.Lz.1 we see thirt Lt : fr if ancl trnl. termines whether or not L : I.6 to cletsrrnirre if tr.for any regular language tr. 5._____ 4.irtd'. For tlgular languages.lr)Ns AIlotTTR..j is ettrpty. Show that there exists irn algoritlrnr to determine whether or not ur E Lr .2. ffi 2. For all the exercises in this strtion.the questions raisrld by Theorems 4.L) fbr any regtilar L. such that w : ufru. The operation tnil(L) is defincd as tail (L) : {u : uu E L. Exhibit an algorithrn that.. Anticipating a.t regrrlar lir. for any regular languages Z1 arrd Zl.L1.rne languagc if .o dctermine wlrcther or not ^L1: LtlL.L*.u..fi bc any string irr E*.L : .nttnltrnnv Qun$.t accepts 1. Let tr be a regular langrrage orr X arrd. I These resrrlts are fundamental.nrlrlvcrrtually irnpossible to fincJ.rrguages are given 1. show that there exists an algorithm for cleteruining if A e . Exhibit an algorithtr that. 4. with u. Fi'd an algorithm for dctermining if a given regular language is a Jralildrorne language.
n'n tn'ucu. There are several ways in which this precision can be achieved. S i n c et h e r e a r e a n u n l i m i t e d i t . .p be inflnite.cgular languages ca.L tail (L) . t'ind an algorithm for cletennining whether a regular language tr contains an infinite numher of evenlength strings' Describe arr algorithrn which. brrt only a finite nunber of states in M. Therr some dla M : (8' {a.L is regular. the infbrmation that has to be rernernbertxl zrt a. d. . when given a regular gramrnar G. but has to be shown prtx:iuely to be used in arty rntlarringful way. 3 . Let . b} . impOses Some limits on the structure of a regular language. . e*') : q. 2 . This is true. for any regular 12. If we put z objects into rn boxes (pigeonholes). ffiS 1 S .{anbn : rz > 0} regular? T}rc answer is no. PrinciPle the Using Pigeonhole The term "pigeonhole principk:" is rrsed by mathematicians to refer to the fbllowing simple observation. say q. show that an algorithm exists for determining if tr contairrs arry strings of even length. the pigeonhole rrurrrber urinciple tells us thtlt there must be sonre state.L be any regrrlar larrguage on E : {a. can tell us whelher or not L (G) : E.t have finite nrcm()ryl however.L. o t )f o r i : of zj's. as rlost of our examples have demonstrated.cns : show that there is an algorithm for determining whcther or uot .o") : q d* (q. . ancl if TL> rrll then at least one box must have rnore tltarr one item in it. N r ldentifying onreguloLonguoges R.fi'lll Efufffnfm 4rS Is the larrgrragetr .q.i. Intuition tells ns that a language is rtlgulir"r only if. such that d* (qu. . N o w l o o k a t d * ( Q o .114 Chopter 4 PRoenRrrus on Rncttl. in processirrg arry strirrg. . b}. as we show using a Proof bY contradiction' Suppose. This is such an obvious far:t that it is surprising how many deep result$ can be obtained frotn it.ny stage is strictly lirnitecl. Sotne ttarrow rcstritltions mlst be obeyed if regularity is to holcl. Thc ftrct that regular languages are associated with autt>mata tha. F) exists for I . {0..
L f o r a l l i : 0 . Let L be an infinite regular language. ' ' z* &g e) (4 2) i s a l s oi n . the pigeonhole principle is just a way of stating precisely what we mean when we say that a finite automaton has a limited merlory. contain a cycle. To accept all anbn. that is. uses the pigeonhole principle in another form. any walk of length z or longer must repeat some vertex. .ul ) m. the one we give here is perhaps the most famous one. such that ^'. 1 . The proof is based on the observation that in a transition graph with n vertices. and leads us to conclude that tr cannot be regular. 2 . it is convenient to codify it as a general theorem. (g.4. A Pumping Lemmo The following result. . Flom this we can conclude that d* (go. In order to use this type of argurnent in a variety of situations. But sirrce M accepts a"bn we must have d. known as the pumping lemma for regular larrguages. b "t : Af.3 InururrnyrNcNorrRpcuLAR Lnlrcuacos lfE with n t' m.b") : 6 * ( q . Then there exists some positive integer ?rl such that any w e L with l. But since there are only a finite number of internal states with which to do this. there are some rr and m fbr which the distinction cannot be made. wi * .b') : q7 €. . . an automaton would have to differentiate between all prefixes a* and a. F. with l r u l1 m . I In this argument. . an'). (d. can be decomposed as w egk. There are several ways to do ttris. and l s l> 1 . This contradicts the original assumption that M accepts a^bn only if n : rn. (qo.a*bn): d. .
rttnot regular.: iy"h thc rniddle part yields another string in . and lyl } t. .ffiEl] Cornider the set of statcs tlte automaton gocs through as it proces$cs ?rJ' say Q oQ t 'Q i' .gr:s. . Finitt: Ianglra..{^Jl. there exists a dfa that rexrogrfzes Now take a.r:annot bc putnped since prrmpirrg automrrtir:a.r. The pumpirrg lcrnrna.L such that dfn. . Ai z of ru such that r d* (qu. and so on.L is regular..tqrr. r d* (g". . Iike the pigeonhole argtrtnettt in Example 4... ' t Q f t Q indicatirrg there tnust be substrings 'x. . u) : q.rT iT anarbitrary .vc'stated here.ry ^.roliE/ nl irr the putnpitrg lemtna is to bt: taken larger tha'tr tlte lotrgest string. at lea"stone state must be repeated.qr. tfuis can always he dqne. so thir. We have givcrr the pumping lemma only f'or irrfinite languages. everyuuFiciently To paraplrrase of numberof repetitions p. Since this sequencehas exactly l.t rro strilg can be purnptld.'. with lrgl { rz * 1 : m. . . Therc is rrothirtg in lhe pumping lerrrrnit.r.Q f. 'l'he but it is Vac. z) : q. ( l r ' . completittg the proof of the theorem.r. is a. Since tr is assumed to be infinittr.as we is ftrr hir. Let such a Proof: If . l".lwaysby cotrtracliction.6. . Wtr say that the rtriddlc strittg is "pumped.ngrtages is ir.havtl states labeled qo. ) : e.aitrla. y isg_!L:e. (eo. . F}om this it immediately follows that . . d* (q'. it. Tlrl tlrtlorerrtdoes hold for finitt: lirrrguages.UAGES long strjqg in 4 cql_b9*b[aken this. which ca.gc regular' it r . d* (80' rY3z) : q7.Q2t.. as well as rf. rz) : Qf.116 Chopter 4 PttopsRrlns or RnculAR LANC. i .string trr in. The demonstration rrsed to strow that cert. Q r t ' .L.r.tt ltence the term puntping lemma for this result. arrd such a repetition rrnrst start tro later than the nth move' Thus rnust Iook like the sequcrrce Q o tQ l . nyzz) : qr.llv " creales an infinite set.nllc rrsc:tl proving that a languir. .'l+ I entries. although always tegttla.5*(qn.
[ltulnflg [.kes choicethat will make the. string .inst an opponent.rrd clearly not in . The correct argument can be visualiztxl ir...> t Thcrr thel string obtained by using i : 0 irr Erpatiorr (a.of length cquerlor grcatcr tharn rn. Asstrrnttthrr. I In applying the purnlrirrg l{:rnrna. it+ 1.r.2). proof thal the language is rxrt rcgulir.llyqrrite difficult) that any pumped stringJrnust be in lhe original larrguir. but wc:rkl rrot krrow what they are. we win the garrrt:.bleto choose tl so that the opponent is very of ) \ T IL\tvdlF** . Orr tlxr otherr hand.t L is regular.. tr and every 2. pick a. wc mrly be rr.ssumptionthat . rsr 'ryz.we must keep in mind what the theorem says.3 Irnrurrr'ylNc NoNREcLTLAR LaNcuacns IIT Even if we coulcl show (ancl this is rmrrnir. the punrping leurua holds fbr cvery ?1.4.nta.r. subject to lzyl f nz. the substring y musl consist c'rrtircly of a's..iirr game we play aga.t'. I'his conLradicts thc pumping lemma and thereby is indit:irtcs thnt the a. if the purnpirrg lernrnil e is violatecl everrfor orro 'iu or i. so thaL Lhe puutping lt:rnrnil must hold. srrb. Wrile we canuot forc:eLhe opponerrt to pick il pa..1. l3rl> tlrr: 1.gc. Givt:n ?7?.mountto a. 'Ihe opporrerrt pir:ks rru. rk:firrcrl irr Erlrir. there is nothing in the stalernent of Theoretn 4. Our goal is to witr Lhe gatrte by tlsttr. A strategy that rrllows us to wirr whiltcver the opponent's r:hoicesis ta. the pr.lse.8 that irllows us to r:orrrhrrleflrrm this that the language is rcgrrlar. t  /1 vdr'e . We cannot clain that we have reached a contradiction just bcr:auscthe pumping lenrrna. 2..L is regular rmrst bc fir. Wc try to pick i in such a way that Lhe purrrpeclstrirrg ti.r. Suppose lyl : ft.rmping lernma. 4. We do trot krrow the va. wirile the opponcrrt triels to foil us. wc ctr.L : {unltn : n > 0} is not regula.' Usirrgt]rc prrmping lemma to show LhaL.2) is 'Ll)O : t1711lrhm' rr. Step 2 is t:rtrt:ia. thcn the language cannot be regular.hodeE t&] Tlxtreftrre.nalways .jer:t 'ro€ . We are guirrarrttxxl tlrc existence of an nz as well as the decomposition rEz.L ancl lrul > rn. We have to assurnctirir"tthr: opllonent ma.'I'herefore. Irr this. but whatever it i$. we We are free to r:]rorisr: irrry ?1r.urin .rticulilr tltx:omprrsitirlrr 'rl. it hardest for us to wirr tlrt: girrntl. l'he opponent c:Iroosrls rkx:ornllositionrgra. to 3.L.lue of nt. If we can do so.is violated firr some speciflc values of rn.blishing contradiction of ir. is not in.l.L. There are fbur moves in the garne.tion (4.
we use i : 0.L.Jsurnc rnirk(r ir wrong rnove. then the opponent could have t:hosen a g with a.a F+tE J Z . the opponent wc:rt:to pick a:Q. for.rrcl that allows us to produce a..118 Lewcua.bh.1l1lly the purnping letntna we cannot ir..'$ on thc lcft tharr orr the right and so cannot be tlf the fbrrrt u.virilrrtion of puttrping letnma on our next rrrtlve. ir.. hor{ lro'/.o I I which is in L. We would also of fir"ilif we were to choosea string cxrnsistirrg all ats. The strirrg obtained in this fashion has fewer o.e restricttxl irr Step 3. In Step 4.:{wwfl:rueX*} is trot regrtltr. Nnl( th at' ili sltorla/ 7. trrrtl the requirement that he o f lle Jr) L0 t / t . In that case.+rtL. is irr L for all i. But any trrgument that assurne$thtr..5.n. the opponetrt need only pick U:ua' Now zr. If... Because of this choice.tuft. that the opponent will Ttr ir. 'til : ttrZ'o. we could not have reached a violation of the pumpirrg lerrrrna orr the last step. y..5 u.ah.q. forcing a choice of r..n.t thc oJrportettt is so accommodating is autornatically incorrer:t.rt:ftrrcL is trot regular.(e : /t (qna6t .tcvclr rrl the opponent picks on Step 1.. To dcfca. .t{ t {o k o{ ' l ffisrestric *cirtirblflbf t e d i n S t e p l J t o c } r o o s i r r g a y t h a t c o n s i s t s a's. s )< . Wha.r. atrd we lose.:(+lrl. flo.'n u). Tho..rl even numher of bts.n Figure 4.l j r'. wc carr always choose a34 aEshetynirr Figure 4.tus. tts I/g &#tra/ t / a 5 ' or h.cns Chopter 4 Pnornnlus or.in the case where wu pit:k 'u) : [tr?"'.f u. / a r n . say. F Note that if we irad chosen ru too short. then t 0 is a strirrg of odd length ilrrd therefore trot iu .'Rr.ha. /ht . . I .. It. 6 1 * t. g.
fo > 0} is not regular. Irr thc sarrrc wily. I Slmw tha.rt pick a g with all a's.use lrgtl cannot be greater than rrz.t L: { e r r l. If the opponent wc piclks. r z > 0 } is not regular.t ca. Now we get the string (ab)tt'a"'.l with ir. Civen rrz.L.ru)< n6 (u)] is not regular. Since we have cornplete freedorn irr choosirrg rrr.gg+cl*UrlrJ:! bqilltp_pg1! g{_tbg p_tlirrg nade up qf.nbe done with y. bo!l. thereby proving our clairn."g!lS.lJ Itrurq'r'rr.r7:.b)a).we pick a. r!h<rn' We now pump up. iqLl. b}.rgument. Ciiven Lhe opponentts choice for rrz. The choice of r does rrot affect tht.L ((a. Because of tire constraint l*El < ?r4. The languirge 1.rrchoost)rl :0 rr.ywcca. t:hoosei = 0 and get ir. using zi: 2.nything br.. Now.'+r.sour string ut: (ab\"'t7 a"'  which is in L. wc pick ut * tt"'b""t r.g.string not in . that is y:ak.b.we pick as 'u the string a"'l (unless the opponent picks nt 13. : {(ab)" ak : rt> h. Therefbre. beca. = lut E X* : rzo(. The resultirrg strirrg ?il2: 5*trlE6m+L is not in . ancl L is not regular is The larrguagtl 7.'ylNcNoNREcuLARLexcuacns 119 Let X : {a. Supposewe are given rn.4. the opllonent r:a. violated. The various .gir. in which case we can use aill as trr).. wc can dea.irr. the purnping lerrrrna. If our opponent picks so (gL:6.ny possiblu r:hoir:el rvltich is _Illl_t by the opponent.. let us seewha.nnotdo a.
the lartguage is not regular.l . but we krrow this language is not regular.lready cla"rsified. Ttrktl : h(a) : u.h.gt: l:{ar.bt. since for rn ) 2 and k { m we havc h!:.. . This ntay be much simpler than a direct application of the putrpirrg lemma.closure propcrties can be used to relate a given problem to one we have a.s which has length m.rnltu>(m. thereforc . This string is irr l.120 an. Show that the language L : { a " b k c ' ' t k I h > 0 . but it is even easier to use closure uuder homomorphisrn.L carrrrot be regular either.:nlI} is not regula.h(c): c lhF" h ( L ) : { a n t h t " n + k: n * k > 0 } :{aici.ngrrtr. I Show that the ltr.(b) a.i>U}. It is rrot clifficult to apply the pumping lemma. Suppose the opponent picks y srrth thtrt lul: h < m' We then Iook at z. I In some cases.k. clircctly. only if there exists a j su<:hthat mth:jl But this is impossible.1)! Therefore.h > 0 } is not reeular. Chopter 4 Pnorenuns or RnGtJt Latqcuecns decotnpositions of tu obviously differ orrly in the lengths of the uubstrirtgs.r.
y rr. ..C)onsequently. Lo * b * ) woulcl also be regular'.re to be congratula.kl n. tr I The purnlling lcmmil is difficult fbr several reasons. But everr if wr: tnaster the teclurique. (t Prove the fbllowing version of the pumping lemnra. ntl . The pumping lernrna is likc a girrnc with r:omplica. [f L is rcgular.rT fr arxl A { rn. whicli we have already classifit:das rurrrrcgrrlilr. Clhoosinga string with ri.tly. alsrl rrrxrd er. r.cns 121 I'Iere we need a trit of irrgemrity to apply the purnping lernrrra dirrx.tedrules.l and I : (rn * 1)!. We rnrrst be rnore inverrtive. tsut L1 : fntlhtt: rz > 0]. it rniry still be ha. Z ancl tire lilnguage I't:T ( r .. Its staterrrerrt is complicated. firrllpose L were regular. E l*2 will not do.4.tingthe conditions of the purnping lernrnir.tecl.u € Z of lerrgth greater than nr.*1)! possible since I'his is rrlwir. tirrx:s to generate a string with rn! + (i. Then.rd to seeexactly how to use it. The riglrt side is tirerefore an inttlgur. by I']rxrrclrn 4. since our opporrerrrt citrr illwalys choose a decomposition that will ma. there is a.ke it impossible to pump thc string orrt of the language (that is.good game. If yoLr can apply the purnpirrg lcmma.il Innur'lpyrNGNoNRECLILARLar'rcuA. irrrtl it is oir. prrurp it so that it has an equal trutrrlrerof a's arrd b's). correctly to sotne of the rnorc rliffir:ult r:asesin this book.t .n be decomposed as 'tD: frUE1 . Howcvrlr. Yon good strategy to win. thcn there is an nz such that. We can geL a corrtradir:tion the pumping lemma if we of r:ilrr pick i suclt that nz!f(i1)A:(rrr. vou a. ca. br.rt ihat alotre is rrot errough to plir.ys nt. rnuch more elegarrt way of solvirrg this prriblem.stray in applying it. If thc ollporrcnt now choosesa g (by necessitycorrsistirrg of all a's) of length h < rt.* I)k a's. : l+ 1 or n. Let us ta.1. Knowledge of the ntlqrs is esseutial. we purrrp r. cannot be regular. irnd we have srrcceedecl irr violtr. every .syto go a.: m.
' i . t ' . Fa. regular? 3 . ffi (u) f : {a.such t.L and cvery one of its decornpositions llr : I.t)l \J''.. \_/ Lrro.z€ X*. If .. o l " .1).^ l ' (a) I : {an : n } 2.8 as well us Exercisc 1. l_ D n"t*.lul ) tn.L is regular.(i) ordisprove the ftrllowing statement. such that the following holds for j every sufficiently lurg ur € ./t} . is the product of two pritre numbers} (f) . = {an : n is either prirne or the product of two or more prirne numbcrs} 6. Is.. n' is a prirne nutrber} . ffi . :2k (e) I : {an : n.. which irrclutles Thcorcm 4.L. r t _ . m {w:n*(ut):"r6(tr')} is trot rcgular. .as special cases. 2.I: (c) I : { o " 6 l o a: A t I n .hat.*+(b) tv tL : {a"' : tt' is rxrt a prime number} {ttn : n: A 2 f o r s o m eh > 0 } for some h > 0} 4 . 1..L:{o.12. rno* that thefollowinglarrguageis trot regular.ulrgi z'uz E L tor all i : 0..t f .t.r.rr. 2 Prove the frrlkrwirrg generalization of the prttrrping lemma. b } " } (fl *.ru)} ffi (f) t .. r r .o"* that the following larrguages are not regular.tltl."Itlttk I+} n+l} : (b) ..L fbr all i.. t .r22 Chopler 4 Pnopnnrtns oH ltr:ctrlan.tz with u. "nl_ _ .: (.{""r.(tu) t' nt (. Lar'{cuacns with lyzl < rn.""If trr arrtl .. '. therr there exists an m.l I ) : I or l f k} {anblak : n (.. : {ut : n. S l r o w t h a t t h e l a r r g u a g et ' : (i)e. lgl ) 1.'grl rn.{a" : n.gt*. . " . il'he middle string u can bc written asu:'rAz1 < with le... r}_1. l s l> 1 ' such that nyoz it irr . r . Rl: e { a .. Apply the pumping lcmrna directly to show the result in Example 4.t) ."L.{unn :'ur€ {4.Lz are nonregular lan@erov* guages. therr Lr U Lr is also nonregulat. ": . .r.b}} (g) L: t { t u r r l . j L'\fl nr la t (r] n>i1r\.L . (c) l."bt:n{11 (c) Z.*inc ou if the following larrguages X : {a} are regular ffi 1 oqK .g x n L:{*n'h = n t5*'i \ n> lL \ (\( * \ ..} " .
8.An.ncuI.l l : 2 } \fOJ l. i ' r ' i * (b) /.use the special nat. .h languagc .bl L J / $ rl+(L5. the languags 7 : Is necessarily rcgular'l ffi .  r. beca. make a corriecture whether or not it is rcgular.Lo.your conjecture. l"t Lr and . Show by cxample that tirc fatrily of regular larrguages is rrot r:losed urrtler irrfirrite urriorr. : { r . The larrguage assor:iaterlwith sut:h a graph is '/5.Lu is the union over the infinite set P. the infinite uniorr is regula.n infinite Lrut courrtable set. b}+ } e W ( 1_t:)A* (a) I :  ". : {a*b{ : rz "1."ht : n. 6}+ . : {w1c'u2 i Lurl.L2bc regular ltr. so tha.is a prirne nurnber} I (e) {a.geregular? [. ffi * 16. r .llyinfinite.b}. < 2nI ( I ) . I'he r+et walks is genera.. it will be c.}5. it tloes rrot irnrnetliately follow that L is regular.tNc. P E P A '. * t ' r .3 IuuvrrnyrxcNoun.ut€ L1. r ' p ) . ju)r + :iuz} € ( il{ J 2. : { a n b t u . 15 1 0 0 } ( e )r : { a " b r : l n .nguages.)fet f be a. L DeP whcrc P is thc sct of all walks through the graph and ru is the expression associated with a walk p. of .2 that the langrrage associated with any getreralized trartsition graph is regular.rrz {o.lenotedby Upep1. I : { a " b t : n > 1 0 0 . u.1>3.L . the followirrg larrguagesregular? {u'*rr.t in light of of Exerr:ise 15.r. Consider the argurnerrt irr Set:tiorr iJ.p. For each.4. Then prove .utr e LJ} {tu : Apply the pigeonhole arguuent rliret:tly to the larrguage irr Exarnple 4.ulcrs 123 Consider the latrguagesbelow. < t. arrd associate with ear. ( a ) Z . :  J L \/.h Sl} (c) .L.ure of P. lrl } lul} dS G4 tr thc following languagcrcgular'l ' "'l l . Thc smallcst sct containing every .''r i'rtrj'Lt1'u) {a..r the following langua. Show that in this case.wE la.1r) e {a. kr r l l l k : > 5} m ffi (b) fI : {a"blak:rr.{a"bl : nf I is an integer} (d) /. : jrLt {tnuwRui u.
L is not a regular all u e {u.Lz is regular? 19. .r.1. Iet .L be the set of that describe rectangles.. In the chain code language in Exercise 22. Section 3.ld}* language. show that .# *fipfr trom thls that .r.Chopter 4 Pnopnnrlrs or RncutAR LANGUAGES tr /+ the family of regular languagesclosedunder infinite intersection? ffi \L /\ dEJ Supposethat we know that Lt I Lz and trr are regular' Can we conclude .
ges are ttfft:r:tive in describing t:elrtilirr sitttple llatterns. one does not rrt:c:tlto look very fa. wc txrrrsider the importa. The rclcvartceof these limitations to programming larrguages becomes evirlt:rrt if we reinterpret somt: of tlte exatnples. ancl grammars.gos.'rrtbtlrship problem. Whilc rcgular langua.rriiliar l.o tnost of rrs f'roru ir stucl.renthesis a atrd a right parellthesis for b.rnplesof nonregular languages. illustrating the dqfirritions with some simplc: cxarnples. Next.nguirgcs.r fbr exir. we rliscoverud that rrot all latrgrta'gesartl rcgular.nguage therefore clescribes sirnple kincl of nestud stmcindicating that somtr llrollerties of ture fbund irr programmittg la.nt nx.y 125 .ContextFree Longuoges n thc la.ngutr. in prrrticular we ask how wt: t:irn tell if a given strirrg is clerivablefiorrr a givtlti graurtnar. If in I'or L: {q*\rn : rz > 0} we sutrstitute a left pa. then parenthesesstrings such as (0) and ((0)) are in L' tlrt a (0 is not.irririg ir setrletrce through its grilrnrnirtical deriva.tion is fir. In rlrrlclr to cover this and otlrt:r rnore complicated fuaturcs we tttLtst enla. by We begin this r:ha. flxpla.rge This leads us to considt:r contextfree langrrir'gcs the farnily of langJrragt:s. The la.st chapter.1lter clefining contextf'rtxr gralrunars a'nclltr. programmirrg lattguages reqrtirer sorncthitrg beyond regrrlar lirrrguages.
so a regular langrragr: is trlso a.rrgutr. Every regular grarrurrar is rxrntextfree.ngua.ges.milyof regular Iatrguirgesis a proper subset of the fanily of cotrtextfrrt larrgrrir ges grirmmar$ derive their trattrt:frorn thcl fa. A lir. To crea.. In computer science.rs..crrntextfree grarnmar.nclother translating prograrrrs. does rrot dt:pcnd on the . where. Parsing is a way of describing sentence structure. so wc s(xl tltat tlxr fer.S. The topic of contextfree languagesis perhirys the most important aspect of firrmal la. It is irrrportirrrt wh(lnevrtr we need to understand the meaning of a sentence.t tr : L(G).tegl'alrtlrrars thirt irlu rntire powerfirl.T.ct tha.while tlre right sicle has a spcc:ialforrn. Wt: ha. we get corrtc:xtfi'rxr M A gramrnar G: (V. a. ContextFree Grommors ffi. thclrc are nonregular languages.this is rclt:vilnt in interpreters. as we krrtlw f'rom simple examples such as {u"h"l.geis said to be corrtextfrtxr and only if there is a contextI.nguageshave many fealures that c:arrbo clescribed elegarrtly try means of contextfree languages. compilers.as we do frrr irrstirrrce tra. on the right.126 Chopter 5 CournxrFRno Lancuacns of natural languagesand is callexl parsing. we rnust relax sorne of tlrrlsc:rostrictions.nslating from one language in to a. lt. rxrntextfree one. We touch rrpon this briefly in Section 5.nguage theory tells us irtrout rxrntextfiee languages has irnportant applic:rrtiorrsin the design of prograrnrnirrg ltr. Actual progritrnrning la.nguagetheory as it applies to llrugrilmming la.rgrarnmar are restricled in two ways: the left side musl be a sirrglr:variilblr'. What frrrrnal lar. but perrnittirrg arrythirrg grarnma.11 thrrt this language can be generirtctl tly tr.3.t the subCouLext1'rtx) stituliotr of thc: variable on the left ol a produr:tiori t:ilrr be rnade any time sttch a. By retaitritrg the rcstrit:tion on the left side.4 e V and 'r t (V u ?).mffim The procluctions itr it rtlgrrlir.nother.ges well as in the constructiclrr clf clfficient as conpilers. if freegrarnmarG srrchtha.P) is said to be contextfree if rrll prodrrctions itr P have Lher ltrrrn A+il. But. variable appears irr il st:rrtential form.ve already shown in Exarnple 1.ngua.
r : ({S} . {o. A .rrdliuear grafiIIIIarS are clearly croutextfitlt:.tlt:r showthnt L(G): : {ab(bbaa)" hha. it is uot regrrlar S .l ori of the consecluenr.xample 4.abR. blt lirrc:ar.1 Conrnxrli*Rps GnalrunRs 127 'I'his f'eaturtr is frrrtrr (lhe contcxt).94. .a. A typical clerivation irr this gramma.Su + aaSe. a contextfree gramnlirr is uot neces$rrrily .uaBb. > 0} T Both of the above erxanples involvcl gral]lmars thirt ate not only c:ontextfree. sirtgle va. to it We is contextfree.l (J The gramma.a + aubSbaa ) 'I'his makes it clear that L(G): 'rrr {u. Longuoges Exomples ContextFree of H $*sqtrsf $.(ba)'' n. with productions S +a5a'.ruB.b}.:et rrllowittg only a.}. is t:orrtextfree. . r). The languagc is contextf'rrlc. n. R + bbAa.vt: to the rea.r is S + a.ria. S. symtrols in the rest of the senterrtiir. letr. b} . e {a. but as shown in F.5.bk: Lhe left sidc of the procluctiorr. Regular ir.5 + b$b.8. A4.bbu'tt'. but linear.
9 a. contextfree grarnmar fbr the language. .5r i a$rbltr. we need to protlurxl a.q. This is done with ^9ASr. and we get thu answer 5 .SrB.11 and we c:a.} is contextfrrxl. (b. lb show this. and b with lc:ft a. then add extril fl.L .'s on the left.respectively.l) We can sec the connection with programming larrguir.\. (ut) and no (u) } 26 (u) . The particular ftrrm of the grarrmrar given here was choserrfrlr the purpose of illustration. However. A +aAla. there are some sirrrple linear ones for this language. In Exert:ise ZE at thc end of this section yolr are asked to firrtl one of them.cns WWWW rhe language 7.. Thc language . there a.ASrl. aababb.pp Lerrc+u..:{a"b*:nlm. It is not difficult to urnjecture and prove that * L : {.remany other eqrrivalent contextfrrle grammars.r that is contextfiee. Take the case ??) rn. We can use sirnilar rea^soningfor the casc n { m. hence tr is a contextfree languager. e {4. B +bBlb. Some strings in . and ababab.mmar is contuxtfree.gesclearly if we rt:place c.5'bl55l.mple 1. the grammar is rrot lirrerirr. I :IIWiliWWilWWMW Consider the grammar with productions .nd right parerrthtrses. 71+ aAla. where u is any prefix of 'u. The castl rtf n : ?7? was solved in Exa.128 Chopter 5 ConrnxrFn. In fact. The resulting gra.').t (G) are abaabh. but not lirrrrar.b} : no (w) : nr. 5r . We first generate a string with an equal number of ats and b's. This is another gramma.nbuild on that solutiorr.a5rblA.
id to be leftmost if irr each step tlrt: leftmost varitr. a derivation rnay involve senwe tentia.ngtrilgt:s' Here again therel ilnr rriany other eqrriva. in the order in which variiltrles G : ({A. 4. rn.lfrrrmswith more thaln tlrrc variable.8*+A' getrerates language (G) = {aznb*' .8 .rntil ()hrrpter I befbre w(t (tirrr auswer this qrltlstiott' T Derivqtions Leftmost Rightmost ond In contextfreegrirfirilrarsthat a. In srrch(iases.> 0].Ab aab.riabltrs replaced in a specific order. L the that this gra. Conuidcrrrow the two dt:rivatiotrs s 1.44.+n 1 e'at'4 aaABb4 q.mrnar trt is ea. F\'om this we see that the two deriva.ced.irely in tlrc order in which t}r: productiols arc aplllied.r. it is rrot so easy to sexlif there are any lint:irr oiles' We will have to wait r.bl . have a chtlice Trlkc for exampletht: grarrrrnar are repla.ppliexl.. If in each step the rigltttnost va. In order to show which producliott is a. M A rlcrivatiou is sa.1 CournxrFRnn GR. we be often reqrrirc that the va.ttutrr.we have numbcred the productiotrs and written the appropriate mrrnber on the + syrttbol.ble in the sententia.5. 2. Io.l forrn is replaced. S + AB. P) with produr:tions t. The clifferentre is etrl.lcrrtgralrtlrla. we call thc derivatiou rightmost.8.a.sy sr:c to rz ) 0. irr contrast to Example 5. .ta i.bleis replaced. Brrt.rs. 5.5. To rem6vt: suclt ilrelevant factors.5} . A +aaA.renot lirrcar.3.ritr.c'Rs 129 incluclessur:hstrirrgs as (0) ancl 0 () () and is itr fa'ct tht: sct of all lrroperlv nestr:rlparetrthesisstructtrrt:s lbr the colrllnorl prtlgratntnitrg la.tsb.ntl 4 aaAB4 naB S uaBb4 aab \ s 4 . 3.tiorrs rtot only yield tht: sarne sentent:tl but use exactly thc sarne procluctiqrrs.
indt:pcndent of the order irr whitlh prodttctiotrs arc usud.lil. is by a derivation tree.7.A rightmost derivationof thc samestriug is + + S + aAB + aA + u.1 ).AB.schildren consistiug of the symbols ou the right side with the start syntbol of that productiorr.\ \_1/ WnsicJer the gI'aIIuIIar wit} prochrctions S +a. fi __+ Then + + + S + uAB + a. In a derivation tree.5. a uode labeled with a variable occurring on the left side of a production ha.Nil$.. a derivatiorr tree shows how each variable is replaced in thc durivation.re tertninals. The followirtg tlcfinition makes this trotiorr precise.n.bABc.hBh abAh+ abbtsbb ahhhh I Trees Derivotion A second way of showing derivat.P) be a c:orrtr:xtfieegramlnar. Let G = (V.\. For example. latrtrlerd and ending in leaves that ir. A +hRh.fii 'f. [Rlnfii.beled with the lcft sides of productiotrs arrcl irr which the children of a node rcpresent its corresporrdirrg right sides..iotrs. Al.ffi. Figrrre 5. Atr ordcred tree is a derivation tree for G if ancl orrly if it has the following propcrtics.Chopter 5 Cor'rlrjx'l'F nr:r: Lnrvc:uaclns Figure 5. A derivation tree is irrr orclered tree itr which rxrdes are la.hBhB abAbB+ abbBhhB abbbbB o'bbbb is a lefturostrlcrivationof the string abltbb.tf. 1 shows part of a tlcrivation tree representirrgthc prodnction A o. Bcginning with the root. .
sir. a2.belfrom T U {I}.. Tlre descriptive term Icft to right ca.tiotttree..2 is a partial derivatiqn trce for G. 2.3 is a deriva. a vertex with a t:hiltl labeled A ca.' .n.is sirid to be the yield of thc tree.e. thrr. and 5.9. Consicler grarnmar with procluctions S +aAB. 5. always ta. fr + Thc tree in Figure 5. is a sentential form of G. Every inttrrior vettex (a vertex whic:h is not a leaf) ha.first rnarlrrer. tttrexplorttd branch. and its chiklrt:rr are Iabeled (from ltrft to right) o.king thc lefttrtost. abbbbis a sentenceof I (G). 4.. ..\'s encoutrtered. The root is latreled . then P must conta.intr. A leaf lahtllcxl\ Itas no siblings.1 CoNTEXTFnnn 131 l . 4 A tree that has properties 13.t is.+ u 1 u 2 .Gnnurutnns 5. If a vertex has labc:l A € V. but itr which I docs rrot rrecessarily holrl and in which property 2 is replacecl by: 2a.1.$u\WWNNWW. a n . A + bBb.llrclductiou of the ftrrrn A . 3 . while tlrc tree in Figure 5. The string abBbB. The yielld of the second trtrtr. partial derivation tree.la. omittirrg itrry . The string of syrnbols obtained by reading the leaves of the trct: frotn Ieft to right. I . rtlA.W$I thc G. which is tlrc vield of the first tree.nhe givt:rrit precisemeaning.nhave no other children.trcl frotn V. Every lcaf has a label fiom V U 7'U {I} is said to be a. Tht: yield is the string of trlrrninals in the ordt:r they are ettcoutrteredwhetr the tree is traversed in a depth. Every leaf has a la.
S.9 r u. Also. P) be a contextfree gralilnar. this explicitness is a great helJr in making argurnetrts. Then fbr every € I (G).l derrivationtree for CJwlxrsc root is labelecl5. there is a corrcsponding partial derivation tree. is atry partirr. Like transition graphs ftrr {inite automata. this follows imrntldiately from Definition 5.rmethat for every sentential form derivablc in n. 't . First.tt)AcHs Figure 5. We do this by indrrction on the number of stcps in the derivation.Let G . if t6.lNc.(V. Conversely.3 qnd Derivotion Forms Trees Between Sententiol Relqtion Derivation treeu give a very explicit and easily comprehended tlescription of a derivation. we note that the clainred result is true for every scntential form clerivable irr one step. we must establish the connection between derivntions and derivatiott trctts.the yield of trny derivation tree is irr L (G).T. tlrcrr thc yield of fs is a senterrtial fbrm of G.' of G whose yield is ir. steps.'.2 Figrre 5. tlNrre exists a derivation tr*. Assr. Now any ?rrderivable in n * I steps .3. thongh. As a basis. Proof: First we show that frrr every sentential fortn of I (G) there is a corrcsponding partial derivatittn tree. Since S + u implics that there is a production .Chopter 5 CournxrFnnn L.
Since a clcrivatiorr trr:c is also a. n .but do rrot give tlx: order of tlx:ir applica. way that thc leftmost variable in thc tree was rr.!J: rAL V tlT. we get rr. Use the derivation ttee to find a leftmost derivation' show that the grarnrnar irr Example 5.ils. }rut we havet rrot cla. W 5. aud sint:t:the gramrnar tnust have llroduction A + a1a2' ' '(trrrLr we see that bv expanding thc leaf labeklrl A. it follows that c:verysententlcin I (G) is the yield of some derivation trcrr:of G anrl tha.titlrr.2 regular? Cornplete the proof in Theorcrn 5.9 is a serrtetrtial form of G' s.2. A € V . Conrplete tire argumerrts in F)xample 5.mtnar.. wc are Iecl to thr: rrot surprisirrg result that any ?tr€ I (G) has a.1. for Give a derivation tree for w : ahhltu.leftmost and a sce rightmost rlerivation (fbr cleta.2. with yield r&rfl"'amA: for all sententirrl forms. 3. gap itl the preceding discussion. in rr. Since try tlte iucluctivc assunrption there is a partial derivatiorr tree with yield :rA'g.4 does in far:t generate the language describerl irr Equation 5..1. .partial derivation tree we therefilre claim that the rlr. showirrg that the latrguage giverr is gerrerated bv the gra.Gnaultarr$ 5.bbaba the grammar in Example 5.. We will loave this a"stlrr exercise. . we r:arrshow that t:very partia. Fillirrg iu a. au gfus(:rvation whidr allows lltJ to close tr.1 by showirrg that the yieltl of cvery partial c{erivation tree with root .imefl that it a.steps. 1 . flowevcr' once wo have a. Is the language irr Exatnple 5. l)erivtltitlrr trees are ablt: to represent atry derivation.tiotr.ct that this tlrtler is irrelqvattL. partial derivatiorr tree whosc leaves a. firw details. aE ( v u 7 ' ) * . By cle{initionr any u E L(G) has a dcrivation.1 CoNTEXTI"IIIJE 133 must be such that S I r A y . I Dcrivation tretls show whic*r productitlrrs are userl irr o[ta'ining a sentclrx:e.lwaysexpantled lirst. rcsult is true In a similar vein.re terninals.u. By itrdrrctiorr. Exercise 24 at the erl(l of this scction). reflet:ting the fa.arrcl W E rAy +'IA1A2"'Q.t the yielcl ()f every derivatio[ tr:eeis irr l/ (G).lsohad a leftmost or rightrrrost derivtr.l derivation trr:c represents some scrrtential fbrrn. derivatiotl tree. wc catt alwaYs get a leftmost clerivatiott by thinkirrg of the trce as having been brrilt irr such a. l)raw the dcrivatiorr Lrcc corresponcling to the dcrivatiorr ilr Example 5.
L : *r5' {uorur. Find a contextfree grammar for head(tr).ul: z} show that the complement of the language in Example 8. Show that Lt J Lz is a contextfree language. L: { a ' b * : 2 n . /'\  ( L2/ Lct L: {a"hn: n > 0}. Show that the following. \ .L is the language in Exercise 7(a) above..q.I:{u.h>o). w h e r e .bl" : no (."h"'. 1 r r " < 3 n } S (*) I : {tl e {a. Section 4.h} ffi {a"6*o"n: n : rn or m I k} {anb*tk:h:nIm) {a'ob*ck:n+2m:t+) {a"b*ck : k : ln *l} W (f) f : {w e [a.l Lz is contextfree.n > 1].(ur)+ t}. Given a contextfree grammax G for a language tr. i rt.. Let .nInzL} (c)I: {a^b*:nl2rn} ( d ) . lul : e l.134 Chopter 5 ConrrnxrFRnr L. rn ) 0) (u) I : {anb* : rt.Lh is corrtextfrec for a"trygiven A ) 1. W . TndacontextfreegrammarforE: \/ x. 'i s a n y p r e f i x o fu r } r r (e) I : {w e {o. (ut) I n.mcuacps b f! Find corrtextfreegrammars for the following languages(with rr. For the definition of head. {o.h I n + rn) {a"b*ch:k > 3}.. show how one can create from G a grammar I so that l.c]* : nn (tu) + nr. 13.w): 2nt.. (w)] (e) I : (h) f: 9' /\\ (10) {a"h"'ck. (c) Show that Z and L* are contextfree. where . language is contextfree.. .. o. L .Lr be the language in Exercise B(a) antt .b} forthelanguage p= {anu^u?b : w E *11. *.17n + 3} ffi (b). rn ) {a^b ch :rl:m or rn !.> 0./ \J (a) Show tl::. b}+ .see Exercise 18.1 is contextfree. ) 0. fe) = head.Lz the Ianguage in Exercise g(d).tt) {a.\L). 14.b.1. b}* : n^ (w) t' nu (w)} ( f ) .{ T r e { a . S (b) Show that . b \ * : n o ( u ) > n 6 ( u ) . L: (e) L: Find contextfree grarnrrrarsfor the following languages(with n.rr. (a) I: (b).I : (c) tr: ( d ) .
C}' ( 24. A + ttB. is contextfree. then every u E L (G) has a lcftmost V ancl riglrtnost clerivation. with }j : {n.* that thc langrrage : {wicll)z i'u.b.481... Irrtuil. ffi for the set of all regular exJrressions on the Find a contextfrec grallllllar that carr generate all thc production rules for contextfreegrammars with T: {a. Give arr algorithm for finding sudr derivations from . properly nestetl strings irr this situatiort are ([ ]).. w aabbabba is rrot in the larrguaE.'inrl a simplc graurrrrar G for which this is the clerivation tree of the strirtg of Thcn find two ntore serrtettccs I (G)' rr:1ntrb' what one rrright mean bv properly rrested parenthesis stnrctures in/ Uu)n"ntte volving two kincls of pareflthescs. 18'. hrouo that if G is a contextfi'ee grammar. (tt ll) t()l' but not (Dl o' ((ll Using vour clefirrition.ivcly. BSb. show that thc striilg gau.tt)2e {o.*.rrrrrt"a.b} and V: {A.5. Civc a verhal tlescription of the language gerreratcd by this gralrlmar' 4ti9)orrsicter \_.1 Cot'lrnxrFRer:ClR.fa*"""^" a derivatiotr tree. Show a tlerivation tree for the string aabbbbwit'h the grarntnar g +. A * bBiiltr. I. .c generatecl by this 2O. Consider the tlerivation tree below.." the grarnrnar with prorluctions S * aaB. say 0 and []. Fincl a rrrrrtextfree glalnlnar alphabet {a.b}.l. ' L fuf) Sir. Show that the cor[plefirent of t]re latrguage in Exercisc 8(h) is cotrtcxt*free. t urit}. fi+Aa.\.nnrlunRs 135 16. give a t:ontcxtfree gramrnar for ge[erating all properly nested parelrtnescs.B. tr} . c}.
we can parse it in a rather obvious fashion. F'ind a lirtear grammar for the larrgua. If so. we will ca. with lul : h ) 1. If norrc of these rcuult in a rrratch with tu. Show that the dcrivation tree for anv ?rl € I (G) has a height h such that I o g i ' l5 h'< q + u. After the first rt)und. but in general. .136 Chopter 5 CournxrFRr:n Larqcu.gein Example 5.c. we again take all leftmost variables and apply all possible productions.5. we have serrtential forms tlmt can be derivcd by applying a single production. wtr svstematically construct all possible (say.tion Lr. In t:itscsof practical applications. Specifically.ll this the exhaustive search parsing rnethod.5 irr one step.q. l ond Ambiguity {ffiffi Porsing We have scl firr c:oncentrated orr the generative aspects of grammars. we start at round one by looking at all productions of the fbrm S+JDr finding all r tha. we are also concerned with the analytical sidrl of thtr grammar: giverr rr. arrd so on.tion of ru. we warrt to know whether ot rrot ru is in L(G).T.t can be derived ftom .1J. It is a ffrrnr of topdown parsing. the nrethod will eventually give a leftmost derivation of tt.P) bc a contextfree gramrnar such that every one of its productiotrs is of the form . we studied the sc:t of strings that c:anbe derived usirrg (J.tchru. On each subsequent rr)und. which we can viuw ir. Ihe tcrm parsing describes finding a se(luenceof productions by which a w ( L (G) is derived.ec: f'rom the root down. This gives us a sct of sentential forms. in which we irpply all applicable prod'ctions to the leftmost variable of every r.ls.ns 2.S.vethe sentential fbrms that carr be derivecl in two steps. Porsing ond Membership civen a string ru in r (G). An algorithm that can tell rrs whether'ru is irr z (G) is a nrernbershipalgoritlrrrr. Thus. then it rrlrst have a leftmost derivation of flnite lerrgth. If u € L (G). leftrnost) derivations arrd see wlrcther any of thern rntr. after the second round wtl ha. For referrlnr:e below. Let G : (V. we may want to find a deriva. Given ergr&rnmar G.4 * u. It rnay be that sorrrc of these senterrtial fbrms can be rtrjected orr the grounds that ur c&n never bc derived from them.string tu of tclrmina. some of thcm possibly leading to ru. we will have on each round a set of prnsibJe sentential fbrms. we go to the next round. 2 6 .sthe c:tirrstnrctionof a dcriva.
ftrr from further <:orrsideration obvious The last two of theser:irnbe removexl forms reason$.2 Pnnstllc .95.R. 5 +. S+$.lrrIctlIrv tBT S .9 + .5. S + aSb. This is certaitrly the cilse in . While the rrrelhod alwir.ahl)is in the larrguage gencrated by the gratnlnar rrnder consideration.\.q. 5+55+5.ws.95b. S+55+aSb. 5+aSb+ab. the which are otrtainedbv re:placing lcftrnost S in serrtentialform l with all applicable substitutes.s serious fla.9+bSaS. 2..9. . 3. I Exhaustive search ltarsiug ha.On the rrcxt round. fiom senteltial fqrm 2 we get the additional sentcrrtialfonns S+aSb+tr.oundtwo thetr yields selntctttial S =+ 55'+ S. l I tlre tr. there is tl rrtore pertirrrlrrt objection.ysparses au € L (G). severalof thesectr.9+ bSa. be Again.9. Sirnilarly. S+ab'b+u'bSab.nd string ru : aabb. .Round tlrregivesus 1. But even wfien effic:itlrrclvis a setxxrdary issue.rsirrgis requirtxl.5. 4. Tlrerefore e.+5 5 l o 5 b l b 5 a . we fiud thc actual targct string from the sequenc(] S+aSb+aaSbb+aabb. it is rrot to be rrsed where erflicietrtpa. it is possible tirat it never tt:rmiuates for strings not in L(G).no Atr. The most obvioqs orrtl is its terliolsless.rr removetlfrorn contentirlrr. S+aSb+aaSbb.
If we do Irot producbiorrs. so that we canttot tell *r.mple. As wc will see in the rrcxt chtr.l. If wtl examine Exarnple 5.l fornts. For each sentential forrrt.tion$we wanl l.Chopter 5 CoFrrnxrFnnn LRwcjunoes the method will go on producing tria.ct. Irr fh..witlt m : a.rs irr ir. B e V. sentential forrns indefinitely urrlcsswe br.pter. Each step in the derivation increases at lcir"st one of these.this restriction does not irffcr:t the power of the resulting gratnrna. this prorhrction can be ustld to clecrease scntcntia.9. cither pr:oduces parsing of trt. P) is a contextfree gralrrilrirr which does not have any rules of the fornr O*tr.r ca.h tirere are two types of produc.l the previous cxir. Thetr tltt+ cxhillrstive seart:hlla. It generates the language irt Exnmple 5. This is clear becauser length of the senteutial ftrrm grows by at lea*stone syrnbol in each rountl.sily when to stop.hl ba satisfies thc given requirerncnts.rsingis relativrly TIte probklrn of nonterminatiorr of exhaustive $c:irrt:h elasvto overcornc if we restrict thc: forrn that the grirrrlma. Proof. have arry srrr'.\ as well as those of the ftrrrn . Supptrse that G : (V. consider both its length and the number of terminal synrbols.7.ny signilicarrt wav.o nrlc out. The idea in this cxample catr be gtrneralized and rnade into a theoreur for contextfiee languages in genera.7. ot a tells us that nrr parsing is possible.Itorlr:ilrr be made into an algorithm whiclt. Given any Iil € latb]+. ftll any u € X*.7 without thc empty string.hb. A* R.n have.rsingmet. the exhaustive seitrch parsing metlxrd will althc ways termirrate in no more than lrul rounds.parsing or we know that w { L(G).then we have rttarry f'ewer dilTiculties. we see that the difticulty cotn(ts tiom the productions thc length of successivc ^5'* tr. Since neither the lerreth of a senterrtial form nor the rrtrmber of . After lruj rounds we have either produ<:eda. those of tlxl firrnt .rildinto it $ome way of sloppirrg.4 + B. pa. The gr*tnrnirr S' *+ 55 laSbl bSu la. where A.4 + .
tlut we can put nome rough upper bounrls cln it. wc refer to such a. its practical usefirlness is limitecl becausc the number of sentential ftrrrns generated try it may be excr:ssivelvlarge.tedby the gra.n be fountl in Harrison 1gz8 and Hopcrofb rrnd Ullrnan 1979.rnethod as a linear time parsing .nrJso on. is still cluitc iuefficient. .r.t. makirrg the cost of the mcthocl prohibitive. of rxrurse. There arer severtrl known rnethods to at:hieve tliis. Frlr every contextfree grarrrrnar there exists a.nrrot everr descritre thern wittrout developing some additiorral results.2 Pansruc ANDAMBrcurrtr 1SS tertninal symbols can excced lrul. A rnethod in which the work rises with the third power of the length tlf the string.2 ) This indic.xponential algorithttt.pirrsing rnethod which takes tiue proportional to the krrgth of the strirrg.5. antl a corlpiler hased on it woultl neetl an excessive amount of time to parse (lverr & nroderately long program. w() can have no more thnn jPl scntential forms after one rountl. and often the rrumber of sentcntial forms is rnuch snraller. a derivation carrnot involve rlore thtr. therefore.lwaysbe done.Equation (5. a. The const. we obsclved that parsing ca. the total numtrer of sententiir. .n be ester. bound. Neverthcless. + e f t * t l (5. brrt all of them are sufficiently conrplicatcd that we ca. while better than an e. no precise gt:rreral resrrlt ca.l forms cirnnot exceed M = l F l +l P l ' + .u cannot bc genertr.ngth of the string.f more eflicient parsing methods for contextfree gralllmar$ is a crlnrplicated mattcr that belongs to a course rlrr compilers.nnot involve more than ? lul rounds.blished.tes that the work for t:xhaustive seatrh parsirrg may grow cxponentially with the le. More detnils ca. In section 6.n algorithrn that parses any ut E L(G) itr a nrrurber of steps liroportiorral to lrirls.2) is only a. no ilrore than lpl? sentential fclrurs aJter the set:ond round. I While the tlxhaustive search tnethod gives a theoretical guarantee that parsing can rr.rrrrrar.l:tion .lforms are generated diffcrs from case to case. We wjll rrot pursue it here excr:pt for sorne isolated results.n 2 lul rorruds. What we would Jikc to havc is a.8 we will take this question up agnirr briefly. onc reasorr for not pursuirrg this in cletail is that evcn these algolithms are unsatisfactory.rrstivesearch parsing is vcry inefficient irr most cases. If we restrict our$clves to leftmost derivations. Exactly how many sr:ntentia. at wlfch ti're we either havera succerssful parsing or . In the proof of Theortlrn s.practical observation shrws that exha.
P) is said to be a simple grammar or $grammar if all its productions are of the form A +ar.but such algorithms can be found for restricted. * € V*. look at the exharrstive sea. we do not know any lindar time parsing methods for contextin free languages general.S __+ ald S + a.140 5 Chopter CorurpxrFRuu Lerucu. a) occurs at most once in P' The grammar g + c. but important.H ir A contextfree gra.fi. As we will see in the next section.'A*.2. the derivntion rmrst begin with S + a1AyA2. (G) can be parsed with an effort proportional to ltul To see this.7.mmar g + a. specialcases' ].q. they are of some interest. and starting with a1 olr the right. we rnust have S 1a4azBtBz"'Az"'A^.. where A € V..rchmethod ancl the string rl : tt1Q.a) occurs in thc two prodnctiolE ... Next. aS I While sgramma.cns algorithm. Illaily featlres of comtnon programming languages can be described by sgrarrrmars' If G is arr $grammar.9lbS$lc is an sgrammar. S .SS. and any pair (A. but sinr:e again there is at most one r:hoice.mmar G : (V. The gra. since there can be at rnost one rule with S on the left. a E T.tifii ltiffiffifiFil.a".r$ are quite restrictive.. we substitlte for the variable 41 .9lb55la$Slc is not an $grarnmar becarrse the pair (^9. We see frorn this that eaclt step produces one tertrinal syrnbol arrtl hence the whole process must be t. then any string trr in f.omplettld in tro more that ltul steps' .
In programming languages. The graurmar in Example 5.4 . unamlliguous fbrtn. rvhere there should be urly one interpretation of each statement. ambiguitv rnust be rerrtoved when possible.4.5. Figure 5. Often w(] can achieve this by rewriting the grammar in an equiva. _ l Ambiguity is a common feature of natrual languages. we s&y "the" derivation tree because of the possibility derivtrtion tree rather than that a number of different derivation trees may exist. A contextfree grammar G is said to be ambiguous if there exists some ut E L(G) that has at least two distinct derivatiou trees' Alternatively.is ambigrrous.4. This situation is referretl to as ambiguity. with productions . The setttence aaltb has the trvo derivation lrees shown in Figure 5. where it is tolerated and dealt with in a varicty of ways. ambiglitv implies the t:xistence of two or more leftmost or rightmost derivations.9 .aSblS^91I. "4" exhaustive search parsing will produce a clerivation tree for tu.lent.2 PARSINcaNn AMeIcuIrY 141 Ambiguity in Grommqrsond Longuoges On the basis of our argument we can claim that given any w e L(G).
esfora*b*c. IIowevr:r.rb + c arc in L(G)' It is easy to see that this grarnmar getrera. (b) Wonsider the grammarG: (y.5. to associatelrrecedencenrles with tlttl operalors * and *' Since + norrrra'llY has higher preccdencetharr *. The EI&IluIIar is arnbigutnts.. * T : { a . Tlre strings (a+b)+c and q. ) }.(E). c .P) with v = {E.nncuacns Figure 5. Puq(rilllike programming Innguages.T'E. * . we would ta.ke Figrrre 5. E .. as is done in programmilg marlrells. .5(a) asthe Correct parsing as it indicates thal b*r: is a subexpressionto be evaluated before perfortring the addition. 1alblc.n Chopter 5 ConrnxrFRnu l. I b*c ha^stwo differtlrrt derivatitln tree5. as shown itr Figure 5. b . the string u.this resohrtion is cornpletely outside the grammar.tesa. EElE. For instatrce. ( .I}.5 Two dcrivatirxr tre. ff + E. and productions E1. It is better to rewrite the gramntar so that only one parsing is ptlsuihle. restri(:ted subset of arithrnetic expressions lbr C and. I One way tq resolve ttre ambiguity is.rE.
taking V as {ll. t t Ttr rewritethc.n equivalent unambiguous grammiir. .11we introducencw varia.7. givt:rr contextfree granrrrrar is amtliguous or whether two given txrntextfree grarnrnars are equivalent are very difficult to answer. E*:E*7.tr'.*qfiplo S.. we will latr:r show thrrt there are no general algorithms by which thesc questions r:iut always be resolverd.5.but. irr general.6 $.6.1} and repla. In somr: instanrxls.mple 5. In fact. It also is cquivalent to the grrrrnrnar in Exarnple 5. I o. F +I.11.? Pansruc Arun $. the questiorrsof whether a.mhiguity is in tire langrrage.ce productiorrs thc with E+7. T ' .blcs. A derivation tree of tlrc sentenrxr lb + c is shown in Figure S. this is rrot possitrle becausc the a. No other tt derivation tree is possible for this string: the grammar is urrarnbiguous. It is not too hard to justify these claims in this speciflc instarrce. howevcr.+F .grarnmtlrin Exa. r In the foregoing examplrr the amhiguity came from ttre grammar in the sensc that it crruld be removed by finding a.lblr:. T+T*F. F + (E).lrercrrr'rv 143 Figure 5.
.13 The languagt: L .. tht: pther with.l 1lrff i.\.L44 Chopter 5 CorirnxrFR.ff .I'"I.5'+ sz.51 A .lfl. Brrt in sotne: exist Sorrr() hi1vecolflicting requirerncnts. Notice tha. somewha.trcl while the scctlnr.r:r:Luqcuaclli ni. tire first puttitrg a rtlstrictiotr orr the uutnber for does thcl seune bts il.'sa. and . of a. 51.llffl'''r' Exomplo 5.ri ifliif 11Spfi g1v611 If L is a cont(rxthee langurrgtl tbr which thcre exists an rtn1mbiguotls Brtt'mrnilr. is The grarrrrrra.teclby the t:ombiuatiorr of these two grarntnars with the additional produt:tiorr ii 51lS2.r ambiguorrrisiltce tlrtl stritrg A''bncn has two distinct o1e strrrting with s =* 5r.{a"bn t:"'} u {atbtttctt'} t It is a.. is to That tr is cxrntextfree ea. Thcrr L is gette*r."'If.. lt tloes of clerivirtions. B .Abl.nd c's.na..bEr:ll.. A ftlw tries will bts. is an irrhcrently anrbigrrous conttlxtfiee la[gllage.a.L2 is rlur.t clifficult tniltter even to exhiit art inherently aIII with rz alcl rn nonnegativr. clA..terJ LtU Lz.ty ['1attd L2 for other tronarnbiguoltsgraIIIIna.logous Sz.sy sitow.:]"".52 atrd llrtr by arr a.:flll::lil llH.t L: by L1 is generir. thcn the languirgtl is called inherently ambigrrous' rwi sor'': th il.l .rS it. tltett L is snicl to be una'rnbiguotts' If t:verY Branlrlrtrr that geueratt:s L is aurlriguous.aSzlB. course not follow I'rom this that L is inhererrtlv a'mbiguous a's there rnight w.:tious grermtnar with start syrrrbtll ..
t sBr'alnlnaris unambigrrous. Let G: (y. S * A E*b. Use the exhaustive scarch parsirrg rrrethod to pa.u.4 is ambiguous.l'r.rtt b). uniq'ely. but that the lnnguage denoted bv it is not.S. one proof cilrr be founcl in Flarrison 1978.5 is unambiguous.. S + aSbS lbSasl A L4. l'ind an sgrarrilnar l. Firrl arr sgramrnarfor L : {.t arr sEirarnmar tr (a. A rigorous argulnent) though.a. : {a''.a.l be neederl to parse any string ru in this language? ./l ancl l7l. In general. using the grammar irr Exarnple 5. 6. W 15. Show that the folkrwing gramlnar is amhiguous. D } . Clive an expression for the maximurrr sizc of P in term. Give the dcrivation tree for (((a "f b) + c)) * o f t).Il:tu E {a. ffi B. ABlaaB. Show that the grammar in Exarnple 5.ru 2}. 11.? Fansrrvc+ Auucurly arun l4E tluickly cortvince you of the imllossibility of combining thgsc requiremelts itr a single set of rules that c:over the case n : m. for a 2.b}} is not inhererrtty ambigu 16. E. Show that thc grammar in Example 5. alAa. ffi Give an unambiguous grammar that gerrerates thc sct of all regular express i o n s6 1 X : { a .rse ttre string abbbbhbwith the gramrnar in Exanrple lr. is quitrl teclurical.13 is ambiguous. {ffi 7.12. how rnarry routcls wil. Sltow that thc following grarrrrnar is ambiguous.r of ll.il. Construct an urrarrrbiguous grammar equivalent to the grammar in Exercise 6. Slxrw that the grammar in Exarrrple 1. 17.5.rr :. L 10. } fbr L9 show thal sno.P) be an ssrarnmar. that the larrguage7: {uu.n. Is it possihle for a regular grammar to be arnbiguous? tt li:* 13. r Gl "u."lt" : ru > 1}. Show that a regular languagc cannot be inherentlv arnbiguorrs.
Termilal symbols are writtr:rr illso uses subsicliary syrnbols such as l. (terrn) .programming precisely arld to use this clefinitiorr a.2. In BNF. but ::: is usetl instearl clf lir.T. in tlre grirrnmtr.rxlin the construction of itlterpreters nrrclcompilers fbr them. aqcl so o1. va.rrplc 1.ria. m Thert 5. s.ngua.re important in achieving this. This forrn is in essence are bqt the appearallceis diffcrr:rrt. ir.riable clescriptious of programrning icleltifiers to make the irrtcrrt of the production explicit.it va.nt) i. we rrefil c:ontextfiee langnages to model more cortrplic:attxl turpects.) + (/octor) . regular lalguagels irrc u$ed in the recognition of certain sitrtpltl ltirttl'rns but as we arguecl in the irrttoduction which occur in prograrrrrning la. rmrt:h in the way we have done. Let CJ: (y.ny parts of a Pascallikc progra.ges tend to use more t:xltlit. As wtl hirvtt $een. The basic problclll }rtlrttis to define a.ngrrages.12 miglrt appear irr BNF a*s I (erpressi.mtning language are susceptiblc to rlclinition lry restricted forrns of txlntext'fiee glammars. BNF.bles enclosedin triarrgular withorrt any special marking. But othcrwise there irre no significant dilleretrccs bctwtlen the two notations' Ma. tiorts of Theorem 5.f (erprt:ssi.on) (the'nclause)\elseclause) ' ::: .ngrr:lge writirrg of c:ffir:icrit a.: (/ar:tnr) (term.me. Thg symbols * ancl + are tetgrirtals. BNF brir. For exarrrplel. Both rcgrtltr. The symbol I is used +' RNF as an alterrrator iL'Jin our rrotatiotl. P) lre a cofltextfree Btailrrrlar iil wlrich every A e V oc(ilrft on the left side of at urost o1c productiol' (J is urrarrrloiguorrs.sthe starting point for the lar. Prove the following tesult.mmar. Itascirl ifthenelse statetnent can be tlclfirrtld as (i.ssion) (term) .r Example 5. wc (:irrl define a programming latrguage by a grir.. It is traditiorral irr writing orr pr()gfammirrg languages lo usc ir rxrnvention f'or specifyilg gIaIIIInars calltlrl the BachusNaur form or the sarne as the notatiorr wc havtt used here. As with most other latrguages. to this chapter.ncl reliable trauslation prograllls. Slxrw that the grammar in llxar. Fistl a grammar equivalent to that in Exa'rurple which satisfies the trrrxli Grummorsdnd ContextFree $ffiffilii P r o g r o m m i nL o n g u o g e s g One of tlte rnost imporlant uses of tlrtl tlxxrry of fbrmal languagesis in the clefinition of programmitrg languagcsa.on) ::: (terrnl  (t:nytre.r and cotrtextfrec lirrrguirgt. Thus.5 2 0 .14 is unambiguous' 1 9 .146 Chopter 5 CorurnxrFnnn LeNcuacns 1 8 .f state.r:kets.'str.
Thtl rules for (erpress'ion) abovtl irre rtot of this type.'irtteqer u&T 'r i ?n. otherwise a program mav yield very different results wherr processed by different compilers or rurr on different systems. tnrt for our purposes it sufficers realize that $rrch granulrars exist and have brlcrr widely studied. the issue of ambiguity take$ orr added signifl(jarr(ic. we see that this looku liktr arr sgranunar production.Laucuacns GnalrlreRs ANDPROGRAMMING 5. Irr connection with this. Keywrlrd$ not otily provide some visual structurqr that can guide the reader of a prclgrarn. so that parsing hecomeslt:ssobvious. whic:h have the ability to express tlrt: less obvious features of a prograrrrrning larrguage.r. This is not a.mmar. to Neither of tht:stltwo cotrstructs is a. AII other term$ are variables which still have to be defirrcd. such iLT . As Exarrtple 5'11 shows. sitrcethey "etrnirrtcger variable cannrlt bc assigtred violate otherr constraints.nn L47 Here the keyword z/ is a terminal syrrrbol.2. We see here a reasorr why we use keywords in prograrrntitrg languages.crccptirble a Pascal compilt:r. The variable (if statement) on the left is always associated with the terrnirral z/ on the right. r.4.blc to recogtrize attd remove arnbiguities.rt:syrrtactically correct in this serlseare in fact acceptatrleprogram$. irnptlssible in tlte most general scIISe'as we will see later' of Those a^spert:ts a programming language which can be modelul by a However. Wtr will briefly touch to on this topic in Chapter 6. If we check this against Definititlrr 5. a naive approach catr easily introdrrce a. z '. Ttl avoid such rrristakes we must he a. it contextfieel grallrlrlar are usuallv rerfilrrcd to as its syntax' is rrortnally the case that rrot all programs which tr. but also make the work of a. What we need for this purpose are algorithms for cletecting attd removing ambiguities in cx)rrtr:xtfreegrammars and ftrr tleciding whether or not a contextfree ltr.s 1)Qrr111 : r'eal. yet allow us to pilrst: irr linear time.r typical programtning language (jaII be trxpressedby an sgramrna. Unfortunately. For this reason such a statement is easily arrd efficietrtly parsed. extensive use has bcen tnade of what are called LL arrd LR grammars.3 Cox'r'ErxrFn. n ::3. A related question is whether a language is or is not irrherently ambiguous. ancl much of it is beyond thc scope of our discussion. The specification of tr prograrrlllring language must bcl urrarnbiguous. theutr irre very difficult tasks. rxlrnpiler rnuch easier. the usual BNF definition alkrws constructs such a. The question then arisesfor what grammatical rrrles we calr pertnit and still parse cflicierrtly' In compilers. For Itascal.nguirgeis inherently ambiguous.mbiguity in the gra.[egel".sitrple rnatter. Unftrrtunatelg not all features of.
5 . pa.s as elegattt atrtl of prograrnrrring lnngrrage semantics. aud Colsequerrtly some semantic features ilrav bc poqrly definecl or ambiguotts. Several methods have beel proposedr btrt none of them have }ee1 as uttivcrsnlly accepted and as successfulfor sernarrtir:tlcfinition as contextfree languages havtl bt''en for svntax. Nothing Prograrrrrrrirrg cxlrrt:ise contextfree gramilrars exists ftlr tIrc Specifica'tion a.ltics are a courplicatctl rnatter.rticular corNtruct." this kind of rule is part of progra. lrr. It is atr orrgging concern both i1 prograrnrrrirrg larrgrtages and itr formal latrguage thexrry to find effective rnetho. 4 .mming latrgua'gesenlalltics.Chopter 5 CorurnxrFRnn Lamcuecns a real value. Give a BNF Hril.s to clo with how we ittterpret thtr rntrir.nent) undefinccl)' 3.tion between a Pascal program ancl its srrbprogrants. Find examples of fcatures of c that <:annot be described by contextfree gtamIIIATS.ge sema.ngrra.r[rrrar t]rat shows the rela. 1' Give a complete dcfinition of \ex'pressiora)for Pascal. 2.te. . Give a BNH dcfinition fbr the Pa^rcalwhile statement (leavilg t]re general concept (sfc.ningof a. Give a BNF definititxr of a FOttiI'RAN do statement.Is ftrr tle{ining prograrnming lauguage scrtrilntics. Give a definition of ttre correct form of the ifelse stateuent in C' 6 . sinrxr it ha.
lent one tha.tisfies certa. In'I'heoretr b.Bccausc of this.S i m p l i f i c q t i oo f n ContextFree G r q m m q r sq n d Normo Forms I efore we carr study context*free languagesin grealer clepth. We itrtroduce two of the tnost useful of these. the chomsky norrnal form ancl the Greibach normal form. Thc tlcfirrition of ir contexlfree grirrrrrrriil irnpOses ntr rtlstrit:tiorr whatstltlver oil the right side of a freedonris rrot necessary.4 .B made the arguments easier. elirninating rules of the forn A . we Ileed to look irt rnethoclsfor' lrirnsfonning an arlritrary corltextfree gta. we saw the corlelielce of certairt restrictiorts olr gralrulratical forlm.mrna. a1y glirrrlrrlirr hlm itrr txrivirlertt trorrrtirlforrrr versiorr.3.nsftlrmir. [n this r:ha. An immecliate applicalion of the chonrsky normal forrn to parsing is giveri in ser:tion 6.tiorrs arrtl sutlstituticxrs thu. 149 . A 1orrnal filrrn is orrt: tlurt.inrestrir:titins on its f'rrrrn.A and . llrorlur:tirirr.pterwel strrdy $evrtrill trrr. However. is a detrirnent il sorne argumenls.lthough rr:strir:tc:rl.ypractical and theoretical uses.nces.l.is llroad errough so that.Both ltave tnatr. a. we must atttlrxl to st)rnt:te:t:lrrrit:irl rnirtters. \A/tl alstl irrvt:stigtrtc rrorrnal forrrrs f'rlr cxirrtr:xtfr(xi grarrrrrrars.t sa. . is it desirablettl place evell lror(l stringcnt rustrir:tirlnsrlrr tht: grirrrrrrriir. corrrplelter ancl in facl.t will bc rrscfirl irr srrbsr:tlrrclrrt rlisrlrssiorrs.r into a. In rnany irrsta.neqrriva.
tnaking Ss the start variable. For the rest of this chapter..tes.S. ttre process does not necessarily result in an actual reduction of the number of rules.they will be usedmany times in later discussions' Grommors ffiffi Methodsfor Tronsforming we first raise an issue that is somewhat of a nuisance with grammars and Ianguages in general: the presence of the empty string. What we meaJl by it is tlte removal of certain types of undesirable procluctions. Therefore any nontrivial conclusion we c8.91tr.P) be a contextfree grammar for L . (G) (see Exercise 13 at the end of this section).lcnrroN CournxrFnEE The somewhattedious na.The various conclusions significant. there is no difference between contextfree languages that include . Suppose that P contains a production of the forrn A +r1Br2.T. * genera.. as we see from the following considerations' Let .n make for . for all practical purposes.tureof the material in this chapter lies in the fact that many of the argumentsare manipulative and give little intuitive insiglrt. but we will use it nevertheless.\.{I}.t. Consequently. looking only at languages that do not contain . and it is often necessary to give it special attention. Here we give one that is very useful for simplifying grammars in various way.\ and those that do not. Let G : (V.rpl. We prefer to remove it from consideration altogether. there is a method for obtaining d such that L (e) : .L be any contextfree language. For our purposes.S.this technical aspect is relatively unimportant are and can be read casually.I 150 FoRMS AND GRAMMARS NORMAL or. Rule A Useful Substitution Many rules goverrr gerrerating equivalent grarnmars by means of substitutions.In doing so.L.4 and B are different variables and that B+atlazl"'la" . we will not define the tenn s'implffication preci$ely. and adding to P the productions 5o. and let G: (V. AIso. Then the grarnmar we obtain by adding to V the new variable . we will restrict our discussion to .96.\free languages.T. Assume that . given any contextfree {I} G. The empty string plays a rather singular role iil many theorems and proofs. {I} grammar will almost cortainly transi'er to . P) be a contextfree grammar.L. 6 Chopter Slr. we do not lose generality. unless otherwise stated.
t h e n . is *secJ again later.(*) =r(G) Proof: Supposethat .6. It follows thrrrr. we can repeat the argument. then look at the derivation the first time (6.the grammarin which F is constructed deleting by A . Thus S 4c urAuz 46 u1r1Bfrzuz +G utixr.rr L(G). we lose nothing by assuming that this is done immediately (seeExercise 17 at the end of this section). .r. . I T h e r e f o r1 f. The subscript on the derivation sign * is used here to distinguish between derivations with different grammars. n Then (6. we completing the proof. (E\. A production A .1) is used.ntBrz can be eliminated from a f{rammar if we put in its prace tnu" w e L(G). But with granrmar d we can get S 16 u1Au246 u1fr1!in2u2. € L ( G )'. tr 1e. then obviously S36.1 is a simpleand quite intuitive substitutionrrrle. . e t ew Theorem6. can strowthat if w € L(t).1 MerHoos poR TRnl.F) U. If it does.1) . Thus we can reach the sarne sentential form with G and d. lry induction on the number of times the production is applied. \ . that S46u.rsFoRMrNGGnelrlreRs LEt is the set of all productions irr P vrhich have B as the left side. The B so introduced eventually has to be replaced. Let E : (V.. If this derivation does not involve the production (6.1). so that e S 4c tr.r.gjr:zuz./ By similar reasoning.ntBrz from P. and adding to it A + rarrz lnraznzl.T. lrtanrz.S.
r d is equivalent to G. c} .Io.t nxqmpli Consider G : ({A.R. this can never lead to a sentence.luu.b. the protluctiorr . Retnoving this production leaves the Ialguage unaffected and is a simplification by any definition' .A4 aaabBc4 aaabbc in G. o.laRfi NOR\. atrd the correspotrdittg derivatiotr A+aaA+aaabbc ln (j.bltAclabbc. B + ahbAlb. In this result.AlabBc.ver take llart itt any derivatitln. While A can occur itr a string derived frorn 5. irr this ca*se. B + ahbAlb. The new gramma. \ l A .lrlgN OFCOlrrnxrli'Run GrtRMH. I _ Productions Removing Useless Onn invariably wants to remove prodlctions from a glammar that can ne. in the grAmrn1l wlxlse errtire production set is Sa ^ 9 b. FOr example. it is necessary that A and B be different variables.5 + A Clearly playS nrl l()ler a$ 4 CAnnqt be trarrsfortrtxl into a terminal string. l AaA. We will see shortly how ulch unnece$$ary productions can be removed from a gralrfnar. P) with productions A +a. are still in the grammar even though they can no longer play a part in any clerivatiorr.MS aNl Chopter6 SrurunrC. B} .152 Fc. A. using the suggestedsubstitution for the variable B.IRL the set of procluctions in which B is replaced by all strings it derives in one step. The string aaabbchasthe derivation A+ aa. the variable B and its a^lsociatedprgductions Notice that. we get the grammar d with productions A +alaaAl aba. The case when A : B is partially addressed in Exercises 22 and 23 at the end of this section.
[xornpl+ s'2 A variable may be u$e]essbecause there is no way of getting a terminal string from it. A aAlA.T. thert: is rro way we can achieve S # nBy.a variableis rrseful and only if it occurs if irt at least one derivation. . A+at B + aa. * (V. Although B ciln derive a terminal string.aSlAlC. the variable B is uselessand so is the production B . The casejust mentioned is of this kind.6. In words.d$$frqirbisli* liminate uselesssvmbols atrd productions from G S . (62) witlr r.rlrnns lEB Let G : (V.7.P).bA. E iinx.5.ble may be rneless is shown in the next grarnmar. A production is useless it involvesarrv useless if variable.. y in (V U7'). BbA. b}.P) be a ctlntextfree granrmar.'l Mr:'r'sor$ FoR TRANSFoRMTNG Gn.aCb. A procedure for removing useless variables and productions is based on reco6pizing these two situations.cI and ? : {4. with P consisting of C .S.a. where v : {s. A. Before we present the general case and the corresponding theorem. I This example illustrates the two reason$why a variable is useless: either because it cannot be reached fiom the start symbol or because it canrrot derive a terminal string. In a grammar with start symbol S arrd productions 5iA.rr. B. A variablethat is rrot usefulis called useless. let us look at another example. Another reason a varia. A variable A e V is saicl to be useful if and only if thenr is at least one ?rr€ tr (G) such that S# rAy 1w.
P ) b e a context*free grammar. In our case. andproductions S + aSlA.1. Next we want to eliminate the variables that cannot be reached from the start variable. 5 . \ / variatrlesor productions. The formalization of this process Ieads to a general construction and the corresponding theorem.. However. we identiff the set of variablesthat carrlead to a terminal r*ring.A} . I L e t G : ( V . thus iderrtifying it as useless. terminalsT : {a}.. we can draw a dependency graph for the variables. Then there exists an not contain any useless equivalent gra*rrrur I : (t. Baa.aSlA. we are led to the fi. A dependerrcy graph for Vr is shown in Figure 6.s. A+0. are led to the grammar Gr with variablesyl : {.. A variable is useful only if there is a path from the vertex labeled s to the vertex labeled with that variable.9. 7 .Chopter 6 SlvtplIllcRtloN oF CourpxrFREE GRAMMARSAND NoRMAr. i : {o}. F) with fr : {S. the variables A and B belong to this set. A +a. . For contextfree grammars' a dependency graph hasits vertices labeled with variables. becarrse + A for c.F)tnutdo*.1 shows that B is useless. For this. B}. and productions S . Removing c and its corresponding we productions.i. Removing it and the affected productions and terminals. Fonnrs Figure 6. with an edge between vertices C and D if and ontv if there is a productiotr form C * rDy.nal answer e : (t .S'A.this argument cannot be made S So cloes. s. A Becarrse + a and B r oG.1 @ First. Dependency graphs are a way of visualizing complex relationships arrd are found in many applications. Figgre 6. *# a.fr.
Thc remaining issue is wlrether every.Level. wc draw the va.'rr. in ck*r. Set V1 t.l rltlrivation trcc correspondirrg to that dcrivation (Figure 6. Figure 6. tht:re are only terrnina.l answer t fro* G1. to lbr every A e V for which l.tionv/ith Gr.I1 (yl u "). 2. we get the lina.riirlile at level a . R. At level A.rldecl.6. The third Lirne throrrgh step 2.4 $ rr : ah. . cxlrrsiderany such .riable dept:rrdencygraph ftrr Gl and from it find all rtnz' .1 will bel aclded to l/1 on the first pa"ss through step 2 of the algorithtr.T.evelr 2 .. The steps irr the algoritirm irre.ncratecl from G by an algoritirur consisting.4 ancl look at the partia.&1 Levcl. Irr thn seconclpart of the constructiorr.'.2 \.l .Lo€T* is possible.5.k* 1l as all thr: productions irr P whose syrrrbols are rr.2 will thr+n be added t. The algorithnr cilnnot terminir.so cvery variablc Aa at level A .reva. of twtr parts. 3.s the in trcc that are nrt yet in I.riablcsA for whir*r A4.T2.riable. Ta. It is eq'ally crear that if A e vr.epeatthe ftrllowing step rrrrtil no rnore variables are arlclecl v. and so on. 1.r'ly this prornclure termi'*tes. Hence A will cventually bt: adcledto [.4 .tt fr.{ ) . tht:rr A 4 ru e 7* is a possitrle cleriva. In the fir'st paxt we rxrnstlLrct an intt:r. all variables at level a * :l will be rr.1 Merlrops non TRarqsFonMrNc GBenruens lEF Proof: The grammar I can be gr.terwhile therc a.P1) such that V1 contirirrsonly va.ls.mediate graqrrrrar G : 1 (V. Ttr see this. has a.ondpass through step 2. prorluction of thc forrn A adrl A to I/1.t with all irr in yr U ?. Any va.4 f'or which.2). v1 on the sex. is acrcled v1 before the to procedure tclrrninates..
frorn G by tlrt: rtlrnoval of prcitlucltious. yut have sornt: the .lprc. r ? + $ i.f.seq*cntly 1.F\.irringtr. / ^ syilrtlols or Becaust:6f the colstnrr:tion.ble for which the deriver.Jf7 o u r s f iF fll'ili!.i?. The grirrnma.tiotr A+A is uossible is crallerinullable. * : i { .ny ternrirurl that d.ic.rrcl arc ttqrtivaleut' I 1. gralrllrlirr of the folrn Arry producliorr of a.t F ^ \ / G togethr:r.' proclrrc:tions'we all $iuce the c:orrstructionof d retains A a.riable Set. { t f" Removing A'Productions One kincl of llroc]uction tha.te tr.\llrocJucliorisor: nullir.. d ir t:orrstructecl g p.s. for each tu E L (fJ) wtr have a clerivation SltAylu.(e) e LG).ched va. (ri3) A.t is sornetirrrcsunclesiratrleis otre in which the right sicleis the ernpty string.riilr"rr i1t..t$ frotrt 5'.riables tha. Also.ltlt:vtlriables.()o. [rt: renroved. These artl removed frorn the va. Putting the two rt:srtlts sr tha.l assoclia.\production.oesrrot cicclnrin sotrrc rrseful prothrction.\prorluctious cillr ..tr. graurrrrilr tnay gt:ritlrate a lauguir'ge not clonta.. The rcsult is ttleglarlrrllirru:\v ' A t*.ted Irave everyttring ueeded ttl rnake the tlt:rivation S t6 nAy 11u ru. G does not contain any trseless procluctions. we see that G a.cotrtextfi'ee AA A is calledir.nr"1f..r56 nnn Chopter6 SrN{1'1.rctions involving them.fi .cr.roN OFCoNttlx'trli'ttEECIRAMMARS Non.lral Fottr. Irr suclt cascls.1 t . .t c:aunot be rtler. Auy variir.rrr. Wtl t:an also elirnina.rethe proth. it$ a.
tirere will he oue prorluction in F with fir r()placecl v/ith ..'oR TRAN$FoRMTNC. marrner.). prrt A into H1y. A in all pousible corrrbina.he following step until no firrther vrrriables ir. > i}.1.swell ar.ted.\productions. . similar.1 Ml:'rHops l. I Let G be arrv r:otttextfiee gr:r.s th<lsegenerated lry repla.n be removed after addirrg new prtiductions obtairred by srrbstitutittg A for 51 wherc it occurs on thc right. and onc iu which both ri and 11 a. ca.\ not in L ((.frm. Gnelrrverr. put B irtto l/r. a.readrlt:d to V. ro. For t:ach srrr:hprocluction of P. where . althougtr rnore r:ornplicil.tes Afreelanguage {a.n.. 51 + a51blA.mple.\.An art:in !fu. Repeat l.\. Onc:c the stlt V7lr hirs been founcl. 1. 2.cing nullable variables witlr all Fcrl exa.( V U ?. Fbr all productions A +.4142. 'Io . . wc put into F tn.41.if z1 and r:.ble variables of G. we are ready to construct F. A. we look at all productions in P of the fbrrrr ul+aDrfft". one irr which ri is rcplaced with A.rnmar with .Az..6. Irr rnore grlrreralsituations.. Doing this we get the gr:Irrimar $'5r aS'rblab.t:ed with . Proof: We lirst find tire set U1s of all mrlla. 2 wherc each nt. substitutions for .$rblab' We can ea"silyshow that this n()w gramrrrar genr:rates the samr: language:as the clrigina.s 167 S n aStb.\.rerepltr.rgenera. For all nroductions fi _..tions.. usirrg the fbllowing strllls. This grammir.i are botlt nullable.nt procluction ir. 'fhen there cxists an eqrlivalcnt gramrnar I having uo . T}re ..nb" : n.\protluctions can ht: rnade in a.'. .l otre.\production the 51 + ).rrrl1..
E.fii Arry production of a. folkrwing the secondstep of the construt. fl.i$fi'f.ble.\ is rrot Put into P.wc get AaClAB alaC lAalE o. in Fhomthc first stcp of tlrc coustntctiorr Theorern6.blA. we lirrrl that the rurllablevariablesare A..la. Bb.initien'0.the prodrrctiotr A + .\productionsequiverleut the gratrruilr defitrtldby S . contcxtfree grilmrrrirr of lhc fbrm A >B. The argumerrt that this grauunnr d is cqrrivalerrt to G is straiglrtftrrwarcl atrtl will bo left to the rearlor.ABaC.i iiil.4. S . prrlrfilctious in which both sides irre a single variable art: at tiurcs uncltlsirahle.Cl A . ('+ D. .BC. where.158 AND GRAMMAR.\.3. I RemovingUnitProduciions 'fheorent As we see from 6..qltoN r)r' CoNrnx'r'lrnEE Tht:re is onc exception: if all r.BlClBC.ABaClEn. B . .rernrlla. D + tl.r to Firrcla t:rintextf'ree without .2. B €V is called a unitproduction.sI NoRMAI Fon. I grarrrma.. C +Dl. Therr.vs Chopter6 Slrrrurlc. D*d.tion. C. '1 .
U* is the set of all rulesin F with B on the left.this carrbe in done if we proceedwith somecare.any unitproductiorrof the form A . Then there existsa contextfree grammar e: (t.lgl ' . whereB + A1lyzl. To showthat the resulting grarnmaris equivalentto the original one we can follow the sameline of reasoningas in Theorenr6.5.\productions. all variablesB such that t4 n.!r.y t l a t l" ' 1 a " . But this will not alwayswork.S. the utritproductionsare not rentoved.1 Msrnons noR TRehrsFoRMrNGGRerr. Note that sinceB . Next. for all A and B satisfying (6. Proofr Obviously.B.F\ that doesnot / have any unitprocluctionsa"urlthat is equivalent\toG.4. we add to P A .A can be removed from the grammar without effect. is takerrfrom F.1. in the specialcase A. we use the substitution rule discussed in Theorem6.\ to replace A. for each.1 directly with rr : nz = .To get arourrdthis. rroneof the yacan be a single variable.i..4).4 anci B.f .. ly. then (6. . As the construction the next theoremshows. I .B witlt Autlazl"'1a".and we needonly considerA + B.1.4 and B are different variablerl.7. P) be any contextfoee grammarwithout . it may seemthat we carr use Theorem6. The new grammur d ir generated by first putting into F ail nonunit productions of P. U4. we first Iind. Let G : (V.At first sight. so that no unitproductions are createdby the last step.4) holds whenever there is a walk between . (6 4) We can do this by dtawing a dependency graph with an edge (C.rRs 159 To removeunitproductions. D) whenever the gtammar has a unitprodtrction C + D.rr\l. where .
160 Chopter 6 SturlrnceuoN oF ConrexrFnnE GRAMMARSAND NoRMAL Fonns Figure 6. n +bb. Note that the removalof the unitproductions has made .3 o @[] . Aprocluctions.L be a contextfree language that does not coutain A.4.a lbclbblAa.\productiorr$. A + albc.S + E.9 + . and unitproductions. .alhclbh. the new rules S . . Let . and . :J G x o m p l6 . we seefrom it that . 6 i Remrtveall unitproductions from S * AtrlB.B and the a*ssociated productions useless I We carr put all tltese results together to show that gramma"rsfor contextfree language$ can be made free of useless productions. fi+bb.L and that does not have any useless productions.we add to the origJinal nonunit prodrrctiotrs ff+Aa. B _ Albb. + B albblbc.3. A + alhclB.theequivalent 5 . or unitproductions. The dependencygraph lbr the utritproductionr is given in Figure 6.4 4 B. Then there exists a contextfree grarnmal that generates . B * albc' grammar to obtain. A + albblbc. B + A. Hence.
and doesrrot createtrproduc:tions 15 productions does not create Aproductionsor urritthc removal of useless productions(Exercise16 at thr: end of this sectiorr).rtlra.5 :+d rlj implies D . In Example 6. are equivalent.1.\productions can createnew unitproductions. I  1" Cornplete the proof of Theorem 6.an.ve Aproductions. and the theorem is uroved.1 by showing that . RemoveAproductions 2. A + o. .4 removethese kinds of productiorts trrrn..4 requiresthat the grammar ha. Also.Gn. The only point that needsconsideration in is that the removal of one tyJre of production may introduce productions of another type. Show that the two Erammars S + abABlba. Removeu$eless The result will therr have none of these productions.using both the original and the modified grammar.ns 161 6.aNsFoRMING given in Theorems Proof: The proceclures 6.aAlbh and S + ahAaAlabAbblba. ffi .we can renroveall undesirableproductions using the following sequence steps: of 1.l B .2.But note that the removalof unitproductions no (Exercise at the end of this section). A + aaa. Therefore. 6. Mnrsons roR TR.+G: tlr.1.3.the procedurefor removing . Removeunitproductions productions 3. show a derivation tree for the string ababbbac.q. and 6. for example. Theorem6.
A + aaAl.rtoNoF CoNroxrFREE GRAMMARS NoRMALFoRMS AND 4. 10. Eliminate all . D .4. B + bBlbbC. In Theorem 6.aSlAB. A * aBlA.cCD. BAAWhat Ianguagedoes this grammar generate? productions from 6. What languagedoes this grammar generate? ffi L Eliminate all unit productions from the grammar in Exercise7.ddd. B + b6.aAlaBB.3 to removeA*productionsfrom the grammar in Example 5.rnce. Eliminate useless g + alaAlBlC. why is it necessaryto assumethat A and B are difierent variables? productions from the grammax 5. B*Aa. Eliminate all useless S . Use the construction in Theorem 6.\productions from SAaBlaaB.41.4. What languagedoes the resulting grammar generate'l . from the grammar all uselessproductions. C . Completethe proof of Theorem6. 11. and all Aproductions S . 7. A*I.1. 4bA. L Remove all unitproductions.\.\. CB. 12.L62 Chopter6 Slupr.3. Completethe proof of Theorem6.
nity : A+uEP E {1 + lul} . we obtain a new grammar d such r h a r z ( S ): r ( G ) .\productions and unitproductions? . 1 6 . give a counterexample. 1 4 .. then the removal of useless prodrrctions by the construction of Theorem 6. Reverse the order of the two parts. then the resulting grammar is equivalent to the original one. but possibly with somc unitproductions. It is possible to define concept of complexity of them is through the For example.S. Suppose that G is a contextfree grammar for which I € . Show that the rernoval of useless productions always reduces the complexity in this sense. ffi 1 5 .1 Mer'sons FoR TRANSFoRMTNG GnertlreRs 163 1 3 . Suppose that a contextfree gramrnar G: the form A+fry. Prove that if this rule is replaccd by A+ By.{ .4 does not then introduce any. one length of all the strings giving the productiorr rules. 2O. Show that if a grammar has no . Give an example of a situation in which the removal of .T.ve it. Let G be a grammar without Aproductions. Does the new procedure still work correctly? If so. where "B { V. wc might the tcnn sirnplificatiorr precisely bv introducing the 'Ihis of a grarlimar.6. therr renroving those that do not yield a terminal string.2 does not introduce any such productions. Show that if we apply the construction in Theorem 6. use (G) compt. can be done in rrrany ways. If not. 19. Justify the claim made in the proof of Theorem 6.\productions and no urritprodurtions. r8.1 that the variable B can be replaced as soon as it appears.L (G).P) has a production of where tr. Oonsider the procedure suggested irr Theorerrr 6. U .e. l/ € (y U ")t. Show that the construction of Theorem 6. first eliminating va"riables that cannot be reached frorrt S. What can you say about the removal of . prt.3. B+x. ffi 1 7 .2 for the removal of useless 'l:pfoductions.\productions.\productions irrtroduces previously norrexistent rrrritproductions. (V.
s.4 is not a suIfix of any :qi.. Prove the following courrterpartof Exercise22. .laEcl. .2.rr'r'ruxrFR...T.!!rlazl .4 on the lcft lre divided into two disjoint subsets A+ntAlr.T. where fii. Use the result of the preceding A + Au.. Divide the set ef productions whose left sides arc soilre given va. fr '* rr. into two disjoint subsets A+ and AnllA*rl'..Bblbc so that it no longerhas prtxluctionsof the forur A + A:r or IJ * Br.n't.. . t ... .n' is etlrivalent to the original grarnmar..L if (G) < torn7tlerZtlv (d) for arry d gcnetating f.164 GRenuaRs nNo Noti[4A.2. ? . A *. ..l*"A and A + . Let G : (y. A contcxtfree gramrrrar G is said to be rrrinimal for a giveu languagc . Lct the set of productiorrs involving the variahle.r7i are in (Ill?'). Prove the following resrrlt. Show by examr:omplexi.. 1 u .1. /).'lA*"..tlp: ?1.r 23. 1. i :1. Show that the grarnmar ohtained try rcplating these productions with A * 'ltrlZth.. gramrnar d: but A is not a prefix of any gl.rrs that have A on the left by A + yil11iZ. ffi *ZZ. s t l a z.tAl .t11 plc that the rernoval of useless produt:tions does not necessarily protluce a minimal gramnrar.S.. Consider the where Z f V and F is obtained try rc(u J{Z}. *24. .l.r t .F). to exercise rewrite the grarrrma... .lZnt..!t. placing all productit. : 1 . 'i : 7.riablc (say.. rhentr(c):r(d).n1..". z: Z + xil'tiZ.\ B .L FoRMs Chopter 6 SrllplrrrcATroN oF C<. .P) be a contextfree grammar. wher'e .2.
g'.$AIb iu iti Chornsky norrnal forrn.l form if all production$ at'e of thc form A _. OI A _+ ill whcre l. br:c:ause their widt: usefhlrress. A is rrotl both proclucticxts5 Dt:{irtition 6.ptl a is in 7'.rrestahlish fot coutextfi'ee grirrrrrrlars. fhe gralrllnilr fi .l Fonus 165 iffiffi Two lmportont Normol Forms Tht:re are nrany kinrls of norrnal forrns we cir.45'la.4.rrlook ftlr is one in whit:ir Lhe mrrnber of strictly liurited. a.. uR' funllfi .'. havtl beetr studof itxl extentively.a violate the conrlitions of I AA5 attcl zt  .ftftilrltiuw A contcxtfree grarrlrrlirr is in Chornskv rtorrna.r() can ask that thc: string rln the right of a productiotr consisLof no trtore tirirtr two symbols.2 Two lr. ltr particular. S Ala'tt' u.tponrenl Nonna.ymholson thc riglrt of a prodrrc:tiotla. A .6. Ci are irr I. B.BC.ASIAAS. wc s. Chomsky NormqlForm One kirxl of nrlrrrral forrn we cir.Sone of these. One instance of tltis is the Chomsky normal form. Wc considr:r'lwo of thettr bricfly.
Pl) from G by consideringall .. For every Bo we also put into Pr the production Ba + a' This part of the algorithm removesall terminaJsfrom productions whose right sidehaslength grea. At the end of this step we have a grammer G1 all of whose productionshave the form Aa. we can assume of without loss of generality tlrat G has no Aproductionsand no unitproductions. terminal since we have no unitproductions.syconsequenceof Theorem 6. where Ct = rt if z.. It is an ea. Proof: Because Theorem 6.T. productionsin P in the form A+ntrz"'frn.+C t C z ' ' ' C * .166 Chopfer 6 Stlrpllrtc.CrCz. F) in Chomskynormal form. .1 that (6 7) l'(Gr) : L(G).P) with ^ # L(G) has an equivaAny contextfree lent grammare : (?.q. (6 I'r) where each 14 is a symbol either in V or T.ter than one.rroN oF CourFxtFnEE GRAMMARSAND NoRMAL FoRMS grammar G : (V. introduce new variables Bo for each a € ?.5) we put into Pt the production A . S. If n > 2.replacingthem with newly introduced variables. (6 ri) A . s.5. put the production into P1. ancl Ca: Bo if ra : s.i. For each production of P in the form (6. .. Step1: Construct a grammar G1 : (V1 T.S. where Cl € Vr.Cn.' is in V. In this case. If z : I then trr must be a. The construction of G will be done in two steps.
to Chomsky normal form. : Dnz + CntCn. r ffi Sxnttpl+.D2.Ms 167 Step 2: In the secondstep. all productions of the form (6.the resulting grammar d ir itr Chomskynormal form.BoBoBa. so that \ . This somewhat informal argument can easily be made more precise. we introduce new variablesBo. B + Ac./' .fi Convertthe gramrnarwith productions S + ABa. For n j 2..ABB. Bo '' a. In Step 1. Ba. Obviously. Repeatecl of applicatiorrs Theorem6. the gralnmar doesnot have any trproduction$or any unitproductions. A .... Ba+c' . and usethe algorithm to get S . (") : LG). put into P the productions and A > CtDr.? Two lMponrerur'Nonrranr.6) as well as all the productiorrsof the form (6. A + aab. B + AB. As requiredby the constructionof Theorem 6..1 will showthat f (Gr) : . (.d).B.. fi. we introduce D1 new variables .6. Dt * CzDz. we introduce additional variablesto reducethe First we put length of the right sidesof the productions where necessary.6. We will leave this to the reader. Fon.7)with n:2 into F. Bn*b.
Bu.BRo. we $ee that the form A * o. a Q.ch norrnal forur a generality not by lrossessed sgrammars.l result SADr. Nevertheless.id to be in Grcibach rrormal forrn if all productions havtl the forrrr ! + o. Here we put restrictions not on the lcngth of the riglrt sides of a prodrrction.nelf'l.+vrr.q'rroN ColrnxrIrrtnn GR. This additional fret:dom gives Greitra. constructirrg a grarnmar irr Clreibach normal form ertlrivalerrt to a given contextfree grammar is tedious.nition 5.l conseqrrences..ri. Dt . .fr.ii1qniy. +b. D2 + RnB6.ch normal forrn and sgrammar$i but Greibach normal form does not carry the restriction that the pair (4.Iandr€V* If we compare this with D<:fi.tnns NoRMALF onlrs In thc secondstep. We therefore deal with this rnatter very briefly. a. Bt. but on the positions in which tcrminals ir.nd variirhles carl appear.!. we irrtroduce additional variables to get tlrc first twrr producLiorrsinto normal forrn and we get the fina. Greibach trormal forrn has marry theorctical arrd practica. Arguments justifying Greibach normal ftrrm are a.Iittle complicated and not very tra. wherca€.r is comrtron to both Greiba.nspartrnt.)occur at most once.ffi tN A txrntextfree grarnmar is sa. p +ABt. .AND oF' Chopter6 Srrururlc. Sirrrilarly. Bc+c' I Greibqch NormolForm Another usefrrl grammatical fotrn is the Greibach normal form.4. A + BoD2.
Here are two simple examples. S+AB. we immediately get the equivalent grammar S + aABlbBBlbB. which is in Greibach normal form. rreither the conversion of a given grammar to Greibach normal form nor the proof that this can always be done are $imple matter$. using the substitution given by Theorenr 6.2 Two IturoRrnrqr NoRl"ler. A+d. I In general.4 and B that are essentially synonyms for a and b.1. However. Bb. A .10 Convert the grammar SabSblaa. Here we carl use a device similar to the one introduced in the construction of Chomsky normal form. A + aAlbBlb. Substituting for the terminals with their a"ssociatedvariables leads to the equivalent grammar S +aBSBlaA. B+b.FoRrrs 169 If a gramrnar is not in Greibach normal form. respectively. tltough. However. B 'b is not in Greibach rrormal form. into Greibach normal form. we may be able to rewrite it in this form with sorne of the techniques encrluntered above. We introduce new variables . from . We introduce Greibach normal form here because it will simplify the technical discussion of an important result in the next chapter. which is in Greibach normal form.aAlbBlb. I Exoilrple 6.6.
Provide the details of the proof of Theorem 6. Let k be the maximum number of symbolson the right of any production in P. ?.aabll BbbA into Chomsky normal form. aSaAlA.Chopter SrupllnrcauoN ColrruxrFnpnGnq.\. ffi 6 . see Example 3. 1 . !+Ba. Ttansform the gramma. A + bABl.r S form.BAalAlA. S. w h e r eo E T . For everycontextfreegrammar G with ^ # L (G). m . A abAlb into Chomsky normal 4. B E V .= r ( G ) .P) all of whoseproductions have one of the forrns A+aB.13). I . B . 3. s u c ht h a t . .S.tr. Ae. Let G : (y. Transform the grammar with productions S * abAB. L A linear languageis one for which there exists a linear grammar (for a definition. A .ABIaB A .6. Show that there is an equivalent grammar in Chomsky normal form with no more than (k . into Chomsky normal form. we only quote the following generalresult without proof.. P) be any contextfreegrammar without any trproductions or unit*productions.T. . Let Z be any linea^rlanguage not containing . . there existsan equivalent Bramrrar d in Greibachnormal form.r S .. 5. Draw the dependencygraph for the grammar in Exercise4.ns NoRMAl 6 FoRtvrs AND oF a conceptualviewpoirrt.1) lPl * lTl production rules. Greilrach normal form plays no further role in our so discussion. Convert the grammar S + ttSblab into Chomsky normal form.uve. Show that there exists a grammax6 : (V. Convert the gramma.
A contextfree grammar is said to be in twostandard forur if a.L(G).{n. Convertthe grammar G: ({S.B.b}. B b.wosta.ll production mles satisfy the following pa.T. e + aBCt irrto twostarrdard forrn. a 10. S + rt. there exists an ecluivalenf grammar irr t. Convert the following grarrrrilar into Greibach norrnal form.. Can every lincar grar[mar be convcrted to a fbrm irr which all productiorrs Iook like A . .prs 171 9.6.l + o. 14. A+ B into Greilrach norrrral form.ttern .an.lb ffi f<rmr is generall for any contextlrce grarrrmar G with A { .B. ItAb W aSblbSola. or * .C €V. A.'I'wo$tandard ffi witli P givena.1 tr.P) S + a.CeUandae?. Prove this.S. 13. 11.rrdard form.rNoRMAL Fon..where a e ? and tr € yU {.2 lI'Wo Ir.B. Corrvert tl:regraurnar S irrto Greibar:h rrorma.94. AaB. A + bABC.s uaAlB..Sblab.rpoRrAru. Convert the grarnrnar g + ab loSlaa5 into Greibach norrnal form.A.C}. wlrcre € X U {I}.P)thereisaneqrrivalent one irr which all pr'64u"tr1on.rhave the forrn A + a. Convert the 6grammar S + ABblo.U? 15. 12. A_+e1 wher'eA.l forrn.BC.S.BC. *16. ShowthatfrrrevervcontextfrcegrarrrtrarG:(V.
lvlel.S) LE {r.r. 3 . .if we prot:ced in tirtl secluetrctl l ..fter its originators J.2 . .lled CYK algorithm.'en.nclonly if tltere is a.i I U An instrrrx:tion t]rc indices in (ti. I n t r t h e r w o r d s .itned. F ORl{s iiffiffiffitil A MembershipAlgorithm for ContextF r e eG r o m m q r s * In Ssrtion 5. Assume that we trave a grarnmar G : (V. . . Vlrnger. wt: cla.without any elaboration.rbserve drrction A .. tr/r*1.llthe F1r.2. a. ... r . A derives ruii if a.a'r'loN oF CON:rEXTFRljtl GRnr. V .ij: a .. C t r m p u t eV r .y. V2r.ai. .8) shows that it catr bc used to t:ompute o[ rr. 1 " a 1f o r s o m e f u w i t h i < h . The algorithrn works only if the grarnrrrar is itr Chomsky rrormal forrn aucl sutr:eeds by breaking one pr()hlem into a $equell(xl of stnallt:r ones irl the following wir. V n . ..j . Ctimpute Vrz. ancl only if G contains a. . . We definesubstrings wij:(ri"'ajt and sulnetsoI U vi : {e Ev :A5 u.. . Vil call llt+ computed for all I < z < rz by inspection of u arxl the prtlrluctions tlf the grarrrmar. We are lr()w in a position to justify the this clrrirn. II.W ..i C {A: ABC. . notice tlrnt for j > i. V 2 t .rt l.r}. V z z . withB 4r1* v.wit}rBEHtr... produt:tion A+ BC. P) in Chornsky llorlllill forrn and a string 'til:eraz. The algorithm wt: will describe here is t...i. Ttl r:ontiurrc. (6. Kasarni. that mcmbership and parsing erlqorithttm f'or cotrtextfiee grarnmars exist that retpire approximately ltol'' steps to parse a string trr. D. .tuaRS At'ro Non.Chopter 6 Stltrurlc. Therefortt. attd srt ott.t A €Viif To cornpute Vl.. . 2. h < r .c€Vr+r. Cocke. r r d 1 ' . Clearly. S. C t r m p u L e t : t . r.T'. u E lt (G) if and orrly if S e Vr".tr. arxl T.}. prothtr.
rrrtl.:r sl.l. V4 : {4. R} . tinrlwc haveH23: {S'. 4 } . r:arr l_re it qrnvcrted iulo ir.wc irlso havt: yz:: {. {A} . that i*.! +B(. (_.r. a grarnrnal iu flhonr. V 4 a= ={ E } . A straiglitftrrward argrrrnt:ltalongtlu:srlIinesthrlu gives Vn: : Yr3 Vr+: V ' n: s ot h a t u t e L ( G ) . ntitic:c Lhat exa.I/rr : {5.. have to lltl cotrtl:rtted.Vt+: {l} .ridcof a prorluction whosu righl sicle is A.J frirrn. V t t : { A } . V:s : {S.dditiorrsto keep trac:k of how tlrc elertrentsof l/1.t the CYK rnemlrershill algoritlrrrr lecluiles ()n. similarly.ivecl.colr.8) to get I4z : {A : .0.ctly rr. l'lr: {.t rriost ll. Sirice ur22: e.chinvtilvr:s 1.$). the sct c:onsists a.Next.4. With sorlc langrra.1) /2 sets of V.inrtxl rt:srrlLlirilows..' t/rr} . V12 is crrtllt.V r r : { 5 . First rxltrl Lhi'rl'ur1 . : { B } .(rz 1.'Il'. B l . l / z r : { . a } . e strtlre reclrtircrl righl sirleis AB. 8 } .iso V11 is thc scl of all vir.: . . { 5 .i.riy gr:leraterl lr. B € Vn.riablesthat irurric:rliI ately rk:rive a.4}.l/+o: {A} .e1rs.xrFnnnClHe.r.tns L7B Ultqftn$f$ld. I fr. E r : { .3 A MnlrnnnsHlr Ar.heeva. O t V22} . alc r1r:r.. Thc CYI( algorithru.. Since V11= {A} ancl V22: {A}. RI . Y r r : { B } Now we use (6. } _tB(i.rn.R } . as clrrst:r'illccl her:e.r Fon {loFl'r. V r . { 5 . Since tlrt:re ale nrltre..dctt:rrrlinesrncrnbershipfor ir. terms irr (ti.gc a.gt: generatecJ lw 5AR.sky lolrng. of so tlrc cla.lrrntiorr a. Eir. Dcttlrmine whetlrrrr the string In: tlul Hratnutar (rabbbis irr the langua.8}.4} at:ci.irarsirtg rnetlxltl.ll variables t}iiit occnr of ott Lhe lefl . B E Vtt.y. A _ BBla. R + ABlb. Iir see thrr.
mfllat in Chomsky norural form. Usc the approach crnployed in llxcrcise 2 to show how the CYK mcmbership algorithm can be trrade itrto a parsing metlxrrl.11. ffi 2 .and obtibb a.r.. * * 4 .re in the language generated by the gtarnmar irr Examplc 6'11' of Examplc 6.774 Chopter 6 Srrvtt. aabba. using tlre gratnrrrar 3 . IIse the rcsult in Exercise ll to write a cotnputer prograln firr Jrarsing with any contcxtfree gra.rprcetloNol CorvrrxrFn. .Ilri GRAMMARSAND NoRMAI Fonlrs 1 Use tlre CYK algorithm to dcterrrrirte whethcr ihe strings nabb. Usc the CYK algorithm to find a parsing of the strirrg aab.
we need more tharr unlimited cxrurrting ability: wc rreed the ahility to store and match a sequence of symbols in reverse ortler. For example. This suggests that we might try a stack as a storage mechanism. we rnust also corrrrt the nrrmber of a's. But as we see from other examples. This gives us a (lass of machines called pushdown automata (pda)' 175 .s.Pu hdown s Automoto he description of cnntextfree languages by means of contextfree gramrrrars is convenicrrt. we understand that this is because finite autotrara have strictly finite rrrernories. The next question is whether there is a class of automata that can be associated with contextfree langua.finite automata cannot recognize all contextfiee As languages. when scanning a string from the language 7 : {a"b. as illustrated by the use of BNF in programming language dcfinition. We want a machinc that can count without limit.st not only check that all a's precede the first b. Sinc:en is unboundrld.. this counting cannot be done with a finite memory. we have seern. allowing unbounded storage that is restricted tci operating likr a stack.whereas the recognitiorr of a contextfree language may reqrrire storing an unbounded amount of information.ge. su()has {wwR}. I n } 0}. we mu. Intuitively.
tre Figure 7. . forming a proper subset of the contextfree languages. Each move of the control unit reads a symbol from the input file. The class of deterministic pushdowrr automata defines a new family of languages. we explore the connection between pushdown autornata and contextfree languages. we conclude ttre chapter with a brief introduction to the grammars associated with deterntittistit: contextfree languages. The result of the. we will restrict ourselves to pushdown automata acting a.r. We first show that if we allow pushdown arttomata to act nondeterministically. In our discussion. Aufomoton Definitionof q Pushdown Formalizing this intuitive notion gives us a precise definition of a pushdown automaton.move is a new state of the control unit and a change in the top of the stack.1. But we will also see that here there is no longer an equivalence between the deterministic and nondeter* ministic versions. Since this is an important family for the treatment of programming language$.seccepters. Each move of the control unit is determitred by the current input symbol as well as by the symbol currently on top of the stack. we get a class of automata that accepts exactly the family of contextfree languages. M A N o n d e l e r m i n i s tP u s h d o w n u t o m o t o ic A schematic representation of a pu$hdown automaton is given in Figure 7. the deterministic contextfree languages. while at the same time changing the contents of the stack through the usual stack operations.Chopter 7 PusHooww Aurorr.1 Input file In this chapter.
and the currrlrrt symbol on toJ) of the stac.llerlthe stack alphabet.k.finite suhscts of Q x f* is thc transilion functirin. tht: current input syrnlrol.rethc current state of the control unit. where q is the next state tlf the control unit trnd r is a string whiclt is pnt on tcill of the stack irr place of the singlc syttrbol there before. irnd the symbol on top of the stack is b. .z). Finallv.a .7.ppen: (1) the t:otrtrol unit gJoes irrto state {2 ancl t}re string cd replirces b on top of t}re stack. c If at any time the control unit is in statc q1.1 NowlnrER. The re:sult is a set of ptlirs (q. ____________'Emunrpl+ f .. X is the input alphrrbet. 1 r . While an npda may have several choit:cs for its move$.stack is done symbol by symbol. The irrgutnentsof d a.rting at the riglrt end of the string. stir.rrsitionrules of arr npda crrntains d ( .MrNrsrIcPrJsurrowruAuroltnrA 177 . t . d ) .. Qo€ Q is the initirrl state of the corrtrol unit.lll lr A nondeterministic pushdown accepter (npda) is defineld the septry trrple M : ( Q .\. Note also thnt d is defined so that it needs a.irRR. 1 . 8 . irrdicating that a move that does not collsurne an inprrt symbol is possible. then one of two thirrgs can ha. In our rrltation we &ssurnethat the instrrtion of a string into a. Note that thc second argument of d may be .tesof the control unit. the input svurbol read is a. \ } . q o 1 z 1 F) whcre Q is a finite set of internal sta. We will call such ir rrlove a . tiris choice must be restricted to a firrite set of possibilitics. Q x (E U {A}) x l' .$lr"q. b ) : { ( .nsitirirr. .rrdtherefore ha"r irrfirrite subsets.fil. I . q 3 . d .\tra.t:ksymbol. F e Q is the sct of linal states. Suppose the set of trir. r €  is the stack start symbol.l. I' is a flnite set of syrnbols ca. sta. or (2) the u)ntlol unit goes irito state qs with the symbol b rcuroved frotn the top of tht' stack.. the requirernetrt that the rarrge of d be a finitrl subset is ner:essarybeca. t r .iirqnri.use I x f* is atr infinite set ir. no m()v()is possible if thrr stack is emptv. The ulrnplicated formnl appearance of the drlrrrairr and range of d rnerits a closer o'xamitration.
b .A. { /.I)}. r : { 0 .:0 ) ( q 3 .. Of course.10).'_ 6(h. This assures that no b precedes the last a.b. there is no entry given for d(qo. What can we say about the action of this automaton? First. we must define what we mean by an npda accepting a language.b } . 1 ..1 ) : { ( q 2 . 6 ( q r . ] d ( q t . These two steps count the mrmber of a's and match that count against the number of b's. 6 ( q r .} z:0.. .0):{(q3.11)}.o){(q'.\)}. d ( q 0 .1) = {(q1. 1 ): { ( q 1 .. and 6 ( q r . For instance.di : d (sor. a . The interpretation of this is the same that we used for nondeterministic finite autoutata: an unspecified transition is to the null set arrd represents qdead configuratiqllfor the npda. \ ) } . notice that transitions are not specified for all possible combinations of input and stack symbols..0). Q t . 1 . 1 1 ) . \ ) }. l ) : { ( q 2 ' . Q z . before making $uch a. which removes a 1 when a b is encourrtered. is read. claim.{anb":n>0}u{"} As an analogy with finite automata. After analyzing the remaining transitions.A ) } . 1 :) { ( q 2 . q. I ) } . The control unit is in state (1 until the first b is encountered at which time it goes into state q2.$.(q3.178 Chopter 7 PusHnowN AuroMlt'rA ffi $xomple 7. I . t r . and . The crucial transitions are {I=.t } p : { a . we see that the npda will end in the final state q3 if and only if the input string is itt the language 7. d(qz. we rnight say that the rrpda accepts the above language. r': {qr}. . which adds a 1 to the stack when an o.2 an Consider npda with Q : { q o .a. . l r .
e F . i . A move from one instantaneous description to another will be denoted by ihe symbol F. .' is the unread part of the input string.7.aw.t .I NoNDETERMrNrsrrc PusHoowruAurouarn l7S To simplify the discussion. a ) d ( q 1 . In words. t i . u ) . p e r . .. € Moves involving an arbitrary number of steps will be denoted Uy [. r ) i u @ . The final stack content z is irrelevant to this definition of acceutance.70/ JTdeF a where q is the state oJ_tlp_gqntrol uaiL tr.E. Together these completely determine all the possible ways in which the npda can proceed. lc ) t. i 1 .F) be a nondeterministicpushdown automaton.. ..y.n/ld Jfn"J (5 {r. the language accepted by M is the set of all strings that can put M into a Iinal state at the end of the string.S ii. The relevant factors at any time are the current state of the control unit.a. . The triplet . (8.t.d. On occasions rvhere several automata are under consideration we will use Far to emphasize that the move is made by the particular automaton M.80. .f. Let M: The language accepted by M is the set L(M): {.b}* : no(w) : nu(ru)}. and u is the stack contents (with the leftmost symbol indicating the top of the stack) is called an instantaneoua description of a pushdown automaton. w .tm$ilnlirtri ril.l i Construct an npda for the language L : {* e {a.b!)  (qz. we introduce a convenient notation for de* scribing the successiveconfigurations of an npda during the processing of a string. } .a . u e f . i ( q o .2. As in Example 7. the solution to this problem involves counting the number of a's and b's. b ) . and the current contents of the stack. TheLonguoge Accepted o Pushdown by Automolon lllfim$.U!) is possible if and only if i t ( q z . the unread part of the input string. i ' Ht(dftfpls y. thus (qt. Here we need not even . which is easily done with a stack. 4 .r'ol u^rt L t' '+ j.
r .( g o . 6 . z ) . f ) : {(qn. ' I z )F ( q 0 .ke$the move. #.b.. 15. then pop olle counter symbol from the stack when a.rc to trc rnirtchtxl irgairrst ats later. 2 . For thc stx:tlnd pa. z} . Brrt the lolcleterrlinistic rrature of the irutornatr)rr hclllls us with this. 'W WMWWiN*t:tlrrstrtrtltilnrrpclilfilrtr. But this is easy to fix.1]:f ''An apparent clifliculty with this suggcstiorr is th. tr.tr. : ( p \unun : ur € {ct. We ca1 iusert a counter symbol' say 0. . a d (qo. 0 4 )' } 1z)} .t wtt rltt rrttt know tfie rniddle of the strirrg.rt.Qt. z ) : { ( q 1 1 . w h e r e .b} .nnpdrr.i. . A s o l u t i o nt o t h e p r o b l e m i s g i v e u b y M : ( 8 . .0 to usr:. . I ' . we r:ompi.4z} y:{a.f . ftlr t:ourrting thel b's that a. z) F (qr. b a a b . . a b . F : {arl .I (qo. 0 0 ) }' b d ( q o . q o . r) : {(q11. We can ll$e iI negiltivtl cotrntttr symbol. sa. tr.r \ we use the fact that the syrnbols are retrieved frorn a stack itr the reverse or<lerof their insertion.z}.lT*il:.'it. i ' ) . 1 ) : { ( q 6 . .re current input symbol with Lhe top of lhe stack.0z) F (qg. {r1l}). w]rttrc tl cnds ilrrd rirE sttlrts.b.1(t.r:k. z)} .y 1. Tlte cotttplete solution is r1o. d (rJs. a b ' a ) f l. 0 ) : { ( r t 6 . t) alrd hcrrt:t:thcl strirrg is ircx:epted.\. In processing the string baotr.A'[: ({rlo.b}. into lhe stack whenever an a is read. .. The only difficulty with this is that if wtl there is a prefix of ur with more b's thiln r?.q. thtr npda correctly guesseswhere the rrriddle is artd switt:hes stirtes at that ptlirrt.s a ( q s . the nprla.. 1.'\)} ' b d ( .:lT:'"'*ni[::r:]:. b}} .O ) : { ( q n .(qo. with ri given as I . z) : {(qt.1 1 ) } . {&.Chopter 7 PusHnowu Auronere w6rry aout tlte order of the a's atrd b's.cx:tlptirrgt}rtllirrrgrri'r. Q : {. mtr. a d ( q o . j n . . Wlren reacling the first part of the string. we push the rxlrrsgr:rrtivr:svrnbols rirr tiru sta.r. 8 . ir. {0. tirirt is.I ) } . a d ( q o .r. b is fbund. cotrtinuitrg as lotrg as the ?iil: ix. z.  : {a.ge I.i:i1. will not find a.
7.1 NolrnnrERMrNrsrrc PusHlowl Arlrouara
181
The transition firnction can be visualized as having several parts: a set to push zr orr the stir.r:k,
\ / ) I
d (so, a) : o, : 6 (qo,b,a) d (so, b) : o, (t(,10,1r,: b)
d (Qo,a, ): z d ( q o , b ,z ) :
J""sh
aa)} , {(so, b(z)} {((t(r, , ab)} , {(qs, {(qo,bb)},
{ ( q g ,a e ) } . { ( q o ,b r ) } ,
{ )
a set to guesstlre middle of the string, where the npcla switches frorn state ttl 9r1 qt '*d ( q o , ,a ) : { ( q t , a ) } , A d ( q o ,t r ,b ) : { ( q t , b ) } , a set to uatch zrrRagainst the contents of the stack, 6(h,a,0) : {(q1,I)}, : { d (,1r,1r,1,) (qt,I)} , and fina,lly d ( q t , A , , ) : { ( q z ,r ) } , to recognizea successfulmatch. The sequence of moves in a,ccepting abba is (qo,abba,z) t (qo,Itaa, az)  (Qo, baz) ba, F ( q 1 , b a , b u z ) l( t 1 1 , o . , n F )( g r , A , . z ) ( q 2 , . e ) . 2 F The nondeterministic alternative fbr loca,ting the miclclle of the string is taken at the third move. At that stage, the pcla har the instir,ntaneo's descriptions (qo,ba,baa) a'rl ha,stwo choices fbr its next rnt)ve. c)rre is to uue d(gs,b,b) = {(qo,bb) ancl make the move i (q6,bu,rt, baz)  (qo,a, bbaz), the secondis the one used above, namely d(q0,,\,b) : {(qr,b)}. Only the latter leads to acceptanceof the irrput.
I
Aurol\lArA Chopter 7 PusHnowtr
1 . Find a pda with fcwer than four states that accepts the same language as the pda in Exarnple 7.2.
D,
Prove that the Jrda in Exarnple ?.4 rloes not accept any string not in {tutrtF}'
ffi
fD C.,nrt.lrct nptla's that atxrept the followirig regular languages' (a) Lr : L(aaa.b) (b) IIr : L(aa,h"aba) ({ the urrion of trr an<l Lz
(d) Lt  Lz
on languages X : {4, b,c}' fD Construct npda's that acceptthe following (a)L={**bt*,rz>0} W (b) .L : {tnctu'r : ru e {n,,b}* } : (") f.L {.a^h*c'+*: n, } 0, > 0} (.1f : {o"^h+*c*:n}Q,m}1} ) (*) t: r: { a 3 b ' o c ' oz > 0 } W
(f)r:{a*b*:nlnz/3n}
( (S)r : {u : no (*) : rru t,) + t} ( I t ) . L : { u t : n o ( u t ): t r , 1  1 y
(i) {* '. n,, (to) + nr, (u) : Tr.. (tu)}
(j) .L : {'u:2no (trr) { n;, 1tr,)< 3n" (ta)} (k) .L = fw : no (tr,) < na (tr')] )<rrrstrtrt:tarr nlrda that ac:t:eptstfue language 7 : {anb* : n' } Q,r1 f v,ltt. Firrrl arr nprla on 5 : {n,,b, c} t}rat accepts t}re language L : i t t r tl ' t r t € { r r ,b } * , r u , I * { } z {zntcutz '
?, Filtt arr rrpda for'the cotrcatetratiotrof tr(a*) and the language irr Exercise 6. @ fitta an trpda lbr the language 7 = la'b(ab)" b (ba)" : n > 0].
9. Is it possiblc to lincl a dfa that acccpts the sarne language as the pda M : (Iqo, qr]1 {a,,1,}, {t} , qo,{qr }) , ,
71 NoNnerERMrNIsTrc PusHpownr AurouArR
1BB
with
d ( q o ,a , z ) = { ( q r , e ) } , d (qo,tr,e) : {(So,e)} , 6(qr,tt,z): {(q',r)}, d ( q r , b , " ) = { ( q o ,z ) } ? f f i 1O. What language is accepted by the pda M : UrIr, qt i qil qli 4+, qsj , {a, b} , {0, r, a} , qo,{qsl) ,
with , d ( q o b ,z ) : { ( q 1 ,1 z ) } , : 6 ( q t , b , 1 ) { ( q ' ,r r ) } , d ( q r ,o , 1 ) : { ( s . , } ) } , d ( q . ra , 1 ) : { ( q n , ) } , , f 6 (q+, z) : {(g+,z) , (gs, u, z)l'l
/+\
1.,
is try S1,/Wtrat language accepted the npda M = ({qo,$,gzl ,{a,b},{o,b,a}, 6,Qo, {qz}) with transitions z,
6 ( q o , a ,z ) : { ( S r , n ) , ( g 2 , . \ ) } , d ( q r , b ,a ) : { ( q t , b ) } , d(qr,b,b): {(q,,0)}, 6(qr,a,b): {(Sr,A)} ffi 12. What language is accepted by the npda in Example 7.8 if we use F : {qo,q'}? 13. What language is accepted by the npda in Exercise 11 above if we use F : {qrt,$,qz)? 14. Find an npdalvith no n:rorethan two internal states that accepts tlre language
tr(aa*ba*).ffi
15. suppose that in Example 7.2 we replace the given vaJue of d (g:1, A,0) with
,5 tr,0): {(qn, . (sz, A)}
_ (rel What is the languageacceptedby this new pda? we can define a restricted npda as one that can increa,$e length of the the stack by at most one symbol in each move, changing Definition ?.1 so that
d : Q x (Xu {I}) x  + ?Qx(rrufu{r}). The irrterpretation of t,his is that the range of d consists of sets of pairs of the ftrrnr (q6,ab) , (qt, a,), or (Q,,A). Show that for every npda M there exists such
a restrictetlnpda fr suchthat L (M) : L (fr)
ffi
184
AuroMArA 7 Chopter PusunowN
dD 
is O" alternative to Definition ?.2 for languageacceptance to require the stack the end of the input string is reached. Formally, an npda to be empty when
M is said to accept the language N (M) by empty stack if
N ( M ) : { t u . E . , ( q o*,, r ) i * 1 p ; , r ) } ,
L
J
where p is any element in Q, Show that this notion is effectively equivalent to Definition ?,?, in the sense that for any npda M there exists an npda fr such that L(M) = t (fr), atrd vice versa'
M
A P u s h d o w n u t o m o t oo n d C o n t e x t F r e eL o n g u o g e s
In the examples of the previous section, we saw that pushdown automata exist for $ome of the familiar contextfree languages. This is no accident. There iS a general relation between contextfree languages and nondeterministic pushdown accepters that is established in the next two major results. We will show that for every contextfree language there is an npda that accepts it, and conversely, that the language accepted by any npda is contextfree.
1 "1d "* 7{ttilrtl+jh
")t)F
[onguoges Automolqfor Conlext'Free Pushdown
We first show that for every contextfree language there is an npda that accepts it. The underlying idea is to corEtruct arr npda that can, in some way, cerry out a leftmost derivation of any string in the language' To simplifii the argument a little, we assume that the language is generated by a grammar in Greibach normal form. The pda we are about to construct will represent the derivation by keeping the va.riables in the right part of the sentential form on its stack, while the left part, consisting entirely of terminals, is identical with the input read. We begin by putting the start symbol on the stack. Aft'er that, to simulate the application of a production A + afrt we must have the variable A orr top of the stack and the terminal o as the input symbol. The variable on the stack is rerrroved and replaced by the variable string r. What d should be to arhieve this is easy to see. Before we present the general argument, let us look at a simple example.
.P*gnrnlS.,fir,5
Construct a pda that accepts the languagegeneratedby grammar with productions 5  oSbblo,.
7.2 PusHoowrr Aurordaraeuu CoNrnxrFRnrLANGuAcEs IBE
We first transform the frammar into Greibach normal fbrm, changingthe productions to 5'+ a1Ala, n + bB, B +b. The corresponding automatonwill havethreestates{go,qr, gz}, with initial state gs and final state q2. First, the start symbol ,9 is put on the stack by d (qu, ") : {(qr,S")} . A, The production 5 + aSA will be si*rlate4 tr lbq]p4gby_fg*oViqg g
to* th* tto,"k o.tt.l "o['lo."it g it o.ith
Similarly, the rule ,5  a should cause the pda to read an a w.h:ilesimply removing S. Thus, the two prodrrctions are represented in the pda by d ( q t , a , S ) : { ( s r , 5 , 4 ) , ( q 1 ,A ) } . In an analogous manner, the other productions give
6 ( q r , b , A:) { ( s r , ) } , B
6 (h,b,B) : {(qt,f)} . The appearance of the stack start symbol on top of the stack signals the completion of the derivation and the pda is put into its final state by d ( g r , A , z ) : { ( q 2 ,I ) } . The construttion of this example can be adapted to other cases,leading to a general result. I
For any contextfree language Z, there exists an npda II such that L: L(M).
Proof: If tr is a Afree contextfree language, there exists a contextfree grammar in Greibach normal forrn for it. Let G : (V,T,S,P) be such a grarnmar. We then construct an npda which simulates leftmost derivations in this grarnmar. As suggested, the simulation will be done so that the unprocessed part of the sentential fornr is in the stack, while the terminal prefi.x of any sentential form matches the corresponding prefix of the input string.
Chopter 7 PussoowN Au'roMA'I'A
Spar:ificallg the npda, will be M : ({qo,{r,{l},T,V l{z},d,rlo, z,{gl}), whetrez I V. Note that the input alphabet oI M is identical with the set of terrninals of G and that the stack alphabet <ffitains the set of variables of the grammar. The transition functiotr will include A d ( q o , , a ) : { ( q 1 , 5 a ) },
( 7l )
so that after the first move of M, the stack contains the start symbol ,9 of the derivation. (The stack start symbol a is a marker to allow us to detect the end of the derivation,) In addition, the set of transition rules is such that
, ( r n , u )€ 6 ( q 1 , aA ) ,
wllenever
(7.2)
A
au
is in P. This reads itrput a and removesthe variable A from the stack, tlte transitiorrsthat allow the replacingit with u. In this way it generates pda to simulate all derivations. Finally, we trave
d ( q r ,A , z ) : { q t , z ) } ,
( 73 )
to get M into a final state. To show that M accepts any ru € L(G), consider the partial leftmost derivation
'**.::::
.";i:o';rf
A*
If M is to simulate this derivation, then a"fterreadirtg ar&z" '4r,, the stack must contain AtAz' ' ' A^. To take the next step in the derivation, G rnust have a production A1 + ttBl , .' Bp. But the construction is such that then M has a transition rrile in which ( q r , B r . ' . E r ) € d ( q r ,b , A r ) , so that the stack now contains Br ' ' ' BnAz'" An after having r€fld a1a2' ' 'anbA simple induction a,rgument on the number of steps in the derivatiorr then shows that if
E+u,
7.2 Pusnnown Auroua.rA ANDCor{rnxrFnEE LANCUAoE$
fgz
then ( c 1 1 , u t , s a ) ( q 1 , . \z ) . t , Using (7.1) and (7.3) we have ( q p , ' r r . , , F ) g r ,w , s z ) i ( n r , ^ , a ) F ( q 1 ,. \ , . e ), .z( so that L(C) e L(M). To prove that .t (IUI) e LG),let u € L(M).Thern by delinition
( q o , r uz ) f k t y , A , u 1 . , Brrt thcrc is orrly orle wfly to get fiom qg to tlt and orily one way from q1 to {1. I'herefore, we must }rirvt: ( q r , * , 5 e ) i ( q 1 , ,z ) . A Now let us write ut = eje,zil;i. " a* Then the first step irr ( q 7 , a L a z a 1 " . . . a n , S z1 q 1 , , 1 , 2 ; )i must be a, rule of thc forrn (7.2) to get .' ( q y ,q a 2 a 3 . , . e n , S a ) F ( q t , a , z f l : r . a , , 7 . 4 2 ). r, Rr.rtthen thc grarnrnar has a rule of the fllrrn S + aturt so tha,t 5 + a 1 t i ,.1 Repeating this, writirrg 'LtL: Auz, we have a ( e 1 , u , 2 t't,;,1 , r , A u 2 z )  ( q 1 u , 1 . . e , , , , , u 3 I t 2,l ) , .1 irnplying that, A + u2rn is in the grailrnrar rrncl tha,t, S # a,1u2'u3'u2. This mir.kr:sit quite clear a,t any poirrt the slack contents (cxcludirrg z) are idt:nticirl with the unnra,tcheNl liirrt of the sentential forn, so that (7.4) implics 5 # a142.',ar,. In conseqrren<:c, L(M) g , (G), completirrg the proof if the larrgrra,gcr cloes not txrntairr .\. If A e L, we add to the constmcted npda the transitiorr d ( q o ,) , z ) : { k t , z ) } so thnt the empty string is a,lsoac:r:cpted. r (2.4)
.afl.s ) : { ( q r . A Tltis rxlrresponds to the derivrrtion S + aA+ aaABC + aaaBC + aaabC 4 ua'ahc. b . the proof in Theorerr 7. a b cA B C z ) . z) S z)  (q1. 1 1 .bc.ssumed that the grirmmar was iu Greibach rrorma.1 ir.(qt.g)}' d ( q t .c . Bb.b.aabc. Cc. A+ aABClbBla. o . t r . t r ) } . . . F ( q 1 ."il Exonple 7.rfu. g " ) } and d ( q . aao.S ) : { ( s ) . we can use the constrrrr:tion in the previous theorerll irnrneditltely.d) : {(q1A)}.l form. In addition to rules d ( q o . : d(qr. Since the grammar is already irr Greibach normal form.A) : {(q'. z ) } ' the pda will also have tratrsition rules d ( s r . Mg). t Irr order to simplify the argurnents. . d(qr. we carr rrrake a similar and only slightly more complicated constru<:tion . F (qr .bc. q. It is not necessary to do this. B ): { ( .Chopter 7 Pussoowt'r AuroN4ArA Jfi rr. A ) \ .a.t r . b cB C z ) .6 Consider the grammar SaA.Az) F ( g 1 .a ) . The sequence of moves mrrde by M in processing aaabc is (go.". 6 (qr.a ) : { ( q r . C z ) F (91''\' z) F ( q 7 .  ( q 1 .tr)}.A) {(qt.
The rrlrrstruction involved readily suggests itsulf: reverse the proce$$ in Theorctr 7. . Quite a few details are needed to mir.1. This equivalence wiL.A): {c1.nd2. while the processed input is the trrrrnirrirl prefix of the sentential ftrrrn.1 so that the grammar sirnulates the moves of the prJa. For example. but coruume no input symbol. This rrreans that the contq:nt of the stack should be reflected in thc variable part of the sententirr.rgurirents as an exercisel(st:c Exelcise 1tj at the elnd of this section).c?.l sta. for productions of the f'orm A +Bx.. Ilere we need to cxplore it further. eaq:h trove either iucrease$ decrcascs the stack content by n sirrgle or symtxrl.he ab in the inprrt agrrinst a similar string in the sta.EE LaNcuacns 189 from a genelral t:ontext*free gramma. To keep the discussiorras sirnple as possible. we remove :. 2. we will assurnethat the npda in question meets the following recluirements: 1.r..o. whelrc : r:a (qi. Frrr procluctions of the form A + abCr. We leirvc thc details of the constnx:tiorr arrd tlrc associatedproof as an exerr:isr:.l ftrrrrr.. Takirrg tltis as given.rexpkrrtxl partiallv in fixercises 16 and 17 in Srxrtiorr7.. All transitions rmrst have the form 6(qt. BC) . we must first rnatch t.r:k anrl then replace A with Cz.(76) Tha.kt:this work.cr}.. we want the senturrtial forur to represent thc content of the sttrck. and of .t is..titirr the npda also involves arr irrternal state.7.1 is also true.gairrwe will Jeavetht: a. tr) .ving properties I rr.1frrlrl the stack ancl replace it with Br. As statecl. Tltese restrictions rniry appeer to tre very sev()rc)but they are not.2 PusrrDowm Aurolrnrn nulr Clt)rurr:xtFn. (7 lit) r:i : (qi. hrrt a. It can be shown thrrt for any npda there elxii*s an equivalent one ha. we now rxrrrstruct a contextfree grarnrrrar ftrr tlx: lirrrguage accepted by tlxl rrllda. But the configura. ConiextFree Grqmmors Pushdown for Aufomolo The converseof Thtlclretn 7. It has a single fina.te rly that is entereclif and only if the stack is cmpty.
'oErelsing" successivestritrgs bv which it is replaced) are retnoverdfrom the stack. Brrt this is exactly how the rrpdtr.nfind a grammar whose variablt:s irrrr of the form (qiAqi) and whose prothtt:tions are such that (qiAq1) u.ge accepted by thc npda. This is due to thc fir.ge grarrrrrrar.lrres Q.nd going here mealls that A atttl its cffttts (i. Suppose for the motnent that wc ctr.ke on all possiblc va.ll the trringing the symbol originally bckrw .clehy the npda. If we calr firrtl srrt:h a. gra. then frorrr q. erasing C. to the top.4. fiom state qi to state 4i.ted by the gralrrlrlar will be identica'l to the Iangua.i) + (t'.5) involves arr immediate erasure of A. the lattguagt: genera.tling u a.Il o' elrd going frorrr state 91 to qr.1) u. and if we choose (qorqt) a"rits start symbol.Chopter 7 PusnrrowN AUToMATA this has to be remembered in tht: sontential form as well. In the last step. Productionu of type (7.cceptedby the h.. tlrtr grtrmmar will have a cotresporrtling production (t1iA(t. To construct a grarrllrrirr thnt satisfies these conditiorrs. but this docs not aff'ect the gratnmar.'. ttr gA. and thc rxrn$trrrctirtn we give here is a little tricky. This is true. where qy is thc single final state of the nuda.i Bqi are uselessvariirbkrs ilnd do not affect Lhe larrgtra. 3 if and only if the npda nlmoves . It is hard to see how this can be done.lly.mmar.ccepts'tu' Therefore. . erasing B. therr (qozq. w€lexamine the cliffererrt types of transitions that c*n trc ma. w}rile reading a.e. Tlxr resulting variables tr.a.ct that to erase A wc {irst replac.z (creating an empty stat'Jr) while reading rr and going from (lo to qf. a.e it with BC. Srrbsequently. as start variable wt: tirke (qozqt). Fina. it may seem that we havc arlrltrd too much' as there may be solrle stat()it ql that cannot be reaclttxl lrom qi while erasing B. Since (7. a in whcrr: 96 nnd g1ta.ti) generate thc set of rules ( (qiAqn)+ (q1Bq1) qCqn). we go frorn t1i t'tt q1. 1 A if and only if the nptla orases from the stack whilc rtltr.
From the first two transitiorrs we get the set of productions (sgaqo)a (qsAqg)(spaSo) (qo.nrqr)  (soAs. (qszqz) + (s0Aq0)(qorqr) l(qoAqt) (qnzsz)l(qrArrr) (. . .A e ) } . o . qo) k. i (g1ag2) . qt) (Sr". A ) } . qo).I ) } .new state 43 and an intermediate step in which we first rcmove the A from thc stack. b . a) : { (q6. z ) : { ( q s . .a. (q. (qn Aq=) (qt. . (ss"sr) . Thc last three transitions are of the form (7. d (gr. ilhe new set of traffi d (qo.toArtr)(qraqr)l a(qoAsz) (qrrs. * A 6 ( . . d ( q r .tzqo) (s0Aq0)(so"qo)l(qnAqt) (qrzqo)l(qoAqz)(qrrsu) I (qoAq. A . then replaceit in thc .raqt) (SoAqt)(Sraqr)l (sozqr) l" (r (qoAs.Zo)l lo il (qoAsz) (qr.A)} Usirrgqo 4qinitial stato aryl ga qqth* fi the npda satisfies txrndition 1 above.5) so that they yield the correspotrding productions (goAqs)a. 4 ) } .r) (qtrqz) . k ..r**t *ou*. a 6 ( q .") : {(sz. . 4 ): { ( q u . 1 ): { ( q r . (qo"qz) a {qs Aq1) (qozqz)lo (qrAr1 (qr rqz)l r) a (qoAqz)(qzzqz)lo (qoAS. (qzzqz) ' (qoAqo)(qorqt) l(qnAst) (sfiqz)l(srArt ) ktzzqs) (qoAqs)(qrrq*) .t) krzrlrtt)lo (qoAqr) (qtzqt) . A:) { ( S t .2 PussoowN AureMArA aruo Cortrux'r'FRpeLANcLTAcEs 191 Consider the lpda with transitions d (qo. 5 (qo.\. 6 ( q o . (s0Aso)(so"qr) l(qoAsr) (qraqr)l (qoAqz)(qr". ))} .4gr)* lr. A .but not 2.7. To satisfy the latter. Az)} .A ) } . we irrtroducea. (gu.])(qtrqr) .4qr))(qo"qr) (So"qr) lo(.r) lo (qnAqr)(sszsc). + a (q0.) (q:aqo). d ( q r . a . + a (qsAqs)(s. a ) = { ( q 2 .tt) I (soAqr) (qtzqt) . z)} .4) : {(ql. z) : {(qs.tr. b .
We use the suggested construction to get the grammar G : (V.S. then.X. The first gi in the leftmost variable of every sentential form is the current state of the: pda.{gl}) satisfies conditions 1 and 2 above. This forms the basis for the proof of the general result.A z ) F (g1. A Xi) ( t i .Chopter 7 PusHpown Autouara The start variable will be (qorqz). to The steps in the proof of the following theorem will be ea^sier understand if you notice the corresponderrce between the successiveinstantaneous descriptions of the pda and the sentential forms in the derivation.The string aab is accepted by the pda.z)  (qo.f. Q i .T.A. r r . Az) F (q3.7) and vice versa. u .z)  ( q o . with ? : E and V consisting of elements of the form (acq. (7.i). with successiveconfigurations (qq. while the sequenceof middle symbols is the same as the stack content.A e f . I Iffi language' u h: r(:r.aah. ? € E * . u .z) F (q2"\' '\) ' The corresponding derivation with G is (qorqr) =+ a (q6Aq3)(qtrsz) + aa(62q2) + aa(qsAq1)(qzq21 + &eb(qrzqz) * aab. wheneverthe npda is such that the symbol A and its effectscan be removedfrom the stack while readirrgz and . x ) implies that (qiAq1) 3 u.S.r)lforsoqenpdaM. h .b. We will show that the grammar so obtained is such that for all e t . Although the construction yields a rather complicated grammar. X € l * .2. Q ( q o .d. E .Lis a contextfree Proof: Asslrme that M : (8.1. it can be applied to any pda whose transition rules satisfu the given conditions. The first part is to show that.ab.P).
e) 6(qr.7. Bd put on.A): {(si.8) or (7. . Notice that (qiAq1) + a(qiBq1)(qfiqn) might be possible for some (qiBq) (qfiqn) for which there is no corresponding transition of the form (7. For the converse. 1 ConsequentlyL(M) : I(G). with the control unit going from state qt\o qj. But. r .1r.crrcuacps f03 going from state qa to qi.)) whereby the A can be popped off the stack.Cq*) .n be removed from the stack. A ) .q.ul ConrnxrF'Rpp L.Similarly. We only need an induction on the number of moves to make this precise.10). then there must be a corresponding transition ( 7.A)} (7. at least one of the variables on the right will be useless. Using the corresponding transition for the npda (78) we see that the A ca.2 PusHoowu Autorrrara . Prove that the pda in Example 7. a ) i ( q . . the argument given holds.5 accepts the language 7 : {an+7b2^ : rz Z 0}. then the variable (StASi) can derive z. Show that the pda constructed in Example 7. We see from this that the sentential forms derived from (qAqi) define a sequenceof possible configurations of the npda by which (7. I 1 . 1A . consider a $ingle step in the derivation such as (rlrAqx)+ a(q1Bq1)(q. in that case. For all sentential forms Ieading to a terminal string. we $ee that this can be so if and onlv if (qszqy) w. reading a.5 accepts the string aaabbbb that is in the language generated by the given grammar.7) can be achieved. if (qiAqi) + a.o. This is not hard to see since the grammar was explicitly constructed to do this. If we now apply the conclusion to ( q o .
S A I b . 1 8 . L 7 .Qr}. In Example 7. d ( s o . For every npda M.2 imply the following. Aa)} .9Slab.and (q6agr) are useless' 14.d.L (M): l. A ) = { ( q ' . 11. > 0}' ffi L0.{a. S . Construct art rrpda that will accept the languagegeneratedby the gramrnar s G : ( { S . Construct an npda corresponding to the Brammar S + aABBlaAA. {A. P ) . A ) } . Section 7. 7 .1.7 accepts I (aab). b. Showthat Theorems7.2.b}. Give full details of the proof of Theorem 7. A u aBBla.z}. Give a construction by which an arbitra"ry contextfree grammax can be used in the proof of Theorem 7. show that the variables (+o. A) = {(qo. ffi 5.5.{0.o. . \ \ / ff/ nina a contextfree grammar that generates the language accepted by the npda M : ({qo.Bqn. A .suchthat. AA)} . 1 6 .Chopter 7 PusHoown AuroNrar. ffi 12.c. 13. Use the construction in Theorem 7.1 to find an npda for the language Example 7.1 and 7. {qr}). 6 .7 generates the language L (oab). t o . Show how the number of states of ff in the aboveexercisecan be reducedto two. e) = {(ge. )C"tt*ttrrct an npda that accepts the language generated by the gramma^t S + asbblaab. l r } .1. ffi 8 . W Construct an rrpda that accepts the language generated by the grammar ^9 aS. (E L: tr"O an npda with two statesfor the language {a'ob*+r: n.o . with transitions 6 (qo.2. / e. Find an npda with two states that arcepts 7: {a*bz* I n I 1}. Show that the grammar in Example ?. Show that the npda in Example 7. w i t h p r o d u c t i o nS + A A l a . Show that for every npda there exists an equivalent one satisfying conditions L and ? in the preamble to Theorem 7.7. d (so. there existsan npda frwith at urost three states. A } .s. B + bBBlA. (if) .
tust be empty for evcry c € E.1. The lirst of these conditions simplv requires that for any givcn input symbol and any stack top.3 Dprnn. This does not affect the definition. at most one mov{} carr be made. 1.finrnqffi"!. l. d (g. so there may be dead rxrrrfigurations.(MJ. qo)zt F) is said to be deterministic if it is anautomaton as defineldirr Definition 7. the presenCbof . AIso.e E X U {I} and b € f. subject to the restrictions ttrat.fl \1 # It is interestirrg to note the differt:rrce between this definitiorr and the corresponding definition of a detcrrrrinistic finite automaton" The do4ra.q. The second condition is that when a Amove is possible for some configuration. =l.clptla may be to the empty set.1. $ornc: transitions of a.caPrtc"we want to {gtain )transitions. .chieved a modification rr.ig be.L is said to be a deterministic contextfree only if there exists a dpda "lb"sn'ch"tlrat t.\transitions does not automatically imply nondeterminism.11 Dcfinition 74 language if and A language .Nc.I. # . This can be ir.uacns Pushdown Automqtoond ffiffi Deterministic D e t e r m i n i s t iC o n t e x t . a.1rathertharyb E. the orrly criterion for determirrism is that at all times at rnost one possible rnove exists. d. lffim. Ar'lrr DptpRL{rNISTrc Corurnx:rFRHH L.il i A puslrdown automaton M : (Q. Since the top of the stack playsE role in determining the ncxt move. bv of Definition 7.F r eL o n g u o g e s r e A deterministic pushdown accepter (dpda) is a pushdown automatorr that neverha^r choicein its move.vINISTrc Pusunowrq Aurclnrnra.7. undefined. gf_!h!_Eg+q4g4 funqlio" iu still asin Definition7. for every g E Q. b) conttrins at rnost one eleinent.il. no inJlutconsuming alternative is available.ffif. that is.
This.since there is tlte possi}rility of irrr cqrriva. o) : {(q. 9 Lt:t L1=la"b".ta. d (qz.) : { (s' .40. aa)} a. It accept.ge../}. in cout.nguages E x o m p l e7 .b. :) { ( q r . I there is ttot dett:rrninistic because Look now elt lixample 7. l ( q ' .d. a)} violate cotrditiorr 2 of Definition 7. nol ecluivak:nt.ngrta. A. o. Thc npder.196 Chopter 7 PusIItowt't Aurtltrlare Exomple 7. The language L = I'tl) Lz . Fl'otn tiris a./o. } .r.n"l 1" i . that ilrc not deterministic... .\. and d (qo. satisfics the conditions of llelirritiorr 7. does rxrt irnplv tha't the language {trlurR} itsolf is nondetertrtinistir:. lr}. There are contextf'reela.ro.rrtlthe next exalrrplc wo see tha.r.rastttr deterministir: irrrrlnondeterrninistit:llrulidowtr automata art: finite irutorrra. ) : { ( q 1t. .l 't : 10} 6 kto.{o.l f l f r{ .{qs}) b . : ({qo.>0].u.lent dpda..t1.1 i) .Qt.u 1 (J"\ r''. : .tr.4.I)}.i ) } .\ \ .Qz}. d (116.. r ) } .zl0) and :n. .t. of course.ngrtage is indeed rxrt detrlrrninistic.L + ' l ' t .l(4r.8 The language 7:{a"b":n>0} M t:orrtcxtfiee language.1) {(qr. \ .b.stlrtl givcn lir.I'he ptlir.4 anrl is therefore deterrninistic. is a deteruritristic with {0. Brrt it is knowu that tlxr lir. o .. 6(qr.L2 is also contextfrtx:.1 obvirlrs moclification of tlrc argrrment that L1is a r:ontextfreelatrguirgrr shows that . a) = {(qe.3.t .o) {(sr.
} F':Pu{fi'eeEFlt. . rrnd so has to rnake ir. this sort of argumelt is basecl9rr a partit:ular algorithrn we havtl in mind.pproach that avoids nrr itritial crhoice' 'lb but it turns orrt that therc is not.r. but ca. shorrld be fairly clear a. Therrl is always tlte possibility of a completely clifferent a.tlPda M for L.:l DETr.la nnp DprnnulNls'l'I(i ColtrnxrFRnn 7.rrd.frce.tthis point. 9 1 1 5 2 } . Of courstl..then L: t ' l ) { a " h " c n : r z> 0 } wouldle a c. behincl the constnrction is to add to the control rtttit of M a The icfura part irr whiclt tratrsitions c:ausedbv the iuput symbol b are replacxxl sirrrilar with similrrr ones fttr input c. b* marle deterministically.ns 197 well.. Q nI Then consider f r : with (d.ut. . is context.t S # U U V2.lRMrNrsTrc LAI'rGu.t recognizes Q. 8 : a u {ao. If . 40.ontcxtfree language..gtl' This seerns rea$onable. .z. d.ncral is contextfreetu'J !+g{:t}H be prcin sentecl the ncxt chaptcr.q. .T.Lz.eginning of the strirrg or in .S1.duF. Lct G1 : (Vr.P).L wcre a dett:rrninistic rxlrrtextfreelangua'ge. .. ' . sint:tl the second part rtlsponds to c'iu the stlrrre wav as the flrst part cloesto b"'.rrinitial choice whetlter the irrput is in L1 }. sint:c the pdn" has either to match tlrre b or two aga'inst each a.L is indeed nondetertninistic' see tlis we first establisli the folkrwing claim.n npda M fbr tr. now also accepts (trrbrrcn. cxrrrrlrinitrg two. X.'Ihis until Chapter 8.52. There is rro informirtion availa.z.9.ybe enterecl rrfter M has reacl atb"'.'"bZ" Figure 7. .8.bleat the t_ry which the choice.q.2 describesthe construction graphically.T. the llrocess thir. This new part of the control utrit mar.c.PuSHDowN Aurorvt. 5. Accepting this. ir.E. we sec that' the a. rr formal argurnent ftrllows. ar. grarrlnlar G : (Yr U VzU {5} . f . it rnay letr.7.P2) be contextfreegrarn that I and rrrarssuchthat Lr : L(G) and 1. then. But .. This will follow from a ge.rgumentwill be clefirrrecl 1.tlus to the r:tlrrect conitlcture. F) with Q : t q o 'Q t . given a. hrrt cloesnot prove anythirrg. w}rtrre P:h U P z U { .ncasily be "d* plultffiHtTHis point.. .L is not a derterministic contextfrce langua. frornd by irrcluding attdfr constructed F ( 0 r A . Let M : (8. we show the la'tter try constructitrg a..F) . brrt the details of generatesLl)L2.2: L(Gil' If we assume the V2 are disioirrt and tha. s:) { ( f l ys )} .P1) attd G2 : (Vz. ..
u ) .lsobe true that ( e o .44 i.b. l'or M to accept anb" wE must have ( q o a n b n r ) i * ( g oA .The conclusionis that L : t (fr).u)} .ccept a.. . b n r l f * . with q4 € F.mains be shownthat no strings ottrer to than those irr .s) {(fr.r) (fr.But we will show in the next chapter (Uxirnpte 8. . .tr. and € : 8(ir. . But then. Therefore. i: . by r:onstruction (ti. BecauseM is cleterministic. r ) .Chopter 7 PusHnowruAtJ'rouara Figure 7.It re.r.! C) Addition Control unit of. ( q j .ru € l*. so that M wille. et EQ. I It :r{ q f r+'' sl for all ql e 4 s_ Il. I . h " . A .our assumptionthat L is a deterministic ctlntextfreelanguagemust be false. .o that i at is contextfree. im zr. for some qj E F. u)} for all ) 6 (qo. " b r n .s: {(qi. .t l r ) . it must a. .nb"cn. a so that for it to accept unbz" we nrust further have ( q r .L are acceptedby M.1) that i is not contextfree.c". . ) t * ( q i . this is considerecl severalexercises in the end of this secbion.s € 1.
{a*bz": rz > 0} is a deterministic contextfree 2} is deterrninistic. Show that [4itt E*u*ple 1 3 . Show that fr} in E***ple 7.3 is not deterministic. ffi n. but that the language in the example is nevertheless determirristic.L : {u e {a. Show that 7: language. . 7 : {a*b D.9 does not accept a"bznch with fr > 0.q. L: in=rno.Lr is deterministic contextfree and lz is regular. Give reasons why one might conjecture that the following language is not deterministic. Ianguage.9 does not accept anb ck fot k I n. b}..b}. 6 . Give a. Is the languag. : n > 1} U {a} in Exarnple 7..b. ffi LtJ Lz is deterministic Iu. the closely related language 7 : {wwR : w E {a. while the language in Exercise I is deterrninistic. Srro*that .NoDnTTRMINISTICCoNr:sxtFnEE LANGUAGES 199 I .rguments that rnake thi. Show alstr that it does not accept a"b*ca unless 'trl: n or m:2n. 7.9 does not accept any string not in ..then the * contextfree.} is known to be nondeterministic. .b}. E***ple 1 4 . I s t h e l a n g u a g e7 : {anh* in:n7 G.l stto* that if ..*g" irz/ 'J ri) V show that under the conditionsof Exercise16.Lr flLs is a deterministic contextfreelanguage.cr.Z. show that.L* is a deterministic contextfree language. .*:k) {a. or n: m}2} deterministic? 8 . show that every regular language is a deterministic contextfree language. * the language{ucwT : w E {a. cir" an example of a deterministic contextfree language whose reverse is not deterministic. Show that 7 : {6nlt't" : rn I n* 3 .B statement plausible' fiJ.L (a"bc)' 7.rg.(u) f n6 (ur)) is a deterrninistic contextfree 1 2 .} deterministic? ffi 1O.2 deterministic? ffi Show that the pushdown automaton in Example 7. Show that fif i.. 7 : {a"bn : n } 1} U {b} deterministic? 4 .. ffi "\ (ro. For the Ianguage L irr Exercise 1. Is the languag.B DnrnnvrNrsTlc Pussoowu AurouRrn .
we can easily write a computer progralrr frrr it. sfl. Supposr:we are parsing topdown. If there is no rule . ref'erring the reader to books ort r:ompilers for a more thorouglt trea. ta.4. As first casc. From the discussion thcre. But in this case there is only one surh rule. r. We can see this intuitively by viewing the pushdown autorrraton as a parsing device. we simply look at the leftmost symbol of ur2.tment. Suppose that to : rul?{rzand tlnt we have developed the sentential form tulAu.3 tt1 d2 d3 d1 t t . we would likc to know exactly which produr:tion rule is to be applied at each step. Input ro 4 . the string ru does not belong to the language. we can achieve our goal. the parsing can proceed.r sentence. . I " *+* a 1   oo. Since there is no backtracking involved.ct that they catt be parsed efficiently. we cannot immediatcly claim that this will yield a lineartime parser. We scan the input ru from lefl to right. Sentential form Matched part Yet to be matched . but if the form of the grammar is restricted. . let us see what grarnmars might be srritable for the description of deterministic contextfrec languages. To get the next symbol of thc sentential forrn matched against the next symhol in u. This would avoid backtracking and give us an efficient parser.dsynrbol. so thcre is no choice to be made. To purrrue this.4 * ay in the grammar. If there is such a rule. but it J. . For the sake of discussion.lfbrm whose terminal prefix matches the prefix of ur up to thc currently scanne.$a. this is not the case. We will provide only a brief introduction to some irrrportant results. it is clear that at every stage in the parsing we know exactly which protlrrction has to be applied. Sittce there may be . To proceed in matehing consecutive uymboftl.\transitions involved. attempting to find the leftmost derivation of a particrrla.rutsus on lhe right track nevertheless. . Herc we enter a topic important irr the study of contpilers.3. but sorrrtlwhat peripheral to our interests.Chopter PusrruowN 7 Aurou. Figure 7.ke the sgrammars introdtrtnd in Definition 5.ue ContextFree ffiffi Grommorsfor Deterministic Longuoges* The importarr(:c of deterministit: <:ontextfree larrgrrageslies in thc fir. we use the approach illustrated in Figure 7. The question then is wltether there are grilmmars that allow us to do this. For a general r:ontextfree grammar. and we rrray expect thtrt it will work t:fliciently. while developing a senterrtiir.
rucuacns 7.:ad"of the next ft . In an .S. Exomplo 7.t stands for the fact that the input is scarrned from left to right.1.10 The grammar S + oSblab is not an sgrarnma. In order to determine which production is to be applied. we mu$t apply the production S .LL gramrrlax we still have the property that we carr. The grammar is thereforc not an LL(z) grammar. Iook at the derivation of strings of length greater than two. Suppose we n()w use a lookahead and consider the first two symbols. given the currently scanned symbol and a "lookahe. we Iook at two consecutivesymbols of the input string. Otherwise. but it is an LL gralnmar. As remarked in Example 5.10. thcy are too restrictive to capture all aspects of the syntax of programming languages.n LL grammar. we can see that .bbab.In the fi. The term LL is standard usage irr books on c:ompilers. finding tlnt they arc aa.r. the first . we have available two possible productions 5 r .9 and S + aSb. the rule S +aSb must be used.9laSbl ab generates the positive closure of the language in Example 7. while in the secorrd it is necessary to use 5 * 55. the second tr indicates that leftmost derivations are constructed. The grarnma. must start with S . In a similar fashion. To see why this is so.1 The grammar S .u$uc CorrrextFnnn L. including both aabb or ao. but the concept is rrrore general. predict exactly which production rule must be used.r is rrot an LL (k) grammar for any h.FoR DorenMlr. Exornpla 7. and the second a b. One type of grammar is called a. Every sgrarnmar is an LL grammar.ris an L. If the first is an a.rst ca*se.ab.q. by looking at a limited part of the input (consisting of the scannod symbol plus a finite number of symbols following it). Does this allow us to make the right decision? The &nrJweris still no.L(ft) grammar if we can uniquely identifu the r:orrect production. To start.I syrnbols. The above is an oxarnple of an LL (2) grarrrmar. .9. The scanned symbol does rrot tell us which is the right one.4. Wc tieed to generalize the idea so that it becomes ntore powerfirl without losirrg its essential property for parsing.Rs 201 Although sgrammar$ are useful.q. this is the language of properly nested parerrthesis structures.4 Gnelru. since what we have seen could be a prefix of a number of we strings.aSb. I We say that a grirmlrra.
hq. To see this. We corrclude our discrrssion with such a definition.9 + q. Then + S + aSbS + ab. When the input uymbol examined is r:r.Lgrarrtrnar nearly equivtrlent to the original grammar.?t)z. The difliculty lies in the fact that we c&nnot predict how many repetitions of the trasic pattern a"b' there are until we get to tltc end of the string. there are always some situations that cannot be resolved. P) be a contextfree grammar.L^Lgrammar equivaletrt to the original grammar. the equality of the ft leftmost symbols of ur2 and tl3 € or implies lJt * Uzt then G is said to be an LL(h) grammar. Rewriting the gramrrrar avoids this dilliculty. We fix this by introducirrg a new start variable 50 and a production to en$urethat some nonempty string is generated. We see that we never have any choice. yet the gralnmar requires an immediate decision.baSbS+ a. is then arr . If for every pair of leftmost derivations Sl Sl 1 wtArt 41il1!!1n1 w1'*2.L grammar for the language if we alalyze the reason for the failure of the original grammar. The grammar g a$b^91. IliMl Let G : (V.we must u$e S . This observation about the grammar does not imply that thc language is not deterministic or that no . (If lu.LL grammar for it exists.'21 lur3l is less than fr.\ is an L.IalA no matter how many lookahead symbols we have.) . The final result 5'6 SaSbS aSb.7.Chopter 7 Pusnnowr*lAuroN. then h is replaced by the smaller of these. we need a more precise definition if any rigorous results are to be develoned.bS abab. when the syrnbol is b or if we are at the end of the string. we must use S + A.aSbS. S. consider the leftmost derivation of ru : aba6. But the problern is not yet completely nolved because the new grammar can generate the empty strirrg.'u)B 7*. We can find an . witlr zr1. wtAxz 1trt1y2r2 l wrtts.SlA f While this itrfcrrmal description of trtr grammars is adequate for understanding simple exarnples.L.
Construct an LLgraurnar for the languagc L (a*hu.L. the next step in the cleriva. trut can bc viewed as constructing the derivn"tiorr tree fiorn the bottom up. . : 70. 4 . But LLgranrmars are not srrflicietrtly genertr.rlv irnportant a.tion is uniquely determined (as expressed bY gr : l/z).L gramrnar.. Give. There is a great dur.L (abbb"). ffi 3 .)tt. m .assumirrg 5: (a) Z: (b). } 0} {a. Sirow that the gramrnar frrr L : lw : n* (r) : rrr. 9 . Ctlrrsequently. If at nrry sta'gein thr: leftrnost derivatiorr (w1Ar) we know the next h symbols of the input. therc is itrterest in other.11 is an Z/: grailrnr&r and that it is equivalent to the original gramrrra"r. rrroregenera. then I (G) is a dcterministic contcxtfree languagc.b.}fl. { a t t l 1 r r t . Show that if G is an /. li'irrd ari 1. Describc an algorithm whi<.(u) < n6 (ur)} (e) f. Show that a dctcrnrinistic contcxtfree larrguageis never irrherently amhiguous. Hunter 1981) or trooks specifically devott:d to parsing mt:thods for fbrmal larrguages (such ns Aho and Ullman 1972).rrr. and rnany rrrrrrpilers have het:n written using . 1 .I'EXTFREE LANcuAcES 203 The definitiorr rnakes precise what has already becn irrdicated.trz>1} (") fI : (d) Z. A nrrrntrcr of progranuning larrguagescan bu defined by LL griilrrfilars.l deal with &ll dertt:rrninisticcontcxtfree lanto guages.L gramrnar$ for the lbllowing larrguages. The topir: of .." (w) + nu (ut) t' n." {a'"tn+2rn':rr. (u)l . Show that arry LL gramrrrar is unambiguorrs.g. Show ttrat the strcond gramffrar in Exa..rs is an importa. 2 . ffi L Let fJ be a corrtextfrcc granrrrar in Grcibach rrorrual form.nt t)rre it the study of courpilers.c}.7.ldrltcrrninistic grammirrs.13 is not arr tr.[ (A) Srammar. wlrich also allow efficierntparsing..'oR Dnrnnlrrtrrsuc CON.. Ibr any given ft.L gramnrar f<rr the language in Exercise 2. Particula.(trr)) given in Examplc 1.rnrnars. ffi i).Lpilrsers. tletermines whether or not G is an Z.mple 7. : {tu : n.h.rether socalled LR grir.0 } W } tzlt*7'n+"n: n. {a. 6 .L. r t t " r : n Z 0 .4 GRAMMARSl.l of material orr this subject that can bt: found in books orr compilers (e.LL gramma.L (k) grarnuar. = {u : n.
.
To study the relationship between langtage families and to exhibit their similarities arrd differences. there are broader language families of which contextfrce larrguages are a special case. 205 .sfor classifying specifit: languages iu an appropriate category. On the one hand. wc look at closure urrder a variety of operatiorts. algorithms for determirring properties of members of the family. As in Chapter 4. and structrrral results such as pumpirrg letnmas' These all provide us with a mear$ of understanding relations between the different families as well ir. (:orrtextfree languages include important but restricted language farnilies such as regular and deterntinistic contextfiee languages.o Properties f ContextFree Longuqges he family of cotrtextfree Ianguages occupies a central position in a hierarclty of formal languages. On the other hand. we inveutigate characteristic properties of the various fatrilies.
f.Therefbre. 1.. Then there exists some positive integer rn such that atty ut F L with lrul > ?n calr be clccotrposed as (8. lJince the Iength of thc string on the right sidc of any production is bounded. a l l t h e s t r i n g s u ' u i .unrtrz.\}.the other for a restrir:ted type of contextcgntextfree languages free language. 2 .2) er. there exist artritrarily long derivatiorrs and corresponding derivation trecs of arbitrary height.. . the Iengtlr of tltc clerivation of any'ur € tr must be at least lwl lh.1. 1 . thcrtr rmrst be some va.. say by k. Since the lrumber of va.c a.2.tedby the graInIIIaI arrcl tlre therefore in L. . sur:h that (83) .ria.L be an infinite r:ontextfiee language.' r.ndsome sufficienbly long path in from the root to a lca. . . wlrert. 'uu"x:'!'z E L1 /R d.Chopter 8 Pnopnnuns oF CoNTlJxrFnnn LeNcuacns ffiM Two PumpingLemmos The pumping lemtna girrcnitt lThe. or Aproductions.Iiable that repeats on this pathr as shtrwn sclrematically irr Figure 8. . since L is infinite.oEm ?37is an effectivctool for showing Similar pumpitrg lrtmmas are knowu art: not n that we will discuss twtl strt:tr resrrlts.1. we have the derivation Sl uAz l uuAgz A'u.l. I t I AI and z art: tr.1) with (8.I fbr all i : 0. .hles G is finite.nd lr vl> 1. Proof: Consider the language L . c a t t be gerrera. Flom the above we see u t l r a t A 3 u A y a n d A 5 .. " .ll strings of termitrals. Consider ttow sut'h a high derivation tre. ' g i z . This is known asthe pumpirrg lemma for contextfree languages.{. Correspotrdittgto the derivation tree irr Figrrre 8. .r.r l : 0 . one for irr gonertr. [onguoges A PumpingLemmofor Conlexl'Free Let . Frrrthermorer in the . ard assurne that we h*ve for it rr grarnmar G without utritproductions. * .
8,1 Two PuuPtllc LetuN,Ias
207
Figure 8.1 Derivation trec lbr a long string.
+
\A)
X
/
\
derivations A I uAy and A 4 ,r, *e calr assuilte that no variable repeats (otherwise, we just use the repeatitrg variable as A). Therefcrre,the lengths of the strings u, fi, and y depend orrly on the productions of the granrmar anrl carr be bounded independently of ur so that (8.2) holds. Finally, since there are no unit productions and no.\ productions, u and y cannot both be ernpty strings, giving (8.3). This completes the argument that (8,1) to (8.a) hold. I
This pumping lemtra is useful in showing that ar,larrguage does not belong to the family of contextfree languages. Its application is typical of pumping lemrpas irr generall they are used negatively to show that a given Ianguage does not belong to some family. As irr Theorem 4.8, the correct But argument can be visualized as a game against an]Urlellig€@!. now the rulos make it a little more difficrrlt for us. For regular languages, the substringzE\hose length is bounded by rn starts at the left end of zu. Therefore*tliEiubstring gr that can be pumped is within rn syrrrbols of the beginning of ur. For contextfree language$, we otily have a bound on luryl. The substring u that precedesura carrbe arbitrarily long. This gives additional freedom to the adver$ary, rnaking arguments involving Theorem 8.1 a little more complicated.
Show that the language [,:{u,nbncn:rz>0} is not contextfree.
Chopter I
PRopnnrtns on ConrnxrFREE Lancuacns

lT*.,/ ),g,,,,' ,".'dir' i''r*lir'lr+r
lHi*: . Dc.t :,!rt,r;rH,H"l;:T""Tl:'i:,ffiT:i lffiJlil,':,,';*#il1iJ *Y, L umpinglemma.Thattris ,
suchasthe con$truction from someother argurnent, must conre contextfree
of a contextfree granrrrra,r. The argument also justifies a claim made in Example 7'9 and allows us
,  .r r q rr Lr{ a E I
which is Once the adversary lrqs choserrrn,,we.pick jl: ttli"efq3id! in tr. The adversary now hasEETffilcltoices. If he choosel urq to contain only a,'s,then the pumped string will obviously not be in ,L. If he chooses 4 slriqg rnntaining a,n equal number of a's and b's, then the pumped string k #;.,ot L* generated, and again we have [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[fJ*ith generaterl a string not i1 .L. In fact, the only way the adversary could stop us froilI wirrrring is to pick ufra so that u3rhas the same number of a's, bts, and cts. But this is not possible becauseof rrstriction (8.?). Therefore, L is not conte4!free. If we try the sarne argutrrerrt on the language 7  {a"b"}, we fail, as we must, since the language is contextfree. If we pick any string in tr, such as u i : e , " ' b " ' , t h e a d v e r s a r y c a t r p i c k r , : t t r ka n d g r  b A , N o w , n o m a t t e r w } r a t
to closea gap in that examPle.Tlq
L: {a"b'} {a^bz^}  {anb"cn}
is not ctrntextfree. The string artlbrnctrl is irr Z, but the pumped result is not.
T
irixUd$lle,raigi
Consider the language
L  {ww : tu
:glt]
Although this language appears to be very similar to the contextfree language of Example 5.1, it is not contextfree. Consider the string a*lfna*b*, There are many ways in which the adversary can now pick uzg, but for all of them we have a winning countermove. For example, for the choice in Figure 8.2, we call use i : 0 to get a string of the form akYa*b*,h < rn or i < m, which is rrot in .L. For other choices by the adversary similar arguments can be made. We conclude that ,L is not contextfree'
8,1 Two Pulrerrvc Lnrvruns
209
Figure 8.2
a...ab...ha...ab...h
F_i_]___+
ilv fr! E
,
nul
n_st
r'eilr rtr
{a"t:n>0}'/
l_p
c.t
is not contextfree, In Example 4.11 we showed that this language rg11g!_tgg!g:. However, for a language over an alphahet with a single symbol, t[eEEfrttle clifference between Theorem S.l a,nd the prrmping lemtra for regular languages. In either case, the strings to bt' 1'r'rped corrsist entirely of ots, and whatever new string can he gcncrated by Theorern 8.1 can also be generated by Theorem 4.8. Therafore, we can use essentially the same arguments as in Example 4.11 to show that .L is not contextfree.
r
t
.\ r ,,,1* *.lo*
fr,#sthplqS,,4 Showthat the language
<J
,
I '1
r^ bb 'trra lrrrcroao6a
t
, 5lf.
is not contextfree. Given rn in Theorem 8.1, wc pick as our strirrgitr,r"'6n,)The adversary now has several choices. The only one that requiris mffh thought is tho
q4ovrn_illFigur* S.?..$q!11g11!jry will yield glr€wstriue;v/ith ,on=q lg + \i  I)kr a'sJand + U I)hz b's.lIf the adversary takesk, I 0, W E;FO@TIffi i:0.Sinrer (*  hr)'g (*  1)' = n 7 , 2 m * r 2 < rrf  lq, the result is not in tr. If the opponerrt picks A1 : 0, frz I 0 or h * 0, hz : 0, then again with i : 0, the purnpedstring is not in ,L. We can concludefiom this that .L is rrot a contextfreelanguage. I
210
Chopter I
PRopeRrtrs or Corutnxl'FREELRwcu.+cps
Figure 8.3
# n  j_E#
u i l l r l
]
d . . .d . . . a . . . a h ' . ' h ." b . . ,b z
A PumpingLemmofor Lineqrlonguoges
We previouslymade a distinction betweenlinear and qog}llrj3rg4textfreg grammaxs.We now maftea similar distinction betweenlanguages.
IIM
A contextfree language .L is said to be linear if there exists a linear contextfree grammar G such that .L : L(G).
. ,1, C'l
Clearly, every linear languageis contextfree,but we have not yet esis tablished whethet or not the converse true
tl ,.''ri.',r )),,.*
I
r?> 0) is a linear language' A linear Brarnmar for it is given in Example 1.10. The grarnmargiven in Example 1.12for the languageL : {.; no (tu) : nr,(tr)} is not linear, so the secondlanguageis linear' not necessarilY I _
Of course, just because a specific gramma,r is not linear does not imply that the language generated by it is not linea,r. If we want to prove that a language is not linear, we mu$t show that there exists no equivalent linear grammar. We approach this in the usual wa5 establishing structural properties for linear languages, then showing that some contextfree languages do not have a required propertY. Let L be an infinite linear language. Then there exists some positive integer ?n, such that any w €. L) with lurl ! m can be decomposedas w:uafrllz with
luuyzl 1nt',
lual>r,
(8 5) (8 6)
8.1 Two Purr.rprr{c Lnlrues
ZLt
suchthat uu'ny'z € L, (,$7)
f o r a l l z : 0 , 1 ,2 . . . . . Note that the conclusions of this theorem differ from those of rheorem 8.1, since (8.2) is replaced by (8 s). This implies that trie strings u and 3r to be pumped must now be located within rn synrbols of the leff and right ends of ru, respectively. The middlc strirrg z can be of arbitrary length. Proof: our reasoning follows the proof of rheorem g.l. since the language is linear, there exists soure linear grammar G for it. To use the argumerrt in Theorern 8.1, we also need to claim that G contains nei urritproductions arld no .\productions. An examination of the proofs of Theorem 6.8 and Theorern 6.4 wilt show that removing Aproductions ancl unitproductions does not destroy the linearity of the gra,mmar. we ca.rrtherefore assurre that G has the required property. consider now the derivation tree as shown in Figure g.l. Because the grammar is linear, variables can appear only on the path from s to the first A, on the path from the first A to the second one, and on the path from the second .4 to sorre leaf of the tree. Since there are only a finite number of va"riablers the path frorn 5 to the first 4, and sirrce each of on these gerreratesa finite number of terminals, u a'd z rnust be bounded. By a similar argurnent, u attd y are bounded, so (g.b) follows. The rest of the argument is as in Thcorern 8.1. I
llH$$$si$.Niiil rhuru,.g,,us*
is not linear. To show this, assume that the langua6Jeis linrrar and apply Theorem 8.2 to the string
,uJ : arnbzrrl(f,ttr,
Inequality (8.7) shows that in this cas. thc strirrgs ut Dt a must a,llconsist at entirely of a's. If we pump this str'ing, we get orrt,lk62nzont+t, with either h > 1 or I ) t, a result that is not in tr. This contracliction of Theorern g.Z proves that the languageris rrot linear. I
F
2r2
Chopter I Pnt)pr:nrrr:soR Cot't:rsxrFRenLeNcuecns This exatnple answers the general qtrestion ra,iscd on thtt relation beof tweel the f"tr,milies contextfree and linear languages' Tlttl farnilv of lineflr is Iangrrelges a, proper subset of the family of contextfree larrguagtrs.
1. Use reasonirrg similar to that in Exarnple 4.1I to give a complete proof that the language in Examplc 8.3 is not cotrtextIree. that the langrrage L : {a* I rz is a prime trutnber} is not t:orrtextfree. @Stto* 3, Show that L : {trr.utrttt : w E {a,b} } is not a cotrtextfi'ee larrguage W 4. Show that .L : {ur e {tl, b, c}" r n2"(*) + "3 (*) : til (to)} is not contextfrcc, 5. Is the language 7  {anb* : n * 2*} corrtextfree? 6. Show that the language I: q (3) {o"' , " } 0} is not coutextfree'
Show that the following languagcs on X = {a, b, c} u,re not contextfree.
( a )I :
{a*ll ; " <.i'} m
(b) I : {alhi :" > (j  1)'} (c)L:{anbich:k:jn\ (d) /,: {a'obich:k>n,f >i}
( e ) L : { a ; " h ic k : n I j , n ! h S . i } (f) .L : {ru : n,,,(tu) < nu (tu) < n.. (tr)} (S) (e) f  {w: no(w) lrr't,('u,): n" ('u')} (h) Z, : : {u € {4, b,t:}* : n,,.(zr)* nr (tu) 2n" (tu)}'
,/
)
,{\
!dr)
Deternrinc whether ol not the following languagcs are cotrtextfree (a) Z,: {a'f"uuHa't: n,} 0,u e {a'b}} ' (h) L:,{a"'bia^H (c) 1, : : ? r} 0 , i l 0 i W
> 0} {a'ltirtib" : tr.> 0,j
(tt) L : {a"H akht I n + i t k + I} (c,) L = {anbi ahlsl,r, a k, J < I} j} (f) r: {u"'b"t! ,," < g. I1 Theprenr 8.I, find a bound for rrz in tertns of the properties of the grammar G. m nctcrmi*e whetrrer .r *.t t6e followi.g language is cnntextfree. ffi
^ '{ ,
i e [, = {.,w1ttn2 ,u]r1,ttz {4, b} ,wt I wzl
8.2 CLosuRePRoreRrlns ewo Dncrsrol Ar,conrrHMs FoR CournxrFnpn LaNcuecns
213
b q
drb V QH
Sfro* that the language7 : {a'ob"a"ob'* n 2 0,m > 0} is contextfree ; but not linear. Show that the following languageis not lirrear.
t,: {w : n*(w) > nu(.)} W is
1 3 . S h o w t h a t t h e l a n g u a g eL : contextfree, but not linear. \' D (4.
{w € {a,b,c}*:rro (w)+na(*):rr"(u,)}
 1} islirrear. {a"bt :i 1,n<2i 15. Determine whether or not the language in Example 5.1? is linear. ffi O e t e r m i n e w h e t h e r o r n o t t h e l a n g u a g e7 = 16. In Theorem 8.2, find a bound on m in tenns of the properties of the glarrrrrrar G.
17, Justify the claim made in Theorem 8.2 that for any linear langrrage (not rxrntaining A) there exists a lirtear grarrtrna,r without }productiorrs arrd unitproductions. L8, Consider the set of all strings af h, wherc a and b are positive decimal integers such that u I b. The set of strings therr represerrts all possible tlecirnal fractions. Determine whctircr or not this is a contextfrec languagc.
* 19, Show that the complement of the language in Exercise 6 is not contextfree.
\t
@
t* the followinglanguage contextfree?
L: {o""' : rz and m, are prime numbers} W
C l o s u r eP r o p e r t i e s n d D e c i s i o nA l g o r i t h m s o L for ConfextFree onguoger
In Chapter 4 we looked at closure under certain operations and algorithms to decide on the properties of the farnily of regular languages. On the whole, the questions raised there had easy arrswers. When we a,sk the same questions about contextfree la,nguages, encounter more difficulties. First, we closure properties that hold fbr regular,rlalnguilges do not always hold fbr contextfree larrguages.When they do, the arguments needed to prove them are often quite cornplicated. Seconcl, many intuitively simple and important questions about srntextf'ree larrgrragesc:artrrottrtt arrswered. This statement rrray seerrrat first surprising and will need to be elaborated as we proceed. In this section, we provide only a sample of some of the most important results.
longuoges Closure ContextFree of
is The familv of contextfierela,ngrrages closed und(lr union, ctltrcaterratiorr, arrd starclosure,
.9) (8. S r .2.L2 and it follows that . rxlnsider . Cotsider rrow the larrguage I (Gr). u'I'2. gerrerated by the grammar Gr : (H u V2 U {S.nd 1. r. if (8. Suppose for instance that u € trr. G+ : (Yr u Yzu {50}.il'.8) rmrst trr: the first step of the derivation. Since sentential ftrrrns durivcd fiom Sr have variables in I{.10) is used first.} . Nr:xt. But it is ca^sy see that to L(Gs) : Lru Lz.I0) (8. Alternatively. the derivation SrSru can involve productions in P1 only. Wt: c:arra^$$urne without loss of generality that the sets V1 and Vz <rredisioint. T z .Sa.L (G3) is the rrnion of L1 a. 63 is a contextfree grammar.2L4 Chopter 8 Pnornnuss ol' CoNrexrFRen LANcuAGEs Proof: Let trr and L2 be two contextfree languages generated by the c o n t e x t . u 72.gain Sa is a new variable and P+: Pt U Pz U {5i SrSz} . AIso. Hence trr must be in L1. P 1 ) a n d G z : ( V z . 1f w e L (G3) therr either 53 + ^91 or 5h + 5'z (8. The productions of Gs are all the prochrt:ti{rrrrt G1 a.Pa). Suppose(8. More precisely. together with an alternative starting production that allows us tr) us(l orro or the other grammars. so that L (Gz) is a contextfree languagc. P:t: Pr U Pz U {5r * 5rl5z}. then tu must be in. e e G r spcctivcly. A similar argument can be made for u € L2. wlrere S'3 is ir variirtrlc rrot in V U Vz.rrcl of G2.9) is used. P3) .93. ? r .f ' r eg r a m m r r n J r : ( V r . .. Obviously. Then S:t+Sr4ru is ir possiblc derivation in grammar G:J. P 2 ) . Here a. ancl V1 and V2 are disjoint. S 2 .
Then I ( G s ): L ( G t ) * . starclosure. P5) where 5'5is a new variable and Ps: Pt U {S5.2 ClosunE PRopERTTES DucrsroN AlcoRrrHlrs AND FoR CoNTEXTFn. r and The family of contextfree languages is not closed under intersection and complementation. But L1 t L2: : {unbncn n } 0}. Thus we have shownthat the farnily of contextfreelanguages closed is under union.5155l. which we have already shown not to be contextfree. For instance.7.3.5'2. Proof: Corrsider the two languages L 1 : { a n b " c n ': n Z 0 . Thus.en LANGUAcES 215 Then L ( G + ) : L ( G r )L ( G r ) follows easily. concatenation. r n } 0 } and L2: { a " b * c * i n . Finally.8.55.L1 and "t2 are contextfiee. consider (Gs) with I Gs : (VLU {Ss} . 5r i a5rblA. 5z + c5zlA' Alternativelg we note that tr1 is the concatenation of two contextfree languages.r n > 0 } .1. } 0 . There are several ways one can show that .U. . the family of contextfree languages is not closcd under intcrsection. a grammar for L1 is S * Sr. so it is contextfree by Theorem 8. .
action of M1 and M2: whenever a symbol is read from the input string. the states of M are labeled with pairs (U.F"l) be an npda which accepts Lr and (P. we also require that if a : tr.E. e d1(qa. a. representing the respectivestates . dr. 0o: (go'po).t. X. In this. * t': I\x Ezt and define d such that o . b ) . But this contradicts what we have just Shown. the closure property holds if one of the languages is regular. In other words.I.3 and the set identitv ::=. q0. . then the right side of the above expression would be a contextfree language for any contextfree L1 and L2.L2' We construct a pushMz: qo.dz.po.r language' Then L1 tt L2 is contextfree. .. p r )r ) e 6 ( ( q a . p r ) .8.: L 1 t 1 L 2 :L t U L z . .f.r.L1 be a contextfiee language and tr2 be a regula. b) . Let . a ) : p t .pi). if and only if (gn. that the intersection of two contextfree languages is not necessarily contextfree. then py : p7. ( ( q r . r While the intersection of two contextftee Ianguages may produce a Ianguage that is not contextfree. ffi simultaneously executes the moves of M1 and M2. r) and 6 z( p i .Chopler I PnornRuns on Cowrnxrl'Ree LnxcuRcps The second part of the theorem follows from Theorem 8. Consequently. To this end we let e:exp.F) which simulates the parallel down automaton ffi : (0. Proof: Let M1 = (8.s.Fz) be a dfa that accepts. If the family of contextfree languages were closed under complementation. the family of contextfree languages is not closed under complementation.
( g . nd d* (Po.q.z ) .Fr and p" € Fz if and only if ( q o .FID Dnctslol AlconrrHMs FoR ColrrexrFREE LANGUAGES 217 in which Mr and M2 (:arr be after reading a certain inprrt string. . Therefore. We can get a muclr treater argument with Theorem 8. if it is in tr (Mr) ) L (Mr) : Ltfi Lz. € .. w .5.2 Closunn PRoRnnrlns .w) : p".8. it is easy to see that. P " ) . Also. ( ( q ' . It is a straiglrtforward inductiorr argument to show that ( ( q t " P t ' )' u ' "z ) i . dtlr q. 7. This closure property is sometime$ useful for simplifying arguments in connection with speciflc languages. Exomplo 8.:{a"bn:n>0}1121. n ) . Therefore. by the closure of regular languages under complementation and the closure of contextfree languages under regular intersection. but the process is tedious. I The property addressed by this theorem is called closure under regular intersection. Then. It is possible to prove this claim by constructing a pda or a contextfree granrrnar for the langrrage. " ) i . because trr is finite.nlI00j is (:tlntr:xtfree. the desired result fbllows. I . Let L t : { a l o o b l o o }. a string is acceptecl Uv fr if and only if it is accepted by Mr and M2. Becauseof the result of the theorem. it is regular. that is.7 Show that the language trla"h"rz)0. we say that the family of contextfree languages is closed under regular intersection.
r. I (VrTr^S. Then L f t L ( a .\ # L(G). For the purpose of this discussion.2 and 6.. there exists an algotithm Giverr a contextfree grammar 6: for deciding whether or not . The pumping lemma can be used for this. if not. We use the algorithm for removing uselesssymbols and productions. Some additional results are explored in the exercisesat the end of this section.L were contextfree.Slight made in the argumerrt if this is not so.\N showrhat the languase L : {* € {cl. * b * c * ): f a ' b " c ' : r z l 0 ] would also be contextfree. no unitproductions.^9.then tr (G) is ernpty.P)' there exists an algorithm Given a contextfree grammar 5: fbr determining whether or not I (G) is infinite.rucuRcns W[*Blii$l'tt. (V.q. Proof: We assume that G contains no .Chopter I PnornRTrnson CorcrnxrFREEL.L (G) contains at least one element. I Closure properties of langrtages play an important rolc in the theory of fclrrnal languages and many more clt$ure properties for contextftas languages can be established. we have already established the existence of a rnembership algorithm for contextfree languages' This is of course an essertial feature of any language family useful in practice' Other simple properties of corrtextfiee languages can also be determined.P).6. we a$$ume that the Ianguage is described by its grarllma. . changes have to be Proofr For simplicity. Suppose that .\productions.T. and no useless symbols.)} is not contextfree. But we alreadv know that this is not so. c}* : no (*) : n. Suppose the grammar has a repeating variable in the sense that there exists some A e V fbr which there is a derivation Al rAs. then . (tr) : t. but agaiu we can get a tnuclt shorter argurnerrt urting closure under regular intorsection. Wtr conclude that tr is not contextfree. If '5 is found to be useless.L (G) is empty. (". b. Longuoges Properties ContextFree of SomeDecidqble By putting together Theorems 5. a^$sume that.
osuRril zfg to Since (J i$ ir. Iarrguage.mmir.ritlblos in sucir a way tha. Is thc conrplernent of the languagc in llxamJrle 8. I'his can be done sirnply by drawing a depenrlerrc:vgraph for the va. the gra.iAlcoRt't'HMSFoR.sa.other sirllllt: properLiesof contextf'rtlclanguages rrr(: rrot so easily clealt with. Show that this larrguage is litrcar Consider the language . and a trre irt 7'*..'oltrnxrFnnE La.n tllgoritltrn for detertniuing wlxrtlrcr I (G) is finite.sstturt: rcpeating varia'bles.8 contcxtfree? ffi 'lheorem B.. grilrnrnitr ltas wc a repeating va.te rr.n a. rrscless svmbol.(. we rreed only to cletermine whclthcr the gramtnar ha.But it trrrrrs rnrt that there is rrclsuc:tr we do uot have the ttx:hrrical tnachiner:vfirr ltrtlllerly clefinirrgtiro rnt:arritrg of "there is no ir. But then S 1u. Sintxl A is tteither nullable nor ir.Pnopnnrrns eNr Dncrsror.ble r:an cver repeat. cycle is ir repeatiug one. where. so thrr.4. wtl rniglrt look for a. Since we now have an algorithm fbr rlu:itlittg whether tt.7.rrtltto unitproductions.$srrrned have no Aproductions ir. 1 .riahlcrif rrrrd otily if the clepelncltlncy graph ha.L1 in .lgoritlrtn.lgorilhn.rierblc. ?. have an algorithm frrr rlctertnining whetlrcr tlr ttot I (C) is infirritc. u. As itr I'heoretn 4. For the mtlrntllrt. we have SluAuSut and A1z.t7cilnnot bc sitttultatreously emptv. 'I'hen any varia.trle that is rrt' Lhe base of a. I Sornewhatsurprisitrgly. then the length of irny derivation is boundecl by lI/1.t I (fJ) is infinite. r and . to get a.cyt:lo. If no varia.Au 3 ux'''Ay"u 3 ur"zynu is possible for all n." but its intrritivt: rncatrilg is clea.2 Cl.L (G) is finite. This is an important poirrt to which we will retrrrn lirter.rhirs ir repeating va. In that t:itsc.r.!t. Corr$e quently. I'hus.lgtr the sarntl ritlrrrr to deternrine whcthcr two contextfree grilrrrrnarsgenera. R) whenever thcre is a corresponding production A +:nB.t therc is att edge (A.NcuncF:s 8.
Show that the family of litreat languages is closed under union..r . {anb": n } 0. *13.n is not a multiple of 5}iscontext 18.[s it necessarily true that . brrt is closed under regular difference.220 Chopter I Pnopnnuns ol CoultLxr!'nnn LANGUAGES 3. Show that the family of linear larrgrragesis not closed under intersection. Show that the family of deterministic contcxtfree larrguages is not closed utrtler union and intersectiorr. r:arr determine whethcr or not A e I (G).: n.Ls is regula.5. Give an algorithrn which.r. that is. ffi 1O. Let . Which of the languagc faurilies we have discussed are not closed under reversal? 7.: be regular. W Show that if . Let Lr be a contextfree language arrd .L. for any giverr trntextfree grarnmar G. 11. {S 22. Show that the farnily of contextfree languages is not closed untler difierence in gerteral.L be a rleterministic contextfree language and defirre a new languagc L1 : lw : aw € L. ffi 16." ('u) : rr. 7 : {tu e {4.Lr is a tleterrninistic corrtextfree language'l 17. Givc the cletails of the inductive argurnent in Thcorcm 8. 23. but not closcd unrler concatenation.Lr arrd . show that therc cxists an algorithm to dctermine whether the language gerrcrated by some contextfree gratrrrnar contains any words of length less than some givetr llumtrer rr.Lr is rrrntext*free and . Show that the farnily of unarnbigrrous contcxtfree languages is not closecl under interset:tion. Showthatthelarrguage : 7 free. I' Show that the farnily of deterministic contextfree languages is closed urrder regular difference. Slxrw that the family of contextfree languages is closecl under reversal. 14. Show that there exists an algorithm to deterrrrine whether or not . 15' Cive an example of a cotrtextfree language whose complernent is not contextfree.L2is a linear language. 4' Show that the family of linear lartgrrages is closed under hornorrrorphism. Show that the family of urtarrrbiguons contextfree languages is not closed under urrion. ffi 6. . then /. 21. 12. 9. if .a € X].L1.2 is regular. Show that the family of t:ontextfree languages is closerl under homornorphism.L2 have a cornrnon elerrrent. b}.t (ur). .Lr is lirrear and 1.tu does not cttrrtain a substring aab} Is the farnily of deterministic corrtextfree languages closed under homornorphisrn'i 19' 20. 5. therr .Lz is contextfree. Slxrw that the following language is contextfree.
Ianguages and investigJatehow one might define new Ianguage families that picture of an inclrrde these examples. If there is no storage. If we compare flnite see that the nature of the temporary storage creates the difference between them. and therefore. where our results showeclthat some simple languages. We also saw that contextfree languages. 22r . such as {atb"c"t} and are not contextfrtle. we return to the SEeneral arrtornata with pushdown automata. For example. we automaton. Extrapolating from thiS observation. a. To do so. we have a finite automaton. Our study has revealed that the regular languages form a proper subset of the contextfree languages. while firndamental to tha study of progranmfng languages.re limited in scope' This was made clear in the last chapter.ta are more powerful than finite automata. we have encountered $Qmefundamental ideas. This prompts rrs to look beyond contextfree {**}. we can expect to disCover even m()re powerful language families if we give the automaton more flexible storage. if the storage is a stack.Turing M o chi n e s n the foregoing discussion. that pushdown automa. we have the more powerful pushdown automaton. in particular the concepts of regular and contextfree languages and their association with finite autontata and pushdown accepters.
T[ring rrrachine is quite rudimentary. we rrsrxl two stacks. the autornatorr that we use as a Thring machirre will have neither an input filc nor any special output rnu:hanism. T h eS t o n d o r d u r i n g o c h i n e T M Although we carr cnvision a variety of automata with complex and sophisticaterdstorage deviuru. eac:hof which is caJrrrbleof holding one symbol.n read arxl write a single symhol on each nrove.apesused in actual curnputers. whatcver input and output is necessarvwill be done orr the machirrc's tape. in turn. Associated with the tape is a readwrite head tha. thrne stacks. a. This array extcnds indefinitely in both directiorrs and is thercfirre capable tlf holdirrg an urilirnited amourrt of infbrmation. . rnost of whit:h turn out to bc rrrrinteresting. whilu the mechanisrn of a. we could retairr the input file arrd a specilic outprrt mechanisrn without affecting any of the corx:lusions we artl rrbout to draw.t can traval right or left orr the tape and tha. The discussibnr:rrlminatesin the Turing thesis. we will see later that this modificatiorr of our general model in sec. To deviate slightly frorn the general schcme of chaptcr 1. but we leave them out becausrl the resulting automaton is a little easier to rkrscribe. Ttrring rnar:hine. irr the general scheme of Figure 1. thel develop sorne feeling fbr whtrt is involved by tloing some simplc programs.tion 1. to a precise elefinition of the idca of a nrechanicalor algoritlunir: computation.222 ChopterI Tunlnc MACHTNES what wtlrrld happen if.rrtomaton and tlrrough it a new langrrirge family? Tlis approach raises a lirrge number of qucstions. Next wc errgrrethat.lJsuch a storage device a tape hecauseit is analogousto the magnetic t.Thring rrrachine'sstorage is actually quite sirnple. We bcgin our study with a fbrmal defirrition of a. Definition q Turing of Mochine A Trrring rnat:hineis an automaton whose ternporary storage is a tape. a queu(]. what carr we $ay about the rnost powerful of arrtomata and thc limits of computatiorr? This leads to the funda.2 is of little r:onsequence.3. It is more instructivcl to ask a more arntritirlrs question and rxlnsider how far the concept of arr arrtomaton can btr ptrshed. onedirrrensional array of cells. which maintains thirt any urmputational llrocess.mentalcorrr:r:pt a Turing of machine and. each of which can holtl tr single syrnbol.such as those carried out by presentday cornprrters. the concept is tlroad enough to c:oververy complcx processes. The infornmtion can be read irrrd changed irr any order. we will ca.or $orneother storage device? Does each utorage device tlcfine a new kind of ir. t:elnbe done on a T\rring machine. It cart lle visualized as a single.t ca. rhis telpe is divided into cells.
d is a pa. ! e f is a special symbol called the blank.1. that the input alphabet is a.r irrput for reasons that will become apparelrt shortly.Inc MacntNn 223 Figurc 9. I is a finite set of symbols called the tape alphabet. In the rlclinition of a Ttuing rnachine. ( t . F C Q is the set of firral states.1.1 THn Srer"rrrARlrTuR.rtiirl futrction on Q x f..R}. The arguments of d are the current state of the control unit and the current tape symbol being read. . that is.1 Reedwrite head A dirrgrani giving an intrritivtr visualization of a T\rrirrg rnachiue is shown irr Figure 9. its interpretatiott gives the prirrciple by which a. Irr gerreral.srrtrsetof the tape alphabct. X . F ) . d is the transition function. where Q is the set of intcrnal states. f .1 rnakes the notion prccise' A T\rrirrg tnachine M is derfirrtxlby M : ( Q .i l .q 6 . I)efinition 9. X is the input nlphabet. The transition funt:tiorr d is defined as d:QxfQxfx{. Thritrg tnachine opcrates. not including the blank.9. Blanks are ruled out a. a new tape sYmbol. The result is a new stattl of tlte control unit.we assumethat E ! f * {n}. qo E I is the initial stntc.
rry ttut ttre very limited: it can $ensea symbol on its tape and use the result to decide what to do next' Tltc onlv actions the machine can perform are to rewrite the current symbol. I . L rtr R.n cs.It) .Chopter 9 TunIuc MACIIINE$ Figure [. It then gotls through a sequenceof steps controlled by the transition lunction d.f ii*tnele +.The move syurbttl indicates whether the readwrite head moves left or right one cell after the new symbol ha^sbeen written ott the tape. flnternal state7o f Internal smte fI I l'l'l'l l'l'Fl(b) which replaces the old orre. this is possible because d is a partiat function. to ctrange the state of thc control. the conterrts of any cell on the tape may be cxamined and changeclmany times' Eventuallv. and wc often call it the As always. which has a finite llrelnorYr and in its tape. This small instflrction set may seem inadequate lor doing complicated things. It has a processing unit. r.1 Figrrre 9. so the T[rring machine will ha]t whenever it enters a final state. it has a secondary storage of rrnlimited capacity. We can think tlf a Thring machilre as a rather simple computer. we will assume that rro transitions are defined for any final state. In fact. and to move the readwritc head. Tlte transition function d defines ttprogram" of the machine.lll.2 showsthe situation before and after ttNr move causedby tlx: . During this process. the automaton stait$ in the given initial state with sorne informatiorr on the tape. T\rring machines are quite powerfirl in principle.i (qo.rJ. but this is not so. the whole process may ternrinate. which we achieve in a Tlrring machine bv putting it into a halt state. Tlrg instructiols that sut:h a c6mputer ca. and a move symbol. A Ttrring maclfne is said to halt whenever it reaches a configuration for which d is not defiiled.2 The situation (a) before the move and (b) after the In()ve. how this computer acts.o) : (91.
b. it v/ill move left one cell. tr) . tr. E.o) .ffin#Ili. b d (qo. H. If this T\rring machine is started in state qs with the symbol a under the readwrite head. R d (qo.. tlte applicable transition rule is d (go.fi.b } .qr} .rlwrite head will replace the a with a b. b .b' ft)' Therefore the rea. then move right on the tape. ft) . E) . L ) . f:{a.r.a.o) : (go. Define dbv d (qo. ) .L).L) .tnRRo 226 Figurc 9.but let F be empty.n) : (qr. s4ren the machine errcounters the first blank.b. d (qo.(q1..Hi$" Take Q.a. tI (qr.b) : (go.tr}. a) : (qo.b.b.TunInG MlcnIbrn 9. n.3 shows several stages of the process for a simple initial configuration.(q1. ft) . I as definedin the previousexample.q.!) : (sr. ) . r : { q r }' ano d (qo. ft) .il) : (qs. The machine will remairr in state q0. . 5 : { a .1 Tnn Sr. d (sr.o) : (qo.3 of A seqrrerrr:e moves. Any subsequent a will also be replaced with a b. I . then halt in final state q1' Figure 9. h't(HtilFld f t Consider the Tirring machine delined by Q : {qo. d ( s r . b ): ( q o . E. but b's will not be modified. d (qo.
This is an instalxjc of a Ttrring rlachine that does rrot halt. we will look at other version$ of Ttrring nratrhines and discrtss their relation to our standard model. 2. 'fhere These corrvnntions were cltosen primarily for the convenience of subse* qrrent discussiorr.4. I Since omr can make several different definitions of a Tirring machine. We will use the notation in which frtIrz or ' ' a t f l z ' ' ' a k . We a"ssumethat at the initial time the tape has some specified r:ontent. Its next strrte is {1 and the readwrite head moves right. As an analogy with programmirtg tcrminology.t the T\rring machine is in an infinite loop. Whenever tlte machine halts..b. whatever the initial information on its tape.. brrt does not change it.se. The symbols et. it is worthwhile to nummarize thc main features of our model. h o .. The machine then reads the a. The Turirrg machine is detcrministic in thc sensethat d defines at most one move for ea. is no special irrput file. Suppose that the tape initially corrtainsc. 1. we uso the idea of an irutantaneous description. but making no rnodifications to the tape. This convention is chosen so that . we say tha.Chopter TuRrNc Macmbru$ Chopter 9 Tunrr. This symbol is also rt:ird and Ieft unchanged. trnd the sequence of moves stilrts again. The T\rring machine hils a tape that is unbounded in both directions.. and the positiorr tlf the readwritc head. some or all of the contents of the tape may be viewed as output. We are rlow track exactly in tlre original state. we cirn trace a typical ctr. The machine goes back ittto state {q and t}tc readwrite head moves left.. the conterrts of the tape.tiln show the tape contents. while q defines the state of the corrtrol rrnit. 3. will run forevet.. Some of this rnay be considered input. It is dear from this that the maclfrrc. k + 1 o' " ' is the instantaneou$ description of a machine in sta. Any configuration is completely determined by the crrrrent state of the control unit.r q 0 .with the readwrite head on the a.rc To see what happens here. allowing any number of left and rigltt moves.chconfiguration. with the readwrite head moving alternately right then left. Similarly. vrhich we will call a standard T\ring machine. there is no special output device. so thai it is now over the b. To exhibit the configurations of a Ttrring machine.te q with the tape depictetl in Figure 9. In Chapter 10.
ru indicates that the readwrite herr.1bq(raI bbqptrF bqlb qoilill bqLb.4 the positiorr of the readwrite head is over the ccll contaitting thc sytnbol immediately followirrg q. $fisrhpls S.3 correspond to the sequenceof instantaThe picrtrrrc:s hqya. llfi i:il 9. bbqsl. the blank syrubol rtray appear in the irrstantaneous desr:ription.nrllt:ft of the readwritc head. rxlrrrrally such blanks are irreleva'nt ir. such as F14.e. Subscripts.ir.ntl are not shown elxplicitly in the irrstiurtaneousdescriptiotr. the insta'nttrrrtxrus description qE. is ntacle whenever the internal sta. A move fron one conliguration to a.nks. II d (s1. arxl the reaclwrite head is on tlte c.il drawn in Figrrrc:9.Tuntuc MncHttqn 9. I'he instantnrrcous description givcs otily a finite amtlrrrt of informatitlrr to the right a. then the rnove abqlctl I abe. r:) : (q2. If the positiorr of blanks is rc:levant to the clisr:rrssion.$ .q2d. the tape t:orrtainsabcd.qRn 227 Figure 9.however. bqlbneous dt:scriptions Qoaa.1 TnE Srlrun. Thus. For examplc. The symtrol F has the u$rral neaning of irrr arbitrary nutnber of mtles. The unspet:ified part of the ttrpe is assumed to <xlrrtain all bla. R) . I ll.nothtrr will tre denoted by F..Et{frr$Sl'$ The action of thc T[ring machinc irr Figure 9.tlis on the cell to the immecliate krft of the first symbol of to and that this cell coutains a blank. are used iu argumtlnts to rlistinguish betweort several ma'crhirrcs. T .ttl is q1.3 can be representul by rl11{r0.
Qo. This situatior plays a.r q I A h A h + l is possible if and only if .lway. ) .lconfigura.chines. is possible if ancl only if b.. 1 ro r ) : ( q z . .so w() u$e a. Let M : (8.*n) = (qz.1 Q 1 0 r i 0 .kLqruht&t1+t.?F co. t t +'t t L n  & 1 " ' qTukrbAp'r1 "' A. carrnot escape.ninstantalttx)lliJ descriptiott a. proceeclingin an crrdlessloop I'rrlm which it.t. special notation for it.rious observations jrrst macle in a forrna. g I and h € Q.it is convenient to summarize the: vir.lJshorvsthe possibility that a T\rring machitrewill never hall. starting frorn the initial l:ontiguratior :t.of M.ant with n.l. fronr sorncinitia. firnclarnentirl role in the dist:ussion of 'I\rrirrg ma. n. Iixample 0. .Chopter 9 Chopter Tunltrtc MncHrlrns For frrrther discussion. .rting rfli:x2 iI to it:. .tionsleaclirrgto a halt statc will be called a computation. X. We will reprt:sent it by x:lqf. b . d (qr. fbr which 6(qi.nystring a1 .rQiIzI tltqiauz firr any qj arr(l a. A . 1 r .Tlrtlrr ir. F) be a . A move A1 "' " t l . The sequc:rrr:e couligtrof ra.1qfr!1the tnar:hirrel never halts.o) is unrlcfined. indicating tha. is ir. tr) .tion M is sa. H d ( .ici halt sta. d. A move { 1 1' " f i .chine. 'fhrirrg ma.
lt.f. t t 1 .w.n easy exercise irr Turing rnin:hiile prograrrttrting.Sr. rro matter how ma.ntl trever ha. Without this could not lirnit the relgionin which it must look for convention. the rna. ) ' F for I i z. or 6rre of two thirrgs ca.$.i E : {0. design a T[ring rnar:hinethat by the regular exPressiou00*. n1qyn2 sorne t 'l'his rlerfirritionirrdir:atesLha. 0:) ( q 0 . It savs Defirrition 9. tr) d (qo. If. r .pe. Io keep track of thc r:omputtrtiotr. If it is.chirre the inprrt.. 1}. If the input txrntains a i any*fiere. 0 .3 tells us what tnust happetr when ut e l'(M)' rrotfiing abqrrt t]re outcorrrefor any ot]rer iuput. t. T \ r r i t r g m a c h i r r c Then thc language L (M): ( * qy t r . (M)' accepts tht: lauguage dcnoted Fbr .nhappen: the rrrachinecan halt in a norilitral sta.9.l state atld halts.tthr: irrput .nksfillirrg out ther urtused portions. € f .tlwritehead positioned on the leftrrtost symhol of tu. wr: reacl each syrnbol anrl tlheck thnt it is a 0.l TuRIlrcj MacnItln 229 AccePters Mochines Lqnguoge os Turing Ttrring machintls can be vitrwed as accepttlrs in the fblklwing sense.F) accepted by M is b e a . r ..t. A string tu is written orr the ta. Any string for whit:h not halt is by clefinitiott trot in 1.tcr M does it can errter an infinite loop a.fitg6l. The reason for excluding trlanks fiom t]re iuput rrow becomesqlcar: it assurosus that a.E. bra. after a sequellce of tnovels.pt:.d.).cketecl hlanks ou the riglrt and left.1 THn Sr^r'lDnn. we contimre by movirrg right. arrclwe halt itt a tronlirral state. : (q1. As trir. . qptt. the Ttrring rnachine entets a fina. we terrrrinate anrl irccept the string.) n. tlten tl is ctlrrsideredto be accepted. This is a. starting nt the lcft end of theri1put.nsition functiotr we carr take d ( s .#. The rnachine is started in the irritial state q1 with the retr.qo.llthe itrput is rcstricted to a welldefined by region of thc tape. te strirrg is not irr r (0i]..ruis written on the tape with blalks on eitfier sicle.'€ X+ . with bla.:.wo interrral sta'tesQ: {qo'{t} and one: final stirtc F: {qr} are sufticicrrt. If we reaclr a blank withtlut eucorrntering alything hrrt 0.ny hlrrtrks it str. I t.f. W]reu u is trol itr L(M).it coukl ntlver be strrtr that thelrt:was not sorntl nonbla. Let M : (Q. fi) .nkirrput somcwltere else orr the ta.ffit*.
d ( q r . say z. starting at the Ieftmost a. d ( q r . . to as the next exarnrrlcu illustratc. replrr. indicating that an a has beru successfullypaired with a b.r7.l state if utarted in stnte {s on a blank.t a conrplete solution for which Q: {qo. then move to thc leftmost b and replrrce it with .qt1qz1qBtq4} 1 F:{qn}' g:{a. working orrt the cletails.'s or b's remain. the rnachine will halt in the nonlinal state q()' $ince d (. but for ttxlhnical rea^s()ns ernpty string is not includecl the irr Definition 9. y . We then Iet tlxl readwritc head travcl right to fincl the leftrnost b. r :y . The set . the head will move to ther right. a) : (qr. b . ) : ( q t . say y. If after some time no c. rcpla. since illrring mac]rincs have a primitive instrrrr:tion set.chineaJsohalts in a Iina. E } . Note thirt the 'rhrirrg ma.:n cumbersome on a Turing machine. still. T}avelirrg back and forth this way' we ma. R ) . we go left again to the leftrnost a. i ( q o . thc:n the strirrg must be irr .tch each .rh":n}1}. a. and so on. Intuitively. T The recogrrition of morr: complicated languagesis more difficult.nsitions can be broken into several parts.nr.ceit with a. f t ) . the machine crrtersa state qz. we solve the problern in the following fashion. we could interllret this as ilt:ceptance of A.i Exorrrple For E : {a. The tra. then causcrJ the readwrite heatl to travel right to the first b. and thc concept is ea^sy understand. when the y is written. .r . which in turrr is checked off by replacing it with another symbol. t]re computations that we carl pregram ea"sily in a higlxlr level lailguaEle ere oftr.nytime a 1 is read.1) is undefined. If at a.b}.r.  : { a . b}.230 Chopterg Trrurrc MACHTNEs As long irs a 0 appears rrnder the re^dwrite head. we arrive a.u . design a Thring machine that ar:r:eprs 7:{a. q. it is possible. b ): ( q z .y ) : ( g r . . a d (gr. ft) .3. L ) . After that. we check it off by replacing it with some symhol.L.with a correspondirrg b. rerplacestlrc leftmost a with an lr.cingit with a y.n r.
tclted with a single b. the rewriting continues this way. d ( q z . . E) . q o e . When looking for the Ieftmost a.a. It will halt becauseno transition is specified for this case. The particular input aabb gives the following successiveinstantaneous descriptions araabbl rqlabbl raqlbbl nq2ayb tsq2raybl rqsaybl nrtnyb I myqlbl rnq2yyl ng2ryy I r:xqoyu rryqy I myqsJ I I rnyyZqal. a . so that a single a has been mtr. d ( s z . E) . Other input not in the language will also lead to a nonfinal halting state (see Exercise 3 at the end of this section). and returns control to the initial state. d (Sr. wittr rL 7 ffit the machine will trventually encounter a blank in state q1. If we input a string not in the language.To termirrate.a ) : ( q z . repositions thqt readwrite head over the leftmost a. b i * q o o . the readwrite head travels left with the machine in state q2. a y h" ' b . ' ' a h h ' ' 't i * * q s " ' ( r A Y' b . a .. and b. This can be done by 6 ( q o . a final check is made to see if all a's and b's have been replaced (to detect input where an a follows a b). a ."a . ' and so on. stop ping only when there are no more o'$ to be erased. After two passes' we will have completed the partial cornputation " q o a o .) R 5 (qz. instead of finding an a it will find a 37. After one pass tlrrough this part of the computation.L ) . R. ) .!) : (ga. indicating that the matching processis being carried out properly.1 Tsn SrannaRn TuntNc MaoHItIp 231 The next set of transitions reverses the direction utttil an z is encountered.n) : (qo. if we give the machine a string a'b*.g ) : ( q z .L ) . ready to dcal with the next c.a): (q:t. When the ir4lut is a string a'b'. When an r is encountered. r We are now back in thc initial state q0. a )= ( Q : t . the computation will halt in a nonfinal state.tr.. the machine will have carried out the paftial computation . r t b b. ' .9. For examplc. . the direction is reversed to get the a' But now. 6 (qz.
t}tey r:an he acceptedby I'uring rnachineswith very similar structufesI Otte ctlnt:hrsion we can draw frorn this example is that a Thring machine can recognize sorrre langrrages that are not contrlxtfree. a.spect more closely. provided that QywI L. The inprrt for a computation will be all the nonblank symbols on the tape at the initial time. b. the output will be whatever is then on the tape. they provide us with a simple abstract model for digital comlrrrter$in general. uo the string aabb is accepted. a"s well arwith some not in L.t Tirring machines are rn()re powerful than pushdown arrtomata. Thus. I i Design a T\rrirrg macrhinethat accepts 7.:{a"bncn:n > 1}. If we want to rnodel comprrters using T\rring rrrachines.lly a simple exterrsion of the previous exarnple. but straightforward exercise. for sornc finrr.r. Notice that even though {o"bn} is a txrntextfree language and {anb"c"} is not.r Qyfr. respectively.mple9. We leave it as a somewhat lengthy. We matclr each a. But its we will shortly see.we have to look at this er.7 are easily carrierl over to this case. we check that all original symbols have been rewritten. u. and c by replacing them in order lty r. we can view a Tirring machine tra.lstate q7.nsducer M as arr implementation of a furrction / defined by fr=f(w). accepters are quite adequate. a first indication th. You are urged to traccl this program with several more strirrgs in L. . The ideas userl to Exrr. At the end. it acts as a transducer. Since the primary purpose of a conrputrlr is to transform input into orrtput. Although conceptua.MACHrFres At this point the T\rring machine halts in a final state. T\rring machines ate tttlt only interesting as language accepters.282 Chopter 9 Tunrr'rc. At the conclusiorrof the computation. Turing Mochines Tronsducers os We have had little reason so far to study transducersl in language theory. writing the actual progralrr is tedious.
(s)i qp (*+ s) 0. f . Given two pgsitive integers l' and y. eyeF. such as addition and arithmetic comparison. We will assume that ro (r) and tu (37)are on the tape in unary rrotation' separated by a single 0."nted by ur (r) e {1}+. or just A function / with clomain D is uaid to be Turingcomputable tnachine M : (Q. all the common mitthematical firrrctions. qo. no matter how cofirplicated. srrch that (r)l lur : z.ItoTunINc M. we first have to choclsesome convention for represerrting positive integers. for all z.!l THn SrAFtrrA.1. so that the addition amounts to nothing more than the coalescing of the two . are T\rringcomputable.*pr". As we will shortly claim. we thcrefore want to design a Ttrring machine for performing the compltation qour 0t.rt tty looking at some simple operations. For sirnplicity. (r) where q1 is a final state.' € l). F) computable if there exists some Ttuirrg such that QowluWf@t).tcsItqn 233 inn. .E. with the readwrite head on the leftmost symbol of trr(z).Plil.t. constructing a prograrn for this is relatively simple' All we need to rlo is to move the separating 0 to the right end of u (g). ru (r + U) will be on the tape followed by a single 0. d.i'l$fimmir. we rnust also cler:idehow r and 3r are placed orr the tape irritially and how their sum is to appear at thc end of the ctlrttputation. We sta. and the reaclwrite head will be positioned at the left end of the result. we will use unary notatiol in which aly positivc irrteger " i. design a T\rri1g machirre that computes nlA.$. After the computation.
one that plays a part in the synthesis of more complicatecl instructions. with Q : { s o . iB) This can be seen from the sequen(re instantaneous descriptions for adding of 1 1 1t o 1 1 . L ) . 5( q z . a fact that is remembered by putting the machirrc into state q1. F : {qn}. although cumbersorrrc for practical computations.1 ) : k t t .j.234 Chopter 9 'Itltrrruc MncnrNns strirrgs. ) . These can also be done easily on a Ttuing machirre. we construct M : (Q. d (Ss.1 ) : ( r l l .?0. other basic operations are copying strings and simple comparisons. L ) . tr) tr. q n 1 1 1 0 1F 1 q 6 1 1 0 1l 1 1 l q s 1 0 1 1 1 1 1 q 0 0 1 1 1 * F F 1 1 1 l q 1 1 F 1 1 1 1 1 q 1F 1 1 1 1 1 1 9 1 t r 1 F 1 1 1 1 1 q 2F 1 1 1 1 q 3 1 0 1 i qstrl11110 qa111110. . [ ) . .i (.chine that copiesstrings of I's. F). is very convenierrt f'or programrrring T\rring machines. such as binary or decinnl. R d ( q r .0.. F utrary notation. d. 1 ): ( r 1 r . 1 . Exotnple fl"10 Designa Thring ma.1 . qs. 1. ft) . To achieve this. for any ?u e {1}* .t r ) : ( q z .1. : (qr. E. ft) . I Adding numbers is one of the furrdamental operations of any comprrter. : (s+. ft) .i q. The resulting programs ara much shorter and simpler than if we had used another representation. l. is needed to remove this at the end of the r:omputation. Note that in moving the 0 right we temporrrrily create an extra l. 1) : (r1:1.1) : (qo. More preciselv.q t . Z .1ww.l (s. The transition d (qr. q z t q J ) q 4 } . . 0 . find a machinethat perftlrms the corrrputation tyl'.X.. 0) d ( S r . d ( q r .
if r j 91. (y) . (E) t qrut(r) 0.teqn iI n { g.1r.w (r) Oru . . ryprr' + if n Z A. . Clorrir.t Iirst.r7 two positive integers represented in utrary notation. 3 . R. (y) qn. ) : ( r 1 r1 .nd that Iirral state q.rr (r)0.et r a.1 THn Sremoann TuRrr'rcMrc. so let us tra.1 . F T trt: I. . z ): ( q 2 . 1 . Rtllllaclc:c:very 1 lry an r. : (q3. f t ) . struct a TLrring urachine tha. a. we irnplemcnt thc fblklwirtg itttuitive process: 1 . l ( q r .c1lr:irt Stcps 2 and 3 until there are no more tr's. 4. Firrd thc riglrtrnost r and replace it with l. ft) .Hrrue 235 To solve the problem. Tlavel to l. . if r < A.l (. r. More specilically. A T\rrirrg rnachine version of this is d (qu. t r ) . will lralt in a nonlinal sta.1 ) : ( q 1 . . : (gr. Il) . t r ) . Tlrc c:orrrputation perfbrmeldin this case is qsll F rrgeI F tru(en I rqln F r l g 2 n F r q 1 1 1F q 1 r l 1 F 1q211 11s21 111q2tr F F F 119i11 lq111F F s 11 1 1 1F q 1 t r 1 1 1 1 q 3 1 1 1 1 .5(Sr. the mat:hint: is to perform the computation (z)0.l) = ({0.D. n) !. n d ( S z .ce the pr()grailr with Lhe sirnple string 11.he right end of the current nonblank regiorr arrd create a 1 there. )( S r . tr) where q3 is the only final stirtc. .t will halt irr ar. Tlfs trtay be a little hard to see a. ft) . r) . 1) : (gz.9. (E)  q.rrd. d (so.1.
b}.l corrfiguration.ve heerr replaced.te machine'l ffi 3. together with a.\rring rnachitre irt Example 9. deprlrrdirrgon the outcome of an arithmetic operation.Thring machine carr bt: progralrurlc(l to tra.m fbr this is straightforward and is left as arr exercisc. At the end of the matching.t the 'I. whrlrc of alternate instrut:tion strea. Irr the second case. we encounter the blank rrt the right of the working space.. Deterrrrirre wha.. 4. The cornplt:tc progra..ke decisions based otr aritlrrntltic cnmparisons.lllnn. we will have on the taoe eithc:r tyr. Tltis t:xample ma.in Example 9.236 Chopter 9 Tunrug MecnrNns To solve this problem.n inititl. This kind of simplc dtxlision is comrnon in the rrrachirxrlangua. t ** L.sa.t a.. deperrdirrg whether u > E or A > fi. we still find a 1 on the right when all I's on the lcft her. Write a Turirrg rnachine simulator in sorne higherlevel programming larrgrrage.7 does when preserrted witlr the inputs aba arcJ.ge computers.rn rn... Is it possi}rleto do this with a twogta. ?. We use this to get into thc other state qr.21ltr. Design a Turing rnachirre with no morc than three states that acccpts the language L(a(al b)). and shoulcl produce ar^s outprrt thc result of the rrrmprrtatiorr.. Insteacl of rnatr:hing tz's and b's.7 goes into at irrfinite loop? .7 with sott..e minor rnodiflcations.. In thc first case. we match each l on the left of the dividing 0 with tlrr: I on the right.kesthe irnporlant point tha.when we attempt to orr match arrotlrr:r 1. Ihis can bc usecla. Such a simulator should accept as input the tlescription of any Ttrring tttacltirre.m]m.msare entered. Assrrrnethat X: {a. signal to enter the state gr. we carr rrrrethe idea. Is thcrc any input for which the Thring trachirre irr Example 9. aaabbbb.
qz. Design to corlpute the following functions for . . l . d . r r r> 1 } {flitbzn:n> I} ( g ) f l l:l { a ' " I ) " t L " b " : r r . (*) r:Z(obab) ffi (b) f : {rr._t. What happens it Exarnple !1. b) : (sr.*.a) : (qr . b) d (qr ..if ru : e. then check your answers by tracing several test exarrrples..cceptsthe ianguage L:{tntr. 'a): x '!.rc MaclHrlrn 237 5.t:rue{a.zrrj where c g f 'I\rring machines 11. d (r70. 'r > a x<A .b) : (sr. 0. the X.. (f) I...E. } 0 } l (h) I: For each problerrr. b. .. R) .{. b.: lrrrl is even} ffi (") .r.a^cetu+t. { a n b " " s " + ' n .R) .ire lunction . Construct l\ring machines that will accept the fbllowing lirnguagestxr {o.nrr TuRrr.. Il) .ir and 3r positive irrtegers represerrtetl irr rrnary. b}.. (a) /(u):3a (u) / (t.b}. ft) . 10. 'I\lring machine Sper:ifically.a) : (qs.tez. Design a Turing machinc that fincls thc micldle of a string of cvcn lcngth. write out d irr rrrrnplete detail.b}+}.i S'rAlrr^.: o {tu: ltol is a rnultiple f 3} (. } 0 .b.ins any syrnbol other than 1? 7. n) : (gs.qt.e. {ffi slrorrltlprorlrrc:e fi:ataz.I THr..b . . 1}+. .a. .i (Sr. : ({qo.i (qz.9.u.: n . ! } .C) . What language is accepted by the rnachine M { a .*e. L Construct a Ttrring uachinc to compute t.I0 if the string u conta. "( u ) : n 6 ( t a ) } (e) /. whereur € {0.qt}. : (q2. Design a T\rring machine that a.e.nf f m.t) . 6.oz'. a.i (qz../ (r) : . 8..1) I : {e'6"''n}I.} : { u : n . Q o . { q s } ) w i t h ii (qo. with o1 € X.
To demonstrate how'I\rring machines can be cornbined.10 does in fact carry out the indicated t:ornptrtatiorr. ffi 1 3 . strt:h primitive operations rrre the brriklirrg blot:ks for mclrc c:orrtplcxirrstructions.? excludes the empty string frorn arry language ac:ceptedby a Ttrrirrg rrrachine. we follow a practice common in prograrnming. 1. u ): 2 n . Write out a complcte solution for Example 9. What happens when this machine is started with 1lt) on its tape? 1 5 . Supposc that in Exarnple 9. Sketch how Exarnple 9. Modify the definition so that languages that contain A mav be accepted. if t is evert r r f f 1 $ (rdd rtl (e) / ( t ) : r m o d 5 (f) the /(*) : Lfrl. n} that. ffi C o m b i n i n g u r i n gM o c h i n e s o r T f Complicoted osks T We have shown explicitly how some important operations fbund in all computers can be done otr a Thrirrg rna.ctua.rntil the program is in the a. Design a T\rring machine with f : {0.9 trlrltl be solvetl if r and g/ were represented iu decirnal. block diagrams or pserrdocodeare the two alpproir. Definition 9.tiorrs carr also bc put together on a Thring machine.chine. 1 9 . Is it generally true that for any Thring rnachine. We can describe T\rring ma.10 goes through when presented with the it4lut 111. Give the sequence of instantaneous descriptions that the Turirrg rnachirre in Example 9. la. lct us seer how thcstr basit: opura. will halt if and only if its tape has a (l sornewhere on it.11.'s will ust: wc .Since.9 we had decided to represent r and gr in birrary. high level.chines$evcral wirys irt ir. You may have noticed that all the examples in this scction had only one final state. whereLfJ denotes cqualto f.r:hr.l 3 a ( d ) f (") : E. there exists another one with only one final state that acccpts the same language? ffi 2 0 . when startcd on any cell containing a blank or a 1.238 Chopier 9 Tunnrc MAcHTNEs (") f ( n . L4. in digital complrters. Cornplete all the details irt Exatnple 9. We start with a highlevel description. 1 7 .nguage then refine it successivelyr. Give convincing arguments that the T\rring mar:hirreirr Exarnple 9.l with which we are working. Write a T[rrirrg rnac]rirre progranr fbr doing the indicated cornputation in this representation. 1 8 . 1"6. largest integer less tltarr or t2.8.
ow(z)Oto (g). Before we accept this highlevel view. we cornbine the machines in Exarnples9. If not. arr erasirrgprogram is started that changesevery 1 to a blank.11.9 and 9. asrrume that r and E are positive integers in unary representation. For the adder. to determine whether or not r> y.mple 9. Nevertheless. wc muut justiSr it. If n < 31.11. but whose interior details are not slrown.11.IcArnn Tasxs 239 Figure 9. The computations to be done by C are qa. we u$e the idea in Example 9. qc. if x < s. The value zero will be representedby 0. The diagram shows that we first use a comparing nrachirre. The cornputation of / (r. we encapsule computatiorrs in boxes whose function is described. we will often use such highlevel. (E)  qa. the cornparer serndsrr. As a first example. In subsequent discussion$.ow(r) 0.. If so. it can be done in a straightforward way. By using such boxes. It is certainly quicker and clearer than the corresponding extensive set of d's.y):nt'!t .start signal to the adder. suggested in Example 9. Design a Turing machirre that cxrmputes the function f(n. . For the sake of discussiorr.like that in Exa. using a Ttrring machine having states irrdexed with C.gur \ut (y) I nop*(r)Oru(s) . which therr computes n*y. we corr$tnrct a Ttrring machine having states indexed with E. The program for the cornparer d is written a. (r) arrd + if r'P a. In a block diagram.2 Cor"lerNlr'rc TURIr'tcMacnrNns pon.5 J'@. ifnlt!. gr)can be visualized at a high level hy means of the diagrarn in Figure 9. What. blackdiagram representations of Thrirrg rnachines. For the eraser E.t) rnost frt:qrrently in subsequent discussions. we implicitly claim thai they r:nn actually be constructed.9. with the rest of the tapc blank. for example is meant by saying that the conrparer serrds a start signal to the adder? There is nothing irr Definition 9.1 that offers that possibilitv.5. with states indexed with .9.0.4. Corr.rpl.
we see that C starts eithcr A or . for all c € f.pc content or moving the readwrite head. for all c e l. : (qp. f o r a l l q a€ Q a n d a l l b € f .ft) d (gio.0 and qp. then regardless of its current state. which is a singlestatement shorthand for a sequence ()f lov/er level statements.g as t}re initial states of . While this description is not usahle on the cornputer. If the Thring maclflle reads an a.b) : (qftg.5. we assume that we can translate it into the appropriate language when needed. Consider the macroinstruction i/ a then Qi else q*.1.a. A. o t( n ) 0 . We then use the macroinstructiorr irr a progralrr with the assumption tha. Thc states qio and {6s are new states. introduced to take care of complications arising from the fact that in a standard T\rrirrg machine the readwrite . respectively. To implement this macroinstnrction requires several relatively obviorrs 'f\ring steps of a machine.c.t o ] . In computer programming. highJevel view of T[uing1rnac]rirresis otre itrvolving pseudocode. u The result is a single Thring machine that comhines tlrc actiotr of C. 6(qt.ne the macroinstnrction in terms of tlte lower level language. y r u ( r * u ) 0 .b. r o .t:rtr instruction. L) d (qon. it is to go into state {i without changing the tir. pseudocodeis a way of outlining a computation usirrg descriptive phrases whose meaning we claim to understand.rwcMAcHrNos If we take {. o w. This idea is very useful irr Ttrring machine programming. n ) O r ( g ) i q o .4 and E.c) : (qi. E a^s T Anotlrt:r useful. The computations performed by the adder will bt: ( u t t .a): (qio. If tha syrnbol read is not an a. and indicated in Figure 9.8. and that of the erraserE will be q n . We first defi. s .c. with the following interpretation.t the relevant lowlevel code is substituted for each occurrence of the ma. the machine is to go into state 96 without changitrg anything.240 Chopter 9 Tun. ( y ) i q r .L) c) for all qt€Q.R) d(qr. One simple kind of pseudocode is exentplified by the idea of a macroinstmction.
we want to change the state.rldreturn to this state. so no irrterfertlrr(lc (jirn or:<:rrr. Wlxlrr B is finished.When B is linished.9. The workspaur ftrr B is seprlrtlte fiom 7 and from the workspace for A. I Goirrg a stt:p firrthtlr.ssinforrnation from A to B ancl vice versa.. At other times.with the readwrite head (which may have rnoved during B's operatiorr) irr its originrrl pla. After transfer.bleto pa.ch occurrerrcci wlrcreiul er. .6 Region separator Workspace forr{ Workspace for. For example. let rrs outline briefly how a 'I\rring rnachine carr bc used as a subprograrrr that t:arr bt: irrvoked repeatedly by another Thring machine. rr.g. milcroinstruction is replaced by actual corlc a. it will return relevant results to region T.debefore entering the drlsirul statc qi or q4. we (:an replace macroinstructions with sutrprtr grarIIS. Ihis requires a new featurc: ther ability to store infornration on tlrt: r:a. Tliis indicates that a lefb move mrrst be ma. Subprograms are fundarnerrtal to highlcvcl prograrrrrning languages. w(l rmurt be ir. but put the machine into a stirtc (Ii1 $ qko.t ctr. Figtue 9.dwhere it is.B .ppensin a rerr.ce. To rnake this plrrusible. but leave Lhereadwritt: hcir. subprogram is a single piece of code that is irrvokcxl repcirtt:dly whenever needed. itt w}tich c:itscc:rlrrtrol shor. A may call B frorn state {i. we carr divide the tape irrto several regiotrsas showrr in Figurtl 9.rerlrrNc Tunrr'rc MAcrrrNrils FoR CoMpllcAruu T.' subprogram.sxs 241 head changesposition in each rnove. where A will expect to firxl it.. Irr this wiry.6. the argunents for B) on thc: tirpr: in some regicln 7.lrxrmputer when a subprograrn is called. the two programs can interact in the required fashion. we would like to resurne prograrrr A itt sta"teq.4's current state.rl then pa*sses control to B by making a trarrsitiorr to the start sta. but they can also be used with TLrring rnac:hirr:s.. say rnachine A in state q.2 Corr. Before A calls B..croinstruction. Note that this is very sittrilar to what actuirlly htr. . Irr the ma. Norrnillly. B will use 7 to find its input. We let the head mrrve right.q.teof B.4 are not afftx:ted by the execrrtion of /1. invokes rrtircltint: B. aucl to assurethat the tetttporarily suspcndcd comprrtations of . To solve this. it writes the irrfrrrurirtionneededby B (e. To solve the control transfer problern. be able to r(l(:r{lntcA'u cxrnfigurationwhen it recovers control from B.llingprogram's configuration so the configuratiorr carr br: rtx:rcilterd on returtr frorn tlN.
Repeat the following steps until r corrtairrs tto more I's. Replace the leftmost 0 by 0y. brrt it should be plausible at this point that Tirring machines can do some quite complicated things. Our examples were not general a"rrd detailed enough for us to claim that we have proved anything. 1.sindicated in Figure 9. Replace all a's with I's. Let u. can be combined irr rnatry ways to make them quite powerful. whereby the string g is added the appropriate number of times to the partiallv cornputed product' The following pseudocodeshows the main steps of tlte process. while rather primitive in principle. Although this pseudocode is sketchy. 2.n actual Ttrring machine program. A rrrultiplication machine can be constructed bv combining the ideas that the initial and we encourttered in adding and copying.'Ja$rJlrrne The process of mulfinal tape contetrts are to be a.Chopter 9 Tururuc MnctIIwns Figure 9. I In spite of the descriptive nature of these examples' it is not too farfetched to conjecture that Ttrring machines. Write out the complete solution to Example 9. Find a 1 in r and replace it with atrother symbol a. tiplicatiorr can then be visualized as a repeated copying of the rnultiplicand g for each 1 in the multiplier r. the idea is simple enough that there should be no doubt that it can be dorre.7 v We can now program T\uirtg rrrachinesin pseudocode.7. Design a Ttrring machine that multiplies two positive integers irr unary notation.14. provided that we know (in theory at least) how to translate this pseudocode into 8. 1. .
For each problern.r'r:hits tape to the right of the current position for the first occurrcnce of the syrnbol a. then sketch a method for addirrg and subtracting such numbcrs. w2. ffi for all positive integers z. and i:0 othcrwise. Establish a conventiorr for representing positive an{ legative integers in unary rrotation.rc Tunnrc Mecumns FoR. W . Give an implementation of the macroinstruction searchright (.wz. use a bhck diagram to sketch the implementation of a function / defined for all w1. which indicates that the rnachine is to sea. (e) /(n) : p'!. > 1} (e) f : {an : n. rrketch the construction of a subtracter for t:omputing a .. define a set . "highlevel" rlescription 5. Therr use them for the solution. b}. or rnultipliers. where i is such that ltui j = max(lturl.qi). is a prime nurrrtrer} 6. (d) /(n):. (b) /(n):r'1'.f appropriate macroinstructions that you feel are reasonably easy to implement.ruturt} {*r.ws) : i. the machine is to go into state ql. comparers. copiers. (u) Z: (b) t: {.r. 4 . sutrtracters. draw block diagrams for Thring nrarhines that computc the functions ( a )/ ( " ) : n ( n a 1 ) ..Provide a for T\rring rnachines that ar:cept the forIowing la'guages orr {a. otherwise it is to go into state 4i. suggest a method frrr represerrting rational numbers on a Ttrring rrrar:hine.luzf .9.qt. If an a is encountered befrrre a blank.: n?.3y. with your conventiorr. using adders.11.2 CortelNlr. 8. fu3l) if no two tu have the sarrre length. Sketch the constructiotr of a Ttrrirrg machine that can pcrform the adclition antl multiplication of positivc intcgers x ard g given in the rrsual decimal notation.rrrrl} ffi (c) The complement of the language in part (a) (d).. : {u"bn' : m. CoMplrcArun Tasxs Z4B 2 . 'rrrrg {1}+ by f (wr. 3 .a. t wr # wz: lurl : l. 7. (c)/(rz):2".
lro puslrdown au[omaton exists.244 Chopter 9 TunIwc MAcHINEs Use the macroinstruction in the previous exercise to design a Ttrring rnacltirte the language L(abab"a).im thilt T\rrirrg lllacllines can pcrfrrrrrt ttot only the sirnple operations fbr which wc have provided explicit ils prograllls. how sever.nt progrirm$ cxplicitly. wt: skctchcd the construction of a T[ring mat:hinc fbr a larrguage which is uot coutextfree ancl for which.tionu cilrr lxl t:orntrirredto solve nore cotnplex T\rring rnttclltirtescan lre cotnposed. In Example 9.nslattt a block diagram or pseudocode into the t:orrtlsporrdirtg error prone' Tlrring rnachirreprogram.ce the rightmost rrorrblaflk syrnbol byab. But doing so is unpleasant and diuouglrt to be avoided if possible. we sholld show the releva.ta. Thc discussionalso ilIustrates how prirnitive opera.9.whikr it takes very little imaginatiott or ingenr.cha rea. stlt: Ext:rcise 2 at the etrd of pusltclowtr aul. irntl irdds Iittle to our understa.sona.n section.10.ccepts 1 0 . Since very complex operations ca. on E: {a.I gram can act a. Use the rlacroinstructiotr searchright irr Exercise I to creatc a Thring machine program that replaces the syrrrbol immediatcly to the left of the leftrrxrst a by a blank. and how olle proproblems. from the examplesin the prtrvitlrs We can rlrirw sornrlsirnple conclusions 'furing macllines appeal to be more pqwtrrful thir. the best we t:tlrnprotnise' To see how we might achieve can do is to rea. or proof for a nontrivial probletn is quite tedious. actually doing it is timtl t:orr$urnirrg.rrcl a way of carrying out a reasonably rigorous rlist:rrssiorrof T\rring machines without havilg to write lengthv. ailtl 9.r. T\rriug nachitres are equal itr power lo a typical digital computer? Htlw trtiulrl wtl .nding. Suppose we were to make tlle conjecture that. but also illrrutrat()sa llegative aspect of working with such lowlevel a. repla. this section).n he brrilt this wiry. Iixamples 9. lowltlvtll code.8. Ttrc first is that . soltrtitlrr.utoma.b} that a. subprogrrur filr arxlthrlr.o[tata (for a. in some $enst:.'rliglrt srrspect that a'ftrring tnachine begins to ilpproar:h ir. r:orrscqucrrtly. Wt: rrow face a dilenuna: we want to t:la. but nrore complex procert$es wtlll. Tht: irrstnrction set of a Turing mais <:hirrt: so restrictecl that any argument. w(. Thesis Turing's 1flffiffitr Thc prtrcxling discussiou not only shown how a T\rrirrg rnachitre can be constnrr:ted from sinrpler parts. attcl tnake sotne simple compilrisons.rity to tra. a. 9. There is unfortutlately no completely satisfactqry way qf gt:ttirrg out of the predicament.11 s]row thal Tlrrilg machines can do sorne simple arithmetic optlrittittns.wo ttrrrr to a sornewhat. Somehow.we would like to firrd tracting.tyllir:irl t:ornputcr itr power. perform string rrarripulations.sa.:omment on this.bkl philosophical issue' such a comprclmise. clescribableby block diagrarrls or pseudocode. If the irrput cofltains no a. To defilrrd such claitns against challenge.
ation is tha.nd r:onceivablv tright do in the future) with computers? An unequivocill "yes" is rxrt possible. we could take a seqrrerrcc 6f increasingly more diflicult problerns arrd show how they are solvecl by sorne Tlrring machine. Alternative moclels have beel proposed for nrechanical ctrrnputation.ndleave us no further aheacl than before. Thin iu rr.rrexa.bleto suggest a problern. Is it farreaching enough to covcr' cvcrythirrg wr) now do (a. This would rrndoubtedly tax our patience. Ttrring and others in the midlg30's to the celebrated coqiecture called the T\rring thesis. Arguments of this type lul A. but frrr which we can show thirt rxr T\rrirrg ma.. If ue take this attitude attd regat'd the TLrlirrg thcsis sitrply as ir dclinitiotr.lgorithm. It is not something that can be proved. Ihe 'rhring thesis is rrrorc protrxrrlyvierweda. Tlxr dirlir:rrlty lies in the fact thir. But rlo one has yct lrr:crr f'rrr able to prorluctl a r:ountt.t r:. we might also takc the ma.lcornputer" arrrl tha. it would not lead to a proof.sa.sis rejecting the hypothesis.. .n a. wrr r'ight try lo find sorne proctxhrru filr which we ca. 3 . fact that all such tries have beerr the utrsuccessfulrnust btt takttn as circunrstantial evidence that it cannot be done. we would hrrvr:trl define precisely the term .t any cornputation lhat can be carricrl orrt lly nxrchanical mea. Some rlrgrrments for a. solvable by what we yet irrtuitivtly rxrnsidcr ar.r:hthe problem from the other side.sweeping sta. Still.n be done or any existing digital computer can also be dotre by a T\rrirrg rnir:hirrt:. Tb do so. If this were possible. This hypoihetis states tha. Arrythi'g tha.s been a. while (lvery .'ha.9. we would hirvt: ir lrtr.t Ttrring machines are in principle as powerfirl a"sarry corrrputer.n$ can be perfbrtrr:d by some T\rring rna.mple.3 TuRrNc'sTnnsrs 245 def'endor refute such a hypotlxrsis'l To def'end it.tement. Every irrrlic. 2 . we crrrr *lsr approa. brrt none of them are tnor:epowerful than the T\rring rnachine model. lrToorre.$s this directiott would strcngthcn our convictjon of the truth irr rlf tIrc hypothesis.t we don't know exactly what is meant by "a typictr.mechanicalrneans.so it is important to keep in rnirrrl what Thrirrg's thesis is.chinelanguage instruction set of a specrific computer and design a Tlrring machine that can perform all the instrr.cceptingthe Ttrring thesis as the definition of a rnechanical comDrrtation' 1 ." This would requirc sorne clthcr irllstrar:t model tr.chinecan exist. but the evidence in its firvor is vr:ry strrlrrg.l cligita. for which a Thring rnachine program canrtot llr: writtcrr. but it ought to be possible in prirrciplc if orrr hypothesis is correct.rctionsin the set.chine.5llcce.nwrite a computer prograrrr. we raise the clucstiorras tti whcther this definition is sufficiently broad.definition of what constitutes a mechanical cornputirtiorr: ir cxlrnllrtatiorr is mechanical if and only if it can be performed by sorne Turirrg rnachirrc.t we have no lneans for rnaking a precistl rlcfirrition. M.
n algorithm . On the other hancl. Slch litws t:arrrrot be proved to be true. . But viewitrg Turing's tllesis simply a"san arbitrary defirritigrr rlis$gs an impottant poinL. Iclentifying an algorithm with a. To avoid prosprx:t$. thc likelihood of this happenilg seems to be very srnnll. There is always the possibility that sorrreorrewill come up with a. tltey are plausible models tha. Hirvirrg accepted T\rritrg's thesis. sorne Of t)trr $trllsctlttent discussions would have to be modifled signifir:ir. we are in a positiou to give a precise rlcfirritirlrr of art algorithttt.t explain mur:h of tlrt: physical world. is based largelv on Newton'$ Ia'wsof r'cttitttr' Although we call them laws. for example. they do not htr. wr: t:ilrr irllltc:itl to T\rrirrgts tltesis atrd clain that Sur:hunpletr. Itr such atr eventu6lity. and Thring's thesis ctr.ntly.str.l result contradicts a conclusion based qn thc laws.rrt ir. we might begin to question their validity.ys tlrt) rltrrrrt: role in cxrrnputer science as do the basis laws of physics and chemistry. We accept them because the conclusions we dra. t0 trurrstrtt<:t is rro algorithm. .terl fir.chine progrirm tilIows us ttr "there exists a. although they can possihly bc invalidatecl. Irr spite of its plausibility. which given as input any d € D on its tape." However.w fiom thcrn irgrtlc with our experietrce ancl our ohserva. This is the sitrra. Thring's thesis is still itn alJsrrmptiorr. eventually halts with the correct answer / (d) e .rrythirrgw() (titrr tl6 orr arrv txrrnputer carr also be dotre otr a Tirring tnachine.vekrgical n(xx)ltitity. In solne sense) Tbring's thesis pla.UI (. Howc)vcr." or "thcr(l prove rigOrously sUch claims as tlxlllit:itly irrr algoritlrrrr for . The conclusions we draw f'rom it agrt:e with wltat we ktrow alrout real computers' and so far' all irtttrrrrpts to invalidate it have failed.ihrrc to invalidate a Iaw stretrgthens our confideuce in it.tions. rather. MacHhrns Tlx.tirlrrfclr TUringts thesis.t qodl nr .we can require thtr.1) e F. even relativelV simple problems is el vcrv lcrrgthy urrdertakirtg. repea.rse irrguulellts ale circurnstantial.246 Chopter 9 Turrtuc.nother definition that will accotrnt fttr somtl subtle situations not covered by T\rrirrg rnitchirrcs lnt wlfch still fall within the range of our intuitivc notiorr of rrrechatrical cotrtputatiotr. Ttring ma. so we have some reason for it c:onsiclcrirrg a basic law of computer scieuce. lM" Arr algorithm for a function f : D + ft is a Ttrring machine M. Classical physics. If an exPerimellta.q1 frlr all rJ € D.nnot trtl provrxl by therrr. Specifically.R on its tape.
but it obviously Ieaves us open to criticisrn.Ttring machinet' in Defirdti'rr 9.nger is more than offset by the fact that we can keep the discussion simple arrd intuitively clear. Hopcroft.3 Turumc's Tunsrs Conseqrrtlntly." This paper talks about the irleas we have introduced here and also gives some of the historical context in which the work of Turing and others was done. There are a number of enjoyable articles on Turirrg machines in the popular Iiterature. it woukl seem that we can actually clairn that a Tlrring machine is more powetful. This greatly simplifies the discussion.9. May 1984.lly. we will go t)ne step further and accept vurtrirl descriptions or block diagrams as algorithms on the a"ssumptiotr that we could write a T[rring machine prograrrr frrr them if we were challenged to do so. E. Actrrtr.entifi.lgorithmsconsiderably. A good orre is a paper in Sci. Get a copy of this article and read it. This would easethe burderr of cxhibiting a.rnguage.ge instructions for a corrrputer of your choice. 2. we stated at one point that Tlrring machines appear to he more powerful than pushdown autorrrata. titled t(T\rring Machines.swe have already done. . But this da. Skctch how the various instnrr:tions in this set could be carried out by a Ttrring machine. by J.. and that we can give cclnclise descriptions for sorne rather c{lmplex processes. we could substitute ttPascal program" for . While "Pir.. The reader who hirs any doubt of the validity of these clirims can dispel them by writing a suihrble program in some prograurrrrirrg l. What irnlrortant factor is not takerr into account in this argurnent'l W ** 3.clear verbal description" is not. a. .cAmerican. In the above discussion. Since the tape of a Turing rnachine can always be made to bchave like a stack. and we are irr rlrrnger of claiming the existence of nonexistent algorithms.5.rr.l program" is well defined. ** 1' Conrrider the set of trar:hine langua. then write a brief review of it.st.
.
It is nevertheless instructive to study rrrore complex models. if for no other teasoll than that arr explicit derrrorrstratiorr of the expected result will demonstrate the power of the T\rring machine and thereby increase our confidence in Ttrring's thesis. to other. in a sensewe will define. there are alternative definitions that could serve equally well. we exper:t that cromplicating the standard Thring rnachine try giving it a morc txrrnplex storage device will not have any cffrtct on the power of the automaton. In this chapl.er we look at several variations.O t h e rM o d e l s f o Turing oc ines M h ur definition of a standard Tirring rnachine is not the only possible one. The conclusions we can draw about the power of a lfuring machine are largJely independent of the specific structure chosen for it. can be dorre by a sta.ndard model. more complicated models. Many variations on the basic model of Derfinition 9. therefore. If wc a<:r:erpt Tirring's thcsis. 249 .1 are possible. For example?we can consider T\rring machines with more than one tape or with tapes that extend in several dimensions. Any computation that can be perforrrred orr surlh a n{}w arrarrgement vrill still fhll under the category of a rnechanical corrrputation arrd. showing that the standarcl T\_rring machine is equivalent.
orl ffiffiffi Minor Voriotions the Turing T Mochine heme irr We first consider some relatively mirxlr {:}rilnge:s Defitrition 9. grarnmable" or Finally.t different tirnt:u. we follovr the precedence established for nfa's ald dfa's and define equivalence with respect to the ability to at)rxrpt larrguages.t of one uachine executing "reprodifferent programs a. This leads to the idea of a "universal" T\tring rnilt:hirre. tigate whether these changes make a. This is unexpected. since T\rring's thesis covrlruonly rnechanical computatious and does not tlddress the clever guessing implicit in nondctt:rrnirrisrrt.ny Whenever we chlnge a dtrfinition.sses automata C1 and dr. we must carefully state what is to be nnderstood by this equivalcrrcc. These ar. of Equivolence Closses Aulomoto of Whenever we define equivalence for two automata or classes Of atrtornata.tir. We have seen tln exampltl Of this in tlte case of cleterministic and nondeterministic Iinite arrtornata. These have quite different definitions. but they a.ration firr lrrtcr cltapters' we look at linear bounded T\rring rnacltirres tltat have atr infinite tape. in prepa. but that automata. we also look at nondeterministir: T\rrirrg rnachiiles and show that they are no more powerfrl tharr dclterrlilistic ones. Atrother issue that is not irnmediately resolved by Ttrring's thesis is tha.re elqrrivakrrrtirr the serrsethat they both are iclentifled exactly with the fa. these difference$ maY rrot have atry interestiug consequences. If f'trr everv tttrtomatort M1 in C1 there .ntlsst:ntiir.ndalntlthcr'l Although there rnay be clear differences in their clefinitions. Extrapolating from of this. ilM Two automata are equivalent if they accept the same language. For the rest of this chapter.mily of rcgular larrguages.1 and invesdilltrrttntr: irr the general cotrcept.tr.re can rnake use of the tape only in a restricted way. Wlrat do we mean bY ir. we carr defirre equivaletrce or nonecluivalence for classes automata in gerleral.250 Chopter l0 O'rsun Molnls or Turut'rc MecHtlrns We will c:ortsidervariants that will be useful in srtbscqucrrt discussious.l clifferencebetweetr one cla.ssof automa.. we irrtroduce a trew type of autotnata and raise the qgestion wlrt:thcr thcse new automata are in any real sensedifferent frorn those we have alreacly encounterecl. Corrsider of two cla.
0  Aadl Therr M siurula. but tltis does not affect anything if we can tell whitlh configuraltions of M a. If the converse also holds anrl fbr every M2 in C2 tlrcre is arr M in (J1 such that L (Mr) : L (Mr).. of M may involve scvt:ral move$or ff.rrlrl have clone.rriedorrt hy M.. the readwrite heacl rnust tnove either to thc riglrt or to tlrc lefl.ccept the same language. he the sequence of instantarreous dr:scriptions of the comprrtatiorr of M.rereleva..+.t err. tha. Lt+t ico..}r qf t}rir. wlxrre d0. We sav that another autorrraton ffi r:a.10. The r:onstruction of Theorem 2. we say that rt . Irr otht:r wrlrd$.r. srr{.. &r€ instantaneous descriptiorrs. Let M btl an rrrrtomir.n determine from it exactly wlnt computations .t M ancl fr a.d1.Hr.Hluo THnlrn 25L is an autorrraton M2 in C2 srrch that L (Mt) : L (Artz) .nsimulate a corrrputrr. thcrr uratters catr bc arrarrgrld so tha.un mimic the computatiorr of M irr the of following rrrarutlr.tion.'s alndnfa's. wc siry that C1 and Cz are equivalerrt. VARTATTONS . sirnrrlate M.o.d1. Note that t.ch them is associa.ss. rt should be clear that if M can sirnulate M.t oN TuRrrqc Mec. given the corresponding starting configura.a. we often use the inrportant ter:hnique of simulation.testhis computation if it carries out a r:orrrllutir. the intermecliateconfigurations in 4 io 4*r rnay not corresporrd to arry r:onfigurrrtiong! M. ancl the two automata arc rxrivirlent. we say that Cz is at lur^st as powerful as C1.ny ways to establish thc equivalence of automata.4 woukl havr: done. if we know the cotnputatiorr r.tion analogous to that of M.the simulation is proper. sta.fr what M w.ndard rhring machine. Turing Mqchines with q StoyOption Itr our clcfirrition of er.ton. we ca. we show that frlr t:vury mar:hine in one cla. Sometimesit is convenient to provide . To demonstrate the equivalenceof two classesof autotnata. As long as we can deterrnine from the cornputatiorr of .2 does this for clfir..nt. If M can sirnulate erely cotrprrtation of M.tion M. if ffi .there is a machine iu the second class capirtllc:of sirnrilatirrg it.\. .t is d.1 Mn'ron. For demonstrating ttrc equivalence in connection with T[rrirrgts machines. There are ma.ted with a unique configuratiorr of M.he simulatiorr of ir singk: move rJi ts*d.
) ..f..5} with the interpretation that 5 signilies rlo IIIoveIIIeItt of the readwrite head.R. a.n define a Tiuing rrtachitre with stayoption by replacing ci in De{initittn 9.F) chine with stayoption to be sirmrlated by a statrdard Ttrring machine M : ^\ //^ {O. u ): ( q i . It is reastlnallly otrvious that every cornputaLion of M llas a corresponding computation of M. tve put into E the corresponding transitions : E ( a . h . o )( i j . f . The sinulating machine can be constructed by M hv defining li as fcrllows: For ea.262 Chopier l0 O'r'ntn Montls oF TURINGMacntnns a third option. R ) .) . If S is involvtxl in tlter rnove of M.n.1 by d:Qxf8xfx{tr. fio. we put into f E(A.rlateM. esscntially identical to the move to be simulated.tr or rt) . so that M can simr.1. to have the readwrite hcad stay itr place after rewriting the cell content. For each rntlvc of M. The class of Ttrring rnachines with stayoption is eqqivtrlent to the class of standard I\rring rnachines. This option does not extcrxl the power of the automaton.. If the move of M does not irrvolve the stayoption. fbrallcel. it is obvious that arry starrtlard T\rring machine can be simulated hy one with a stayttptiorr. F ). the simllating machine perfbrms one nl()vc.nd F.b.S ) . c.^ 0 ((. lhen M will make two moves: the first rewrites the syrnbol artd rttoves the readwrite heacl right. b . L o r R ) . Thus. we ca. = (Qj . b .{o.cJ : .X.a ) : ( qj . t . Frrr cach Stransitiotr 6 ( q r .IetM : (8. be a TlrringtnaTo show thtt txtrrverse. r.chtransition d ( q r . Ieaving the tape t:ontents urtaltc:rcd. . ' f . .. the secorrtl rrrovesthe readwrite head left. \Qi . d.tr. t}te simulating machine M does \ / the following. Prooft Since a Ttrring machine with stayoption is <:ltlarly iln extension of the standard model.d.
hrrt use o11e with a ta. The reader will find it instnrctive to sketch each simulatiotr in some highcr or level la.pc that is unbortndtxl only in one dircctiotr.chuynrbol is composed of several parts. Tope with Semi'lnfinite Turing Mqchines Many authors do not consider thtr rnodel in Figlre 9. artd the forma. calltxl tracks. Befrtrcl irrtroducing other rntldels.1). . Ba.1 Track1 Track2 Tiack 3 b t Simulation is a standard techniqut: for showing the equivalence of automata. The simulations artt givetr only iu brotld be outlirre.rd.reklss cottttnotr are explclred in the exerc:iscs at the end of this section. we keep ortr disc:rrssiou rather thtr.rd Ttrrirrg rnachine. In thc picture. It is implicit in Definition 9.1 (Figure 10.10. which are sometime$ us(xl as the standartl defilition.sa tapc that has a left bourtdary (Figure 10. each crontainitrgotte memhtlr of the triplet. bgt it should not he trirld to see how they r. This can be macle morc explicit by drnwirrg att expanded versiort of Figr. t'his Tlrring nrachine is otherwist: identical to otrr statrdarcl nrodel.but we generally mahe tto attempt to dcscribe everything in a rigortltrs and detailed wtr.pe svrlbol can be a compositc of characters rtltlrt:r than just a singlt: tlrre.In our suhsequernt latiol freqrrently.nirr theoremproof forrn. otlter ther txluivaletrce with thc stattdard machint: ltas to be demonstrated. To avttid this. Some variarrts tha.rrguage in pserrdtltrrtle.chirttls descriptive.1. hrrt such a view in no way extends Definitiott 9.a.1 that each ta.rr made rigorous.we use tht: rrotiotr of sitnuabout equivalenr:c.1 MrNoR V.rtlThring machirrc M by a machirre fr with a semiinfinitr: tape. I{ere wc look at two srrctr trtodels.as shown in thc above theorem. To sirrtulale a sta.t a. we ufJetht: arrangenrent showtr in Figrrre 10.ndir.sedon this visualization. It is ttot difficult to see that this restriction does not affect the power of tlrtl tnachine.lismwt: have described makes it possible.Complete simulatiorrs with'I\ring ma. such an arrtomaton is sometirrrescalled a T[rring tttachine with multiple tracks. cxcept that no left move is permitttxl when the retrrlwrite head is at thc boundary.y.nda. we make one rcrrtark otr the sttr. to talk about the proces$precisely and prove thetlrtlrns discussiolr.1 as standa. are often crrrlllersonte. in whic:h tlte tape symbols are triplets fiom sorrtesimpler alphabet. we hervt:divided each cell of the tape into threN:parts.2).rre9.tRIarIoNs oN THt TLIRINGMncHIun THntrae 25S Figure 10. so Howcvcr. sitrce all we need to do is rnake I au a'lphir'bt:tirr which ea.3. 'fhring machitre urodels involvc a change of defirrition. We can visualize this a.
c . M is programrned so that it will use information on the upper track only a. For example. Now. The distinction carr be made by partitioning the state set of M into two parts.? Figure 10.3  Treck 1 for risht nart of standard tepe" Tiack 2 for left part of stefldfid tape The simulating rnrr. the simulatirrg at machinesees(ff.L). (b) Simulating machinc. b ) . r . special end markers ff a. The simulatirrg machine will first rnove via tlic transition : (f ( E ( A . a$$rrmethat the rntr.c. { (a) . On the upper M onli.ted gerrcratedbv is d(qr. only irrfrrrmation the rrpper to in fi trat:k is considered this point. wlrerefi € Qu. ) )r . ff) Figure 10. @ . The lower trirc:k conta. we keep the irrfrlrmation to the right of some refercrrr:e point on M's ttrpe.chineto be simulated arrd the simulatirrg machine are irr the respective corrfigrrrations shown in Figure 10. The refererrccpoint could be.ins the left part of M's tape in r{}verseorder.4 and that the move lo be simrrla.slong as M's readwrite head is to the right of the refererrr:epoint.a):(qj.use belongs Qu. say Qu and Q. and work on the lower track as M moves into thc left part of its tape.: the flrst to be usr. tlrc second to be usud on the lower one. L ) . Ref'ercncepoint /'' h a )q.'d when workirrg otr tho upper track.re put on thc left boundary of the tape to facilitrrte switching frorn one trrlck to the othcr. Bcca. for example.chine has a tape with two tracks.254 Chopter l0 Otnnn Monnls or TuHrNc Macnrruns Figure 10. the position of the readwrite head at the start of the computation.4 (a) Mat:hine to bc simulaterJ.
5 Setltretceol conligurations in simulating d ( s r .1 we disr:arded the input file fbr reasons of simplicity. claiming that this tnade no difference to the T\rring machinc concept.a ) : ( q i . and what is seen try the readwritc head. putting it into the conliguration will work on the lower track. tr. F\rrtlter the rnachineis in a state from Q. Itt such a machine.10.1 Mrr. In Definition 9. each move is governed by the intt:rilal state.6.I tlefinition of an arrtomaton in Chir. What we Figure 10. .6 Readonlv input file . what is currently re:ad from the inprrt file. but we will ltrave this as an exercise. # ) . shownirr Figure 10.c . .qn.l )0t in state ti e Qu.Now with fr € Q1.nd details of the simulatiori are straightforward' Turing Mqchine TheOffLine The genera. If wc put the iuput fitc back into the picture.5. we get what is known as an offline Thring machine. It rrcxt uses& transitiorr t ( t . We now expand on this claim. # ):) ( F i . ( # .Inuolts oN THtl TURIr'rc MnclHttrs Tsplrp 255 Figure 10. ( # .pter I containcd an input file as wcll as temporary storage. A schematic representation of a'n offline machirrg is shown in Figrrre 10.lon V. R ) . L ) . A forrrral definition of an offline Ttuing machirre is easily made.
All that needs to be rlone by the simulating machine is to copy the input from thc input file to the tape. Give a formal definition of a Ttrring rrrachine with a semiinfinite tape. the tape contents shown represent the specific configuration of Figure 10. Then it carr proceed in the same v/ay as the standarrl machine. track 4 is searched for the position of the readwritc head of M. The simulation of an offline rnachine M by a starrclarcl machine fr requircs a lengthier description. Firrally.te the position at which the input file of M is read. Starting from some starrderrdposition.ft end. and the fourth shows the position of M. we rrow know that M is to do. M scarches track 2 to lor:a. 1. In that picture. all four tracks of M's tgqe are modified to reflect the move of M. Next.s or. The first track has the input. the readwrite head of M returns to the standard position for the simulation of the next move.256 Chopter l0 Ornnn MoDFlr.6.TuRnrcMACTTTNES Figure 10.7. Then . tlNr third representsthe ta. of First. The syrnhol found in the cgrresponding cell on track 1 is remernbered by puttirrg the control unit of M i'to a state chosen for this purposo. the second marks the position at which tho input is read.chmove of M requires a number of moves of _ffi. Each of the frrrrr tracks of M plays a specific role in the simulatiorr. With the remembered input and the symbol on track !. Next. the behavior of any standard T\rring machine can bc simulated by some offline model. This irforrnation is again remembered by M with arr tlppropriate internal state. A standard machine can sirnulate the computation of an offline machirre try using the fourtrack arrangement slnwn in Figure 10. say tha le.pe of M. and with the relevant informatiorr rnarked by special end markers. The simulation of ea.s rc:rrdwritehead.7 want to do briefly is to indicate why the classof offline Thrirrg machines is equivalent to the cla^ss standard machirrert..
E. Consider a version of the standa. 6 .lso accepted by some standard machine. Give a precise definition of such an automaton and sketch a simrrlation of it by a standa. then a must be !. Suppose we make the restriction that a Tlrring machine must always write a symbol different from the one it reads. Suppose we make the requirement that a Ttrring marhine can halt only in a final state. tr or R) . For examule 6 (qt.10.a) : (qi .re made if the current tape svmbol is not one of a specified set. then a and b must be different. Show that no generality is lost by making such a restriction. L or R) . ffi i}. Consider a T\rring machine with a different decision process in which transitions a.{a. a) be defined for all pairs (g' a) with a €  and q t' F.{tr}. then sketch its simulation by a standard T\rring machine..b. Make a formal definition of such a machine. b must be in f .L or R). the distance and direction of travel being one of the arguments of d. Give a formal definition of arr offline Ttrring machine.R) . 4 . if d (Sr. ffi . on any particular mover cart either change the tape symbol or move the readwrite head. This can be achieved by the restriction that if 6 (qr.rd Ttrring machine in which transitions can depend not only on the cell directly under the readwrite head. Does this restrict the power of the Turing nrachine? 9 .q'cHIIIn 257 prove that the class of T\rring machines with semiinfinite tape is equivalent to the class of standard Turing machines. but not both. that is. Show how such a machine can simulate a standard Thring machine. s.rd T\rring machine. (a) Give a formal definition of such a machine' (b) Show that the cla^ss such machines is equivalent to the clags of of standartl T\rring machines.a) = (qj. Give convincing argunrerrts that any language arcepted by an offline T\rring machine is d.1 MINoR VlRhrIoNs THurae oN THE TuRINc M. c. Consider a model of a Ttrring machine in which earh move permits the readwrite hearl to travel more than one cell to the left or right. but also on the cells to the immediate right and left. b}) : (fi. that is. that is. for all d (gn.. Does this limitation automaton? ffi reduce the power of the 1 0 . we ask that d (q. A nonerasing Tirring machine irr orre that cannot change a nonblank symbol to a blank. Consider a Ttrring machine that cannot write blarrks. 1 t .b.o) = (qi. Consider a Ttrring machine that.
1.8). with a specifies what lnppens on all the tapes.reas in Definition 0.r .l. L . Forrrralize this conccpt and show that thisr modification is equivalent to a standard Turing rnachine. For example. we define an rztaperrrachine M : (Q.8 Tapc 2 . If n : r:urrent configuration $hown in Figure 10. The formal delinition of a multitape T\rring machine goes beyond Definition 9.RI" 2. l'he synrtrol on the first tape will then be replaced with an r and its readwrite head will move to the left. where Q.258 Chopter  0 Orrrnn Mor)ELS or. d. . since it reqrrires a modified transition function. E.n e. Typically. The transition rule carr be applied only if the rnachirrc is in state Qsand the first readwrite head seesarr a and the second a.8. But this is not the ca$(]rL$we now illustrate with two examples. Multitope Turing Mqchines A multitape Thring machine is a T\rring rrrar:hine with several tapes. gs.F by a. R ) * is interpreted as follows. the svmbol on Figure 10. g0. then d ( q n . y .n.' TuRrr. ffi M T u r i n gM o c h i n e s i t h M o r e w C o m p l e xS t o r o g e The storagc device of a standard Thrirrg mrrchine is so simple that one might think it possible to gain power by using more complicated storage clevices.1. F).E.I.rc Mecruruns will allow the indicated movc if the currerrt tape symbol is neither a nor [r. brrt where d:QxI"Qxlu x IL. At the same time. e ) : ( q r . each with its own irrrlependently controlled readwrite head (Figure 10.
10 7 a lr t ) E "f h .tiorr of a machirrc with only one of its tapes doing useful work' mriltitrrpe machine by one with a single tape is a little more complicated. the twotape macldlle in the conliguratiotr depicted in Figlre 10. M. tr b Tapc I t v f Tape 2 the sccond tape is rewrittt+n as'g and the readwritc head moves right' The its control unit then chtr. The second part of this clirirrr treedsno elabclrtrtion. To show the equivalence between mrrltitape and sta. that any standartl T\uing rnachine can he simulated by a multitap() orle.ndartl Ttring machines.2 Tunrt'tc.yselect to run a.cHIunswIrH MoRl. The simulatirrg singletape machirre will have f'ogr Figure 10. we arguc tltat any given multitelpe Thring machintl M cau be simrtlated hy a statrdard Ttrring trachine M and. but corrceptually stra.q.nges state to 91 and the machine goes irrto the new conliguration shown in Figure 10.iglrtftrrward.9. Cotrsider.tplnx Sronacn 259 Figure 10.10.multitape 'Ihe simula.9 o. since we can alwa. converscly. for example. Corr.10.
Wherr we rcat:h the end of tht: a. exctlrt firr a.ccepted.il#T:.11). The fir:st track rcpresents the corrtcrrtsof ta.rticularly.. Tfa.ge a {a"'b"}. Wratever can trt: done of M can also be done on M.lso much the same.t a. :1fl :illi.unique corrtli.l' Considertlte ltrngua. the ones that have the indic:atedform).n deternine wlxlther there are arr cqrral number of a's and b's without rcpeated backanrlforth movement of the readwrite head.lHJil"'m. for the relevirnt txrnfigurations of M (that is..f M.tiou. It is irnporta.cks3 and 4 play a similar role for tirptr 2.1Tli:iJ'il1'#'JH. .*"'"IJ"ilili.lJJi' t . copying Lherrrrlrrto tape 2.lHf. The nonblank part of the second track hrrs a. Figure 10.single 1 tnarkirrg the position of M's readwrite iread.n initial strirrg e.lte nts.n offline rnachine. We tlrcn read all t. In Exampleg.pernachine is similar to that ust:d in the sitttulation of a.11 makes it clcirr that.rre10. the only differencebeing that thcrtl ilre more tapes to rxlnsicler. The actual steps in the simulatiorr irre a.pe 1 of M..rrr:e being lengthy and elaborattl.tionof a multitrrllc rnachine by a sirrgklta. wc c:ir. there rs a.11 trat:ks (Figr.nt to keep irr rnind the followirrg point. can txqm. we clescribecl laborious thir.'fhe outlinn given for the simulation of offline tnachinescat:riesover ttl this casewith rrrirrrlrmodifications and suggests a. Certainly. rnachines. The represerrtir.260 Chopier l0 Otrnn Moonls oF TLrRrNcMncnrrrps Figure 10.ll zeros.'s. While it is rrot difficult to nrake the corrstnrction precise.t'bttt is writtun on tape I at tlx. thu computations of M given the appeara. proceclure try wirich the transition function f "f fr can be constmr:tecJ fiom the transitirtn ftnction 6 of M. it takes a lot of writing. llrt this has no of beallng on the cottclusion.':i1?l.H':.7.il'Hl pa.ponding configrrrirtion of M. what la.we match tlrt: b's on tape 1 against the copied a's ou Lapc 2.laim that l.il'1il.l". This way. When we c. beginning of tlrc cornlltta..p[* ilCI.nguages be a.
3) contains b is shown iil F'igure 10'13' Note can invcllve arbitrnrily large intcgers.rt of the simulation of eat:hrrrove.10. not witlt respect to eilsc of progra. where [/ anrl D specify rnuvement of t]rc readwrits ]read up tr. ): Q x f .the rcadLelt us assume write heaci of thrl twoclirnensional ma. ancl cell (10.rrbe viewed as extcrrditrg infiniterly itr tnore tharr one dimensiotr. so orrclcornplica.rrrlrrgetnettt. wc associa. To simrrlate a move.12.ttxl in Figurc 10.shion indica. fbr example. twtr dimensiotnl Turing rnachine is shown in Figure 10'12.Ilrc MecHtut:s wIrH MoRn Colvtplnx SroRlGu 261 Remember that the various models of Thring ma. Wc will return to this importtrrrt point in Chnpter 14.12. T[e formal rJe:firritionof a.chintl M and ttre readwritcl head of the simrrlatirrg ma.2 Turr.12 Twodimensiond addressscheme . In thc scheme of Figure 10.tiorr:the cell a.t:hinesare considered equivalerrt only with respect to their ability to do things.2) contains a.tlrlress trse a fixedsizc field to sttlre addresstls. tr. the address track caunot we must rrstt a variable fieldsize a. Thc twotra.dclress. U . at tlre sta. the configrrration in whictr cell (1.ck tape of the silrulating rnirchiue will use clne track to stortl cell conttlnts and the other one to kcep the a^rsociated a.ck moclel depictcd in Figure 10. Instead. the simulating tlachine M first comprrtgs the addressof the tnll to Figure 10. D } .nd down.I x I ' x { .te ordering tape.13.l T\rrittg machine is otre in which the tape ca. L R . . twodimensional Ttrring machitre involvt:s a transition function tI of the fbrm . oll corre$poncling cells. rcspectively. First. A ditlgrart1 of a. Mochines Turing Multidimensionol A multidirrtensiona.mmirrg or auy othtlr efficiency mcasure we rnight consider. we (iaII use the orr twotra. To simqlate this machitte on a stanrlrrrd't\rring machine.chinc ffi ut* u]{tw. This t:atr Lredone irr or acldrnsswith the r:clls of the twtrditneusiona. in the twodimt:rrsioual ftr.using some spt:c:ialsymbols showu itt thc picture' to delimit the fields.l a lumher of ways.s that.
irrdependent rearJwrite heads.262 Chopter l0 Outtlrr MoDELSor. M Iinds the cell with this a. trriu is a sitnple computation. a clueue. the number of rcquirecl states in Exercise b as far as you can (Hint: the smallest possible nurnber is ihree).utomatorr in which the temporary storage is . A queue automaton is an a.chines is to lend cre'rhrirrg's dence to thesis by showing how seemingly rrrrlre complex situations cirn be sirnulated on a starrda.rrnter is tr stack with an alpha. In the exercises trelow.le readwrite heatl. cach with a sing. . Give a formal definition of such an autorrraton.tions are v()ry tedious irnd conceptually uninterr:sting.rd T\rrirrg machine. unfbrtunately. with the strirrg to be pror:essedplaced in thc queue prior to the start of the cornputation.. it has rro input file. A counter automaton is a deterrrrinistic autorrraton with orre or more counters as storage. 'I'uRrr*rc Mecrrrr{r:s Figure 10. Only the counter symbol t:an be put on the stack or removed frorrr it. then investigate its power in relation to Ttrring machines. describe the sirnrrlrrtions in just enough depth to show that the details can be worked out. Show that any T[rring machine carr be simulated using a counter autornaton with four counters.bet of exactly two symbols. Aga. ffi 2. The purpose of much of ortr discussiorr of Ttrring rna. a stack start symbol and a courrter symbol. detailed sirnuler. Give a formal dcfinitiorr of a multihearlmultitape Turing rnachine.pe and a single control 'nit but with multiple.h a multitape rrrar:hine.ck 2 and then changes the cell corrtents to account for tll rnove of M. show how srrch a machine can bc sirrrulated wit. and the[ show how suc:h a machine t:an bc simulated with a standard Ttrring machine.ddressorr tra.iu. A multihea. once tlrc trddressis computed. * 4. that is.tr: * 7' A c.in. 'Ihen sh6w how such a r'achine ca' be simulated by a standartl r[rring rnachine. . ffi * 5' Sho* that for every il\rring tnat:hine there exists an equivalent standartl Ttrring uachirre with no rnore than six states.'nuional addrcss scheme. Give a formal definition of a rrlrltihead T\rring machine.<lTrrring machirre can be visualized as a T\rring machirre with a single ta. using the twodirr*. 'lot"R*.. 3' Give a forrnal definition of a T\rring rnachine with a si'gle tape but multiple control units. 1. there is a stra.ightforwirrd construction fbr 44.. given M.It which M is to rnove. Assume that sur:h a machifle is an onli[e machine.
an appeal tg T[ring's thesis is inappropriate.. R) . M N o n d e t e r m i n i s tTc r i n g o c h i n e s iu While Thring's thcsis makes it plausible that the specific tape structure is immaterial to the power 9f the T\rring machine. Write out a detailecl prograrn for the computation in Example 10. *fimrttplClq. the range of d is a set of possible transitions.il 6y has If a Thring ma.c. ffi 9.& and qa1znal qzlcaa are both possible. atry of wlfch can be chosen by the machine. L)} it is nondeterministic.IES 263 g. showing that nondetermirristic behavior can be handled deterministit:ally. except that d is now a. b.a) : { (qr.3 NONDETER. I . Since nortdeterminism involves an element of choice and so has a nonmechanistic flavor.c"hirre transitionsspecilied . We must look at the effect of noildeterminisrn itt more dertail if we wa.MINISTIC IIURIIIIC MNCHII.mecannot be said of nondeterminism.l (qo. the sa. As a.nt to argue that notrdeterminism adds nothing to the power of a Turing machine' Again we resort to simrrlation.1.I O. A nondetermirtistic Thring rnachine is an lrrtomaton as given by Definition 9. (q2. show that everv cornputatiorr that can be done by a standard T\rring machine can be done by a multitape machine with a stayoption and at most two states.lwayswhen nondetcrminism is involved. The rloves qouaal bq$.1.function d:Qxf+2Qxrxtr'fi}.
Consider a Thring rnachine with a twodimensional tape (Figrrre 10. But. we need to provide a dcterministic equivalent for tht: nondeterrninisrn. unlirnited replication is certainly not within the llower of presentday computers.ruun Moonls or.2 and its succ:e. let us mrnsider an altcrnative view of nondeterminisrn.te or to an infinite loop. Nondetermirrism can be vicwed as a deterministic backtrrrcking algorithm. To see hovr this can be done simply. Figure 10.ining the rrrachine'stape. whenever a new machine is to be created. one which is ustlfirl in rnany arguments: a norrdrlterministic rnachine can be seen a$ one that has the ability to replicate itself whenever neces$ary. wiih q7 € F. A nondetcrministic 'rhrirrg machine is sairl to accept u if there is any possibre sequence of moves such that t111.the top track conta.rkersand so can always be found. It then carries out the irrdicated moves. Figure 10. these aJternatives are irrelevant. The simrilating machine searches aJl active tracksl they are bracketed with special rna.''IuRrrrc MACtrrNEts since it is not clear what role nontlcterminism plays i'computing function$. when more than orre rnove is possible.r2.264 Chopter l0 O..the processcarrbe simulated by a deterministic Thring machirre. To show that a nondeternrirristic Ttrring machine is no more powerful than a deterministic one.15 representsthe initial r:onfiguration of the machine in Example 10. nondeterninistic a. A nondeterrninistic machin{} may have move$ available that Iead to a nonfinal $ta. Nevertheless.re interested in is the existerrce of some sequence of moves lcading to acceptance. and a deterrninistic rrrar:hine can simulate a nondeterministic one as long as it can handle the bookkecping involved irr the backtrackirrg.reasonableoutline of the simulation. but we will leave this to the reader. Quite a few dctails have to tre resolved befcrrewe can claim to have a.14 Tape contents Internal state and position ofhead Tipe contents Internal state and position ofhead . all wc ir. This view of rrondeterminisrn rnav seern partit:ularly nonmechanistic. We have already alluded to one. the hottom one for indicatirrg its internal state and the position of thc readwrite hczrd.$sor configurrr.ut i qq. as always with nondetcrminism. two new tracks are rrtarted with the appropriate information.14). Each pair of horizontal tracks representsone machirrc.utomata are usually viewerl as accepters.tions. activating new machines as needed. the machine prt)duces as marry replicas as needed and givcs eaclt replica the task of carrying out one of the alternatives.
Iated bv a deterrnirristic machine. Show how a twodimerrsional nondeterministic T\rring machine can be sirrru3.3 NoNDETERMINIsTIc 265 Figure 10. Use the: corrstruction suggested atrove to show that anY nondeterministic Ttrring rnachine can be simulated by a deterministic one.ccepts the 7={ww:ue{a. Because of its irnportance. r I.TunInc Mncutt'lps 10. Proof. b}+ . t €. Itow artive machines are identified. . our conclusion is that for every nondeterministic: Thring machine there exists an equivalent deterministic one. Based on this simulation. Outline how one would write a program for a nondeterministic chine to arcept the langrrage L: Tltring ma  ( { . l"l } lgl} How would you solve this problenr deterministically? . \ r U ' u t . ) Ttrring machine that accepts 5. Irrdicate explicitly how new machines are created. W 4. 'I\rring machines and the class of nondetermirristic The class of determirristic 'I\rring machincs are equivalent. 2 . Write a simple pl'ogram for a nondeterministic the language 7 : {***"a i 'r1a1ue {a." u : t r . " 'a.bl+I. Contrast this with a deterministic solution. and how machines that halt are remove<l frorn further consideration. Discuss in detail the simrrlation of a nondeterministic Ttrring machine by a deterministic one. we state this formally. Writc a progranr for a trondeterministic Ianguage T\rrirg machine that a.11'r Sirmrlation of a rrorrtletenninistic fitove.b}+}.
Consequently. We rnay. With the initial rrrrrl final state and the bltr. To construct such irn M. on the other hand.mple.a3. assume that Q = qt. Desigtr a rrorrdeterministic Turing rnat:hirrethat accepts the language L: {a" : rr is rrot a prime uumber} . and f : { a r. rI r with q1 thc irritial state.1 is a special purpose cxrmprrter.. given ils input the descriptiorr of any T\rring machine Jl[ ir. a2 as 11. . tht: ma. is arr autclma. q2 the sirrgleIinal state.rd way of descritring T\rring machines. a1 is encoded as 1.x f . string ?u.ffi Cortsider the fbllowing argurncrrt a. etc.. . ffi * 7 . Thring machines cannot be considered equivalent to gerrcral purposc digital computers. A twostack autornaton is a nondeterministic puslxlown arrtomaton with two indepcndent stacks. can simrrLlte the computatiorr of M on ro. & r n } . 1 1 .ton thal." This ohjection can be ovr:rrxlme by designirrg a rcprogrammahle Ttrring machirrc. Similarly. The transition function is encoded actxrrding to this scherne. .grr. vrith the arguments and result itt sornc prescribed sequencc. where a1 rcpresents the blank. Digital computers.Qz. without loss of gerrerality.uring rrrachines.r. . qz is representedby 11. A rrnivt'rsill T\rring rnachine M. T<r clefine such an autornaton. d(gt. and so on.'s. uz) : (qz. o A movc dcpends orr the tops of the two stacks arrtl results in new values being pushed on these two stacks.chine is restricted to ca..1 so that d : Q x (tU{A}) x l' x f * f i r r i t es u h s e t s f Q x l . we modify Dcfinition 7. Once d is defitred. ."L) might appear a.t MAcHTNES 6 .nd a. & 2 . .nk defined by this corrvcntion. called a. Wc then select an encodirrg in which q1 is representedlry 1. The symbol 0 will be rrsedas a separator btrtweenthe J.. NN A Universol TuringMochine ffiffiffifif. . ir. .inst Thring's thesis: "A Ttrring machine as preuented in Definition 9.rrying out one partir:rrlrrr type of computation. Slrow that the class of twostar:k automata is equivalent to the class of T. .universal T\ring machine. For exa. 1 0 1 1 0 1 1 01 0 1 0 .s . we first choose a standa.266 Chopter l0 OrHnn Monnls oF TrrHrNC.Q.nyTtring rnachirre r:an be described cornpkrtcly with d only. arc guneral purpose machines that can be prograrnmed to do rliff'erent jobs at differerrt tirnes.
4.Tz.T:1. we need to review some resrrltufrorn sct theory.nt irnplications. but the process is 'I\uing's rrninterersting.c(:Hr'lr: 267 Figure 10. and Qualitz 1978)... 1 } and the stnx:trrrc of ir.ngua. call carry out the comprrttr..nd tapc 3 thc irrtertral state oI Il[. we carl clecodeit uniquely. given any encodingof M.pe I will keep a.44 Tape contents of ll4 It folklws frotr this thirt arry Thrirrg rrrachirrehas a fitrite ettcoditrg as a string trrr {0. of far:t.16. thrlrr has itn irrprtt alplnbet tlrrrt inits cludes {0.T\uing trachine that.rrlrltititllt: milt:}rirrt:.onettl_oncrxlrrt)tporrrlerrce this we mean that tht: clernerrtsof the set can be written in ${lme ordtlr.chinecan het rcprtlserrted by a string of 0's and l's ha. Wrr prcfer irrstead to appeal to hypothesis.4 A Lilrvllnsnl TuRrNc M. lrut most of the interestirrgst:ts (and languages)are infinite. for exiltrplt:.s iruporta.16 Control unit of M. 1}+.2. thirt cvtlry elertretrtof tire sel has some fitrite index.rr11 stl . Internal state of .Thereforc. Some strings will not represent any Turing machine (e.lurriversal Turittg tnaciritre (see.nbe put into a. ta. higher level la..tiorrssptrilied by that progl'an ancl tha. It is within rea$en to txlnstnrr:t an actuar.10. Wetarc tlrt:rrjustified in claiming the existerrceof a.1 is a rea.I)enrris.g.. wtl tlxpect that it catr also be done by a stanclard T\rring machine.. For infinite set$. but we can easily spot these. Sone sets are finite..pcs 2 arrd 3 to deternine the tape 1 to seewhat M would do in this configurtrtion of M.t t]rc cxlrrtcrrts of ta. Section 9.reof no concern. sirlr. utriversal Ttrring machiue in a... It thc:rrcxrrrsults c:onliguratiorr. Firrally.ge. thtl l)r()grilrrrsuggesteditr Exercise 1. By ca. The in irnplerrxlrrtrltirlrr tlc:irrly catr be done usitrg solne programming la.thc set of all even iutegerscau be written in therordcr 0. A set is snirl to trc coutrtable if its eletnents with the positive integers. so they a. we distinguisir trtrtwccrrsets that are countable ancl sets that are uncountable.lizir.\rring ma.tion ir. givcrr any program. For atry input M and u.t is therefbre ir proper urorlt:l firr ir gcneral purpose cotnputer.n encxrdeddofirritiorr of M. sltowtt in l'igure l 0.. tapes 2 and 3 will be rnodified to reflect the resrilt of the rnove. A universal T\rring rnilr:hirrc M.ttguage.. a. Brrt bcfbre we explore these implications. . The observation lhat every T. Tape 2 will contain the tnpe cxrntentsof M. rrrrclthat. the strong 00011). M* looks first ir. Fclr cxilrnplt:. Dt:nrrirrg..
n carry out the secluenceof steps qoni q"rr # ELi Q"'rz sz . We see from this t:xirrnplc thir"t we can prove that a set is courrtabltl if we can produce a rnethotl try whir:h its elements can be written in sorrre $equence. or:r:rrrsin the eiglrth place.y seem counterirrtuitive. occurs irr position n { 1.We call such a rnethod arl enumeration procedure.bet E. irr This eives us 1 1 2 1 2 t ' z 'T ' 3 ' i ' " Here the elerrtcrrt f.{#} . Lrlok ir.tesignifying rnernbershipirr S.268 ChopterI0 Ornpn Morrtlt. 'I'ake the set of all quotitlrrts of tlte fortrt ytf q.there is a. wlrt:rc p ilrxl q a. # witlr 1llr1 I'* . irr this r:asu. How should we order this set to show that it is courrtabk:'l Wc t:ilnnot write 1 1 1 I I't'3'4'. tlxr strirrg folbwing f rnust be in S.ppear.tr:q. Since an proce$$1 can use a {lrurrnr:rationprocedure is sorne kind of rrrer:}rirrrit:al we Thring trracltirrtl rnork:l to define it formally. the set is countable. whenever q" is entercrl. This does not irnply tlnt thc stlt is uncountable. e finite number of steps. ancl every elerrrent has sotrc place in the seqrrcrrc:t:.sol''l'unrlc MAcHTNES Figure 10.. This shorrldnot be too surprising. Let 5 be a set of strirrgs orr rromrtnlpha.9 is prodrrt:rxl irr a. and write dowtt the elerrrerrt thc rlrder encountereclfollowing the arrows.repositive integers.. becaust:thtrrr fr worrld never a... ".t is. Tht: set is therefore countable. clever way of ordering the set to show that it is in fact courrtablr:. Then an enurneratiorrproc:+ 'Ihrirrg dure for 5 is a rrrirc:hirxr thilt r:rr.. sorrre of whit:h ma." is rr. .17. tha.17 Since any positive integer 2n. but there are lrlore corrrplic:irttxl cxilrrrllltis.sta. in such a way that any s in . Tlxr stir. € S'.t the scheme depicted in Figure 10.
nenumeration prclcedurethrr. ca. r.10. Witlt this enProof: We ca. in which wtl takc the lerrgth of thc strirrg as thc first criteriorr. But any set for which an enurneration procedure exists is corrntable bccarrsethe enumeration gives the required sequence. finitc number of steps. cc.If so. 2. Genera.chines a.tethe next string irr {0. iln (:rnrrnerirtiorr llrttcedur{l crrrrrrot be called att algorithm.aae.chT\rrirrg rna. all worcls beginning with a a. the order used in dictionaries is not sr.t prodrrcesits elermrrnts sornr:rlrrklr.f'tcrir. However.it r:an bc c:onsirlcrcrl it rnerrrrirrgfrrlllrot:cus.r:trrrrr Stcp 1. As we will have severa. 1}* irr properl orcler.ac.te when S is infinitel. As we will seein the next chapter. Let X : {a. bc{:ir. Instead.hb.n enumera.3 that any given string he listtNl a. But when there ale an infinite number of a worcls. since it will not termina..r:hirrtl coding. It.pe in the form rr:tpircd by Definition 10. cb.lt]xtug]rirrfirritc.tion..luses for sur:h an orclerrirrgwcl will crrll it the proper order. tr. ba. This is a.TunlNc Macnrr'm 269 Not every set is countable.3. we will never reach b.chines.4 A Ur{rvnnsal..u$e prochx:cs wcllrlc:firrcxlarrd it prtxlir:tabk: rtlsults. is countable. ignore the string.recorrntable. ua.ting the condition of Definition 10.relisted befbre the stling b. If not. $trir:tly speakirrg. we (:an rrse il rxldificd rlrdrlr. sa. ab. to Since every Tlrring rnilt:hirrc hirs a linite description. I Arr irrrportant consequence of the above discussion is that T\rrirrg mir.n errcodeeir. Tlrrirrg rnirchirre.r. I  . We can show t}rat the S : X+ is countable if we can find in n. thus viola.ry.bc:.t protlut:us thc sequence d. R. followed by an alphabetic ordering of all equallength strings. c}. b. we thrlrr rxrrrstnr(:tthc following euunreralion procedure. 3.lly lltl gtlrrt:rateclby this process.ry. NevertlNrluss.ritablewithout modifica.tionprocedure tha. write it on the ta. In tr rlictiona. The set of all T[rring nta.rin a dictiorra. usirrg 0 and 1. Chcck the getreratecl string to see if it defines a. 1.yin the order in which they would appea. there are sorne uncountable sets. any specific milr:hirx: will eventrrir.
{h 4 .tis restricted to being used like a stack.L) . find a function / (tt) that gives for ea. 2 .?.he encoding we use..pe rrsrr. 3.ce. positive irrtegers. 4 r . What is the irrrlex of ()tlr in Exercise 3? D. tnpe thar.. d (93.s TtrRrN(+ oF The particular ordering of T\rring machines clepends on l. We have already seenan example of this with pushclowtrautornatir. ffi L 9 . Give a complete encoding. Suppose that Sr and 5'r are coutrtable sets. 7: {anb" : ri } 1} 6 . For Example 10. a . A llrshdown automaton can be regarded as a.chine. if we use a diffcrclrrt crrcxlrlirrg.r:kto finite arrtomata.7. az. Sketch a Turing rnachinc program that enumerates the set {0. we nright permit only a finite part of tirc tirlrrl to tlc ustxl as work $pa.3.$ffiffi1f BoundedAufomqto Lineor While it is rrot possiblc tri qrxtendthe power of the standard 'I\rring machine by complicating the tirpc: stnrt:trrrt:. I t ) .a r ) : ( S r . This is of rro conserlrrcrr(i(lr howevcr. Slrow that then Sr U Sz and . using the suggestcclmcthod.chtl its index irr the propel ordering. Sketch an algorithm that exarnines a strirrg in {t).a r .1}t o. . irr proper Design a Ttrirtg rnat:hirre that emrmeraterr the following set in proper ordcr. ir.9r x .wc rmrtlt oxllrx:t a rlifferent orrklrirrg. . a : ) : ( g r .j.h) with . We catr also restrict thc tir. nondeterrninislic I'uring rrrachirrewith a. it is possible to limit it by restricting the way itr which tlrt: ta. 1 . (i.i. 1 . gpi.270 Chopterl0 Mncslrur:s OurnR MoDEr. What rrrattcrs is thc cxisttlrrr:t:tlf sorntl rlrrlt:rirrg.Je. It can be shown that this leads us bir. ) .92 arc also countallle..ge other ways. I} I to tleterrnine w}rether or not it represents an encoded 'I\rring rna..rrd slxrws that the ordering itrrelf is utrimportant. Show that the set of all triplcts.1lc r:an br: rrsed. (see . is countk able. Show that the Cartesian uroduct of a finite trurnber of countable sets is rrlrntable. fbr in tlxample. i ( g r.a r ) : (S. for thc Ttrring rnachine with d ( g r .
ccepted a linea. suclt tlnt d (qr.rhorrndcd irutornirtorl is assumed to but essential he nondeterministic. a. For sorne exDloration of this. The errd rrrarkers cannot be rewritten.t in this clefinition a.'nc 27L Ext:rr:isc3 rr.$[iP'i. Rut there is ir wiry of lirnitirrg taptl usr: tha.St*.l) catr contain only elerrrents the forrn (qi.sin Definition 10.pe. Tlxl lirrrguirgcacccpted by the lba is the set Note tha.10.ttho end of this section)r $o we need not pursue this. has an unbounded ta. We sometimes say that the readwrite head "bounces" off the end markers.n2 € f*. Thus. generating another class of machines.5 LINITAR Brturunr:tr Au'r'oNl. see Exercise I at the end of this section. While one can deline deterministic lba's.rlsto rl rnore interesting situation: we allow the tnachine to use only that part of the tape occupied by tht: input.d..1lffi A linear lrounded a. we can envision the input as hracketed by two specia.l symbols. but how much of the tape can be used is a function of the input. 1t).l. For ilrr irrput rl. Iike a standard T\rring machine.ninPii1.rtl ecprivalent to the nondeterministic version.]) can contain only elementsof the of form (qi. In particular. A linear bounded automaton.harr:t:pttxl strings. To enforce this.n.rbounded autornaton if there is a possible by s(xllr()rrcc{lf movr:s qn i l*rqr*rl lrl fbr some et E F. of irll suc.l rxrrrfigurilti<lrr thc T\rring of rnachine is given by the instarrtaneousdescriptiort qn[r].E.L).q. we restrict the usable part of the tape to exactly the cells taken by the input.t lt:ir. the linear bounded automata (or lba).srrhiectto the rerstric:tion that X rmrst r:orrtairr two spct:ial synrbols land ]. the leftend marker ([) and the rightend marker (]). i i l A string 'uris a. and the readwrite head cannot rnove to the left of I or to the right of ]. This is not iust il rrrilttt)r of corrvetrience to tlxl rlisc:ussiorr lba's. it is not of known whrrtlxlr thc:y ir.]. r.l. tlrtl irritia.. F). .2.rings Lhan for short ones. Iinea.t)1.Qo. more space is available for long input sl. and d(qi.utomaton is a nondeterministic Tirring machine M : (8.
Tht: tapc of a linq+ilrboundtxl automatorr rnay be multitrack.n be accepted by a linear bounded automaton. and the extra tracks can be used as scratch space. since neither of the languages are contextfiee.8.ii$ti$.5 a. and continue rmtil we either find a nonzero remainder or are left with a single a. 1 8 . a more direct approach is srrggestedin Exercises 5 and 6 a.t the end of this section.:{u. This follows from the discussiorrin Example 9. For this problem. IJsirrg the divisor on tlte secortcltrack. we dividc the rrurntrer of c. The computation ouLlined there does not require spaceoutside the original input. I Thc last twcl cxilrnplt:s suggest thirt lincir. I 6xil. l.t$ orr the first track.. One way to solve the problem is to divide the number of tl's successively by 2.Find a linear bounded autornaton that accepts the language L:{ont. we can use a twotrack tape. After this. are mortl powerful than pushdown automata. To prove srrch a conjecture. sirrce art atrount of scrirttlh spircc l)rol)ortiorral I ' i g r r r cl { ) .r bounded automata. say lry retrovirrg all syrnbols exccJrt those at multiples of tlx: clivisor. if not.18).We sketch the solution to point out one tacit implication of Dcfinition 10.rz}0}.. Problems like Exarnple 10.ilL4li l'he Ianguage [.tl.mF.3. until we can either accept or reject the string.4. The first track contains the nurnber of a's lefb drrring the processof division. and the second track conta. we still have to show that any contextfiee language ctr. wc irrcrement the divisor by one.. eventually there will be a single a lefl.re invariably solvable by a littear borrnded aut{rilratorr.rise..4. It is not so ea"Ey to rnake a conjecture orr tlrtl relatiorr betweerr Thrirtg rrtachirres arrd lirx:ar botrrrrlcrl arrtorrata. at some point a nonzero remainder will a.272 Chopterl0 Ornen Morrus on Tunrlc MAcsltlss Exanrplle. Ifthe input is in L.nltncn:z>1} is accepted by sorne linear bounded autouraton. We will do this later in a somewhat roundabout way. The actual sohrtion is fairly sirrple. so it can be carried out by a linear bounded automaton.insthe current divisor (Figure 10.
b}+.rtt*.ure a." : n is a prirne mrrnher} (") .utomaton.5. 6 . . 2 . is not a prime number} (d) r: ( e )I = {ww. Show that such a machine is equivalent to a finite autornatot.b}+} { D.5.5 that does rrot require a rrecontl track as scratch space. On its work tape. W Use thc observation in the abovc exercise to show that anv contextfree langrrage not containing A is accepted by some linear bounded automaton.w e {a. Examine your solutions to Exercise 4. L: { a ' " : n : n f . it is quite difficult to come up with a concrete and explicitly defined larrguage that r:rr.utornata? If not.n>1} W (f) f : {rururty't:.5 LINFTAR FJor.iruuutAurcruere 273 to the length of the input is availatrle. In Chapter 11 we will show that thc r:la. Find an lba fbr the cornplernent of the larrguage irr Exarrrple 10.10. we carl use Definition 10.ss of lirrcar brlrnded arrtomata is less powerful than the class of unrestricted Thritrg tttachirres. it can use at most n extra cells fbr work space. is fixed for all inputrr. but il dc:monutra. Are the sohrtions all deterministic linear bounded a.nnothe accepted by rrny linear hounded a. arrd not rewritten. such L To define a detenninistic linear bounded autornaton.4. L Give details for the solution of Exat4rle 10. try to fin<l rrolrrtions that are.b}. ( * ) .nr}l} (b) fI : {o. Consider an offline Turing machine in which the input can be read only once. ffi 3 . Find a solution for Example 10. Irr ftu:t. Fittd lirtear bountlerl automata for the following languages. that the number ofsymbols in the stack never exceedsthe length ofthe input string try urore than .tion of this requires a lot more work. but require that the Thring machinc be deterministic. assurrring thatE:{a. 4.b}+} {wn:w €{a. rnovirrg left to rigtrt.L : {ai'o : n. Show that for every contextfree language there exists an accepting pda. whete rt.
.
remore la. This leacls to a . I'he proof is slrort itntl c:lt:garrt. Our irrrrrrediirtcgoal will bc to cxa.ges.crx.A Hierqrchy f o F o r m q lL q n g u o g e s ond Automqtq e ttow returtr our atterrtion tt) (lrr rnairr intelrc$t. so that thtlrc rmrst llt: sornt: Iarrgrrilgcs for which there a.tnplesthat actually allow us t.t I ti .milies.nguages 'I\rring associat. thtlsc grarnmilrs rlnd relgrrlrrra.lgrlrithrnir: r:ornputation.ed with tnircltirrcsilrrtl srlrncof thcir rostrit:tions. Re'Ihritrg cause tnachiues can perforrn arry kirrtl of ir.clrine. lbr this rcas()rr.but also the various exarnples we have crrcourrtereclthat lic outsidc tht:sc fir.l questiorr is whether there are arzy languages that arc rxlt a.re no Ttring rnachitres. Ilrt norrr:r)n$tnx:tive.mmars.minethe la.tionwill be to look aL the relatiorr betweerrThrirrg rnachirrtlsirrrrl t:crtilin tyllt:s of grirmmar$ and to esta. we expect to finrJ tira.blish connection between a. and gives little insight into the probleut.the study clf fbrrrral Ianguages. Arxltlrcr avenue of investiga.nd r:ontextfiee gra.t the family of languages a"ssociated with thern is rpitt: llroarl. It int:hrrlcs not only regula.'pted some Tirring ma.wt: will cstalllisir tlx: cxistenrx: of latrguagestrot recogrrizablcLry Tlrring rnar:hinesthrrnrgh mrirc cxplicit exa.r a.The nontrivia.ncJ contextfree langua.o iderttify oric suc:h lirrrguagc:.nguages than T\rring rnachines. We will answer this queshy tiorr lirst by slxlwing thrrt tlxrrc a.
Thc dtrlirritiorr says rrothirrg about what lnppetrs for 'rr.tncl cliscussecl this r:haptor.ngua. . is said to be recursively enumerable if there exists a.t accepts it. for la.tc. Ber:inrsea T\rrirrg rnat:hirrt: does not necussarilyhalt rlrr irrput that it rloes rrot accept.king. This deflnition implies only that there exists a Ttring rnachine M.tccvurythirrg so tlrrrt . tyyut I y :I:tqII:21 with gy il finill ster.r not irr .memhership algorithm. fbr every ?0€ L.ges that do not inchrrlc thc tlrnpty strirrg. w{: rnake t}re tacit assutnptiotr l}rat the languages adcl errepr.L. canuot accept the ernpty strirrg. do ttot contain A.chinetha.Sorrresettheoretic cliagrarnsillustrate the relationships tx:twt:crr various language families clearly. we have defined theur.ctthat T\rrirrg urac:hirrt:s. it rnay btl that the rrrachine halts irr a rrotrfinal state or that it never halts and goes into an infinite loop.\ is irrcluded. To irvoitl havitrg to replrrase the definition or having to disr:lairncr. the first does trot irnply the second.ra. We can Lremore demanding irnd ask that Lhe rnachine tell us whether or not any given input is in its Ia. many of th{r irrgurnetttsitt this clmpter are valid only Strictly spea. It is a in trivia.ted with Thring rnachines.276 Chopter I I A HlrnnnclHv or FoRtr.ge. In cloing sor we rnust nrake the important distinction between Ianguagesfor which there exists an accepting Ttrring machine and klnguages for which there exists a. but we will leave this to tire rerrder. such lhat. rrnll:ssothcrwist: statetl. Recursive ond Recursively irilmmffiffiilt E n u m e r o b l[e n g u o g e s o We start with some terminology for the languages associa.rc:hyof grilmrnars arrd tlrrough it to a method for classifying ltrnguerge f'rrrrrilics.tll Leucuacns axn Autoraana hierir.ngua. A language1. This restrictiotr arises as fiom the frr.Ttrring ma.l matter to rtrstir.
'r'hat thcrr: is a. the machirrc M.ture of either recursive or recur$ively enrurrerallkllanguages.lsoa. It is easy to sec tha.. lf 'ur E . We cannot use the erhoveargumerrt ir. finite numlrer of steps.2 . cvery string in . 'I'hese delirritions a.rFirqflir$.2 givc: rr$ very little insiglrt irrto the na. Defitdtions ll. The order of ltrlrftlrma.. let us sa/ '{t1. clifferent way.J ancl 11.te arrrl lt:t M execute one rrr()ve it.llyproduced by M.Iriion A languirge tr on x is said to be recursive if there exists a 'rhrirrg machine M that ar:rx:pts L and tlut halts on every .'i irr the enurneratiorr. and the processr.tiorrprocedure. Wc Iirst get fr to glenera.ncunslvl:lv Exulannenlt Larvcuacns 277 I .tr:h.L is cvcntrra. language is lrx:ursive. Trr make sure that this does not hrlppen. Wc first construct anotlrrtr lhring rnac:hirre. Since any ur e tr is generatecl by ffi ancl acceptecl by M in a. when startecl with ttj ou its tirltt:.1 R.they becourethr: irrprrt to M.ted by the errurnera. Sr4lposethat M is rr T\uing machine thir.but shcd u2 Figure 11.1.ssociated with rtrring rnachines. wc will eventually get a rna.t cvery language frrr which a.1. a Ia. that gcrreratesall strings in X+ in proper orrler. beciruseif sorne rui is rrot in 1. As wltir:h is modifled so that it writes strings orr its tape only if thcy are in L. We sirrrply cornpare the given irrput string agairrst successivestrirrgs gcriera.t rleternines mernbtirship iu a recursivtr la. Then we ltlt M generate u1 on ur2 irnd let M executt: one rnove on ur2.r that M will never get into an infinitcr Ioop. If a.11.nceis depictcxl in Figure 11. then there cxists an easily cclrrstnrctedenuttterirtion procedure. that follow rr. may never halt rrrrrltirerefore never grlt to the strings irr r.. After this. In olher word$.ECURsrvrrt AND R.L. the third step on [r1.nenurrrerationprocedure lbr cvery recursively enumera'ble languagc is not as easy to srlrl. the cornllrtation is perforrnetl irr a.nenumeraticlrr pror:edure clxistsis recursively crurmerahle.nguageL. Flom this. we gerrrlrate?lr3a.1 First move wl j uj Secondmove Third move ..nn be terminatecl.nguage rrlcursive if and only if there exists a rncrnbership algorithur for is it.r)2. folklwod by the second ilr()vc on urr.ro in X+. the second step on ur2. rrrd sri on.nd clo one stcp on ur3. say M.ttilr:h narnes to languagc I'nmiliesa. it is clca.. thesc strings are generatrlrl.sit stands.
replace 0 with 1.n 0 f 0 ) 0 0 1 t f 0 ] \*. is represented 10101"" Clearly. as shown in Figure 11.2. Proof: I.use differs from fr through s1.tR...} represcntcd by such a sequencc.ts. But it camrot be f1 bet:ir. Thtrn any element f of 2" can be represented of by a sequenc:c 0's attd L's. and compklrnerrt each entry. Suppose that 2s were countable.. describablt: sotttehow. the elerments resrrlt. "r. In this ta. that are not recursively emrmtlrable?tt While we will be able to supply some &nswers.. say t!... For the same Figure 11.s6} is representedby 01100100.2.nguagefarnilies we have tlrrcountered before... and we could enter these into a table.et S : {"r."r. Let S tre an infi.take the elements in the maitr diagonal..en iu some ordcr.lly Enumeroble Longuoges Are Not Recursively Thqt We can establish the existence of latrguages that are rrot recursively enumerable in a variety of ways. t t (t) \. s3. the set {"r. say f1 for some it i.2 'l 0 r l t L l l t. that is..278 Chopter I I ANDAuToMATA A Hrnnancny o!' FoRMALLANcuAGEs no light on thc nature of represerrrtativelanguages in these families. as the example in Figure11..hk:. and vice versa... arrd atry such sequentlc represents a unique clernent of 2s. then its elements could be writl. The trew sequencerepreserrtssome element of 2D.we will not be able to produce very thc second one' explicit examples to illustrate these questions.. Then its powcrset 2* is not countable. N6r do they tell rrs much about the rclationships between these languages or their conmlt:tion to the la. Orte is very short and rrsesa very fundamentir.. For example. We "Are there languages are thercfore immediately faced with question such as "Are there lanthat are recursively orrurnerable but not recursive?" and guages.../j : . any eletrerrtof 25 can be by {rr..so we get 0011. In the are 1L00. especia... with a 1 in position i if and only if sa is in while t...I and elegant result of matherrratics..}.nite countirble set.t2.
this irnplies that there rrlrst be some languageri on x thar arc rrot recursivelyenurnera. it gives us no ferrling at all for whirt these la. I This proof. there exist larrguagesthat are rrot recursively errurnerable. Cantor.it is possitrkl.I 1.ssurnption lhat 2s is countable. This contradiction crcates a logical impasse that can be rernoved orrly by throwirrg out the a.srrbsetof E*.nguagesrnight look like. arrclevery such sutrset is a language. For any nonempty x.renot recursivcly enumerable. It is completely rronconstructive ancl. who used it to clernonsrrirre that thc uet of real nurnbers is not countable. we will see a similar argurrrent in several contexts.ren. is called diagonalization.l is diagonalization in its purest form. By Exercisc 16 at the end of this section. becausc it involves a rrranipulation of tlxr criagonal elements of a table. so the set of all recursively enurnerable la. A longuoge Thotls Not Recursively Enumeroble Since tlvery language that carr he described irr ir direct algorithmic fashion can be acct:pted by a I\rring rnirc:hineand hence is recursively enurneratrle.nguagesis courrttrble. .rurr. But the set of all T\rring rnachincs t:tr.there are fcwer T\rring nrachineu than there are languages. the description of a larrguagc that is not recrrr$ively enumerable must be indirect. F. As an irnrnediate consequrlnt:e this result. Nevrlrtheless.ges that are not rur:ursively enurnerable. Since X* is infinite. eilthough short arrd simple. The techniquc is attrilruted to the mathcrnatician G. Thercftrre the set of all Innguagesis exactly 2E. In the next set of results. Theorern 1l. Proof: A language is a. 1 R. or arry other fi.c. is irr rnany ways unsatisfyirrg. Theorem 11.srvgLy Er. we investigate the conclusion rnore explicitly. In the next fcrw chapters. I This kind of arflument. in some of serrue.ble. while it tells us of the existence of sorne larrgua.nbe enumerated.eLH LANCIUAGES 275 rca$on it cannot tre f2. so that there must be sorne languages that ir. we can show that.ECUR. The argumerrt involves a variation orr the diagonalization theme.ncuRsrvg AND R.1 tells us that the srrt of all languagrn on E is not countable. f3. Tltcre exists a recursively enumerable lirrrguage whose cornplenrerrt is not recursivsly enurnerable.
L is recursively emrmerable' I  .twn AuroMere Proof: Let E : {o. by Definition 11. The contradiction is inescapablc. For each i ) 1.1) well definctl but. sucltrthat T: t lttt*1 (1r.' e tr' Therefore.For cach Tirring rlachile Mi.Iso We will show this bv srtrtradiction. Is it in . By Theorem 10. if we assume tha. The combirred effect of this is a Ttrring tnachine that accepts every o. is ttot recursively enumerable. say Mp. and consider the set of all Ttrring machirreswith this input alpha. with its c:lements.1) now irnplies that o *f T .L is well defiiled. that simulates tlte action of M orr a'.ourrtable.rrguage .. we give its description along with a" to a universal T\rrirrg rnachine M. We can use fOr this the knowrt enumeration procedure for T\rring machittes. If this is tro. we corlsidet the complcment of L. since the Btatement u' € L (Mr)' and ltettce a' E tr' firust either btl true or fhlsc. ke L ( M i l .cus. we first find i by counting the number of a's. tlre string a'. (11.L..1.swe will show. then there must be some Ttuirtg machine. It is clcirr that the la. the computatign carried out by M. We now cotrsider a ncw language tr dcfined as follows.M2.. strrrting from the assumption that . Next.t ak is in ..L if arrd oilly if ad E L(Mt). which is rr. ) } .2) implies that ak 4 t' ltvt*1. will eventually halt.1) we get that ak eT.L is recrrrsively enumerablc. qirch recursively etrumerable larrguage on X. If a' is in tr. To complete the proof of the theorem a$ $tated' we must still show that .. is in. and we must t:onclude that olr assumption that Z is rccursively enumerable is false. Conversely.280 oF Chopter  1 A Htunnrrc:IIY FoRMAI Leucu'q.bet. therr oo # L and (11. But (11.L or in Z? Suppose that ak eZ. there Conversely.2) this implies that a . a.}. T: { o o' a i 4 L ( M . Given a'. Finally. there is some T\rring machirrt: that accelrts it. But thett from (11.3.so wc can associatc arr orcler Mt. this set is t.L is recqrsively enumerable.2) Consider the strirrg aft.By (11. We therr use the enumeration procedure for I\ring machirres to firrcl Mr. for is an associated recursively enumerable lirrrguage L(Mt).
If a lattgutlge tr and its complernt:nt Z are both recursivrlly enumerable. that is.anln LRrucu.{. otherwisc it is in Z.qcns ZBI The proof of this theorern cxplicitly exhibits. Proof: If tr and Lare both recursively enunrerable.ECTTRSIVELy _ErruunR. . we need do so in a rather roundabout way. wc: next let M generate u2. Nevertheless.. If they are not thc same. assume that tr is recursive. If we need to continue. The language tr irr Tlrrx>rem11. by Theorem 11. For tho (ionverse. and so on.1 RFrcuRSrvE ANDR. giving us thr: krokedfor example. zrrbelongs to L..then there exist TLrring machirres M and M that scrve a.L is rcr. sively enumerable. so eventually we will get n match. But this brxxrmesa member$hip algorithm fbr Z by simply corrrplt:rnenting its conclusion.in. we Iet M gerrerate fi1 and compare again.then Z is also rccursive. thc uecond fr1.rlt2. throrrgh (11. Thiu is not to say that there is arr easy. we cxrnclude directly that the family of recursively enurrrertr. so they are both recursive.srrbsidia. The first will produce rily.4. A Longuoge Thotls Recursively Enumeroble But Not Recursive Next. then both languagesare recursive.s enumeratiorr procedures for L and Z. Sirrce any recursive language is recursivcly enumerable.3 is in the first but not in the second family. Suppose now we are given any ?r.fr2. Then there exists a membership algorithm for it. Arry ru € X+ will be gerreratgd either by M or .renot identical. There exists a recursively enumerable languaga that is not recursive. the proof is completed. Thereftrre.3. Proof. The process is a membership algorithrn for both L and . and consequently lloth are recursively errrrnrerable. I Flom this.4. If. Coruider the la.blelanguages and tlx: f'rr.lrr$ive. We begin by establishirrg tr. in L.ngrrage of Theorerrr 11. Therefbre Z is recursive. the family of re:cursive languages is a prolrer subset of the family of recrrrsivelyenurnerablelangrrages. respectively.11. We first letff generate ?x1and compare it with ru..milyof recursive larrguages a.ry result. then M gerrcrate fi2. I : ... This language is recur1.€ E+. intriltive interpretation of Z. but its complement is not. Aga. If the rrmtching string is procluced by M. in tr. Z is llroperly defined. we show there are some languagersthat are recursivrlly enumerable but not recursive. a welldeflned Ianguage that is not recursively enumerable.1). it is not recursive.L. it would be difficult to exhibit more than a ferw trivial members of this language.
L4. Show that S'r must then r:orrtairran infinite number of elements that are not in.1 fail when S is firrite'l 19. If L is ret:ursive. Slxrw that l+ arrd suggcst a. 13.r'gumcnt in Theorerrr 11. Show that the family of recrrrsively enumerable languages is closed under union. and with it. Prove that the set of all languages that are trot recursively enurnerablc is not countable. In Exercise 16. Let L bc a finite larrguage.Sr' 17. ffi 7. 18. Prove that the cornplcment of a t:otrtextfrce language rnust be recursive. Show that. Let L bc a contextfree language. and Sr C 52. Show that the fatrily of recursive lartguages is closed under union and intersection. Why does the a.r' encoding for T\rring rrrachines. L S[ow that the farrrilies of recursively emrrnerable and recursive larrguages are closed under reversal. Choosc a particula. find one clement of the larrguagc Z ir't Theorerrr I1.roirrdeed welldefincd latrguages fbr which one cantrot crrrrstruct a memhcrship algorithrn. 6.Lz . 52 a sct that is not countable.Ilv FoRIrnAt.L is recursive. 2.3.9r be a countable set. show that in fact Sr .n entrrttetation procedure ftrr it. Is the family of recursivcly enumerable languages closed urrder intcrsection? 8. 12. Show that the set of all irrational mrrnbers is not countable' ffi ffi . its complement carrrrot be rer:ursive. Let Lr be recursive and L2 recursively cnurnerable.+ is also recursive? W L5. is it necessarily true that 1. Let . Show that then .hat L is such that there cxists a'firring rnachine that emrmerates the elements of tr in l)roper ordcr.qnc. Isr the family of recursive larrguages closed under concatcuation? 11. Show that this rneans that . Suggest arr etrumeration procedure fbr L+. Wc cotrclude from this that there tr. 16. 1. if a language is not recursively enumerable. Show that . 4.Sr cannot be countable. is recursively enumerable 5.Lr necessarilv rccursivelv ernrrtetable. Prove that the set of all real mrrtrbers is uot countable.282 Chopter I I Ln'lqcuecnsnNrr A(r'ro\'Iann orr A Hrun. Suppose l.Lr is recursively enumerable. 10. ffi 3.
nd these can occur in ilny order. tha. we .1 the production rules were allowed to take any form.\ is not allowtrd a.ble.. we return to the genarnl definition of a grarnrnilr in Chapter 1. rrnrestricted gralrrrnars gcnerate exactly the farnily clf recrrrsively enurnerablc languages. thcrc will he a finite number of srrr:hstrings.ffi$ffi. ur is derived irr orre step.P) is called unrestricted if all the produt:tions U+U. In fact. where z is in (V U ?)+ ancl u is in (y U 7). Since the set of the productions of the grammar is finite.' (:ernlist all ur in .{tilli l'iirj A grammar G : are of the form (V.sthe left side of a prodrrction.11. the flrst is quite straightforward. In an unrestri<:tedgrammar. There is orrly one restriction: . Proof: The grarnnrar irr cffer:t defines a procedurr: frlr enumerating all strings in the language systt:matically. w(. For exarnple. Next. but various rcstrictions were later made to get specilic grirmmar types.n hope to recognizc try mechanical rneansl thir.t is. Anv language getreratcrl by tln unrestricted grirrrrrnirr is recursively enurnerir. essentially no conditions are irrrposedon the productions. As wc will see. unrestricted grrrmmars are nruch rnore powerfirl than restricted forrns like the regular and contr)xtfiee grammars we have strrdied so far.t S+u. wc gert rrnrestricted grammals.T. a.2 UNRESTRTCTED GRaMunRs 283 U n r e s t r i c t eG r s m m d r s d Trl investigate thc connection betweerr recursively enumerable languages and grarnrnars. Any rrurrrber of varia. unrestricted grarrrrnars txlrrespond to the largest faurily of languages so we ca. but the second involvcs n lengthy constructiorr.n be on the left or riglrt. If we take the gcrreral form and itttposc no restrictiorn.ffi.L such tha. We show this in two parts.t is.S. In Definition 1.fi{. iiilffifr.bles and tertrinerls ca.
t can he dcrived itr two steps . see is how to rnake the connection hetw(:t:rr(11.the grammar. (11. tratrsforttts 3 . has thr: following properties: 1 . The grammar generates strings by a welldefined algorithmic process.L (G): L (M)' The idea bchind the constrrrctiorr is relatively simple.5 carr derive qou for all ur e X+.mmars is rrot surprising. we describe how any Tlrring machine can be mimickccl by an uurestricted grarrrmar.a) is possibleif and only if (11. therefbre. so tlte derivations can he dclnc on a T\rring ma. have atr enumeration procedure for the langrrngtl. and want to We are given a Ttrring mat:hineM : (Q.9+r+tr. wltat is more difficult to if and only if (11. namelv.uel Lancueons AND AUToMATA Iist all w itt L tha.r this string into the origirral'u. r This pnrt of the correspondenccbetweett recursively emrmerable langueges and urrrestricted gra.llywirrrt.4) 'Ihis is not hard to do.chine.E. Wherr a string rqly with qy € lr is gerrerated.?. 2 . We can simulate these derivations on a.4) and wha't we rea.f. . Tltring rnachine ancl. (11.d.3) holds. To show the converse. we construct a grarrrrrlarwhich' ftrr all tu satisfying (11. and so on.84 Chopter I I A HtnneRcHy or.n. . (11. To irc:trieve irr broad outline.go. Hence it is recursivelycrrutnerable.3) we will try to arrangerit so tltat the corresponding grarnmar has the property that Qxu tf'I:Qy!. S4u' this.3). Since the computatiorr of the T[rring machine carr be described by the sequerrce instantancous clescriptions of qou I rqfA. but its itnplemeutation bet:tttresttotatioually cumbersotne.' !'on.F) prodnce a grammar G such that .3) holds.
(11. s ) ( 1 1 . which is saved in lhe first indices of the V's.}r d ( q 0 . p € EU {l}.11. How can the grrrmmar remember u if it is rnodified dr.5) can berac:hieved the encoded fbrm) by (in 5 .t" VpiqVoa. while the second is used in the steps in (11.mmtr. 'Io produce two copies of tr and to harrrllrl thc statt: syurllol of M (which eventually has to be removed by the grammar).. For cirr. q e I'. R) productions of M. we inchrdein G VpqVo. If in thc st:t:ttrrrlstc:p. d . p e }JU {tr}. h € I. ( rr .i .bove deriva.fheseprocluctions allow the glarrunar Lo gerreratean encoded version of any string q0urwitlr a. Tlx: first step irr (11. q e f. the grammar erases everything except the saved'ur.5) Tlrc thirrl step in the a.l (qu. When a fitral corrliguriltiorr is tlrrtcred.J . Thc first is saved..1. 1 0 ) . L ) of M. thc statc: r. the gratrrrnar tnust thert get rid of everyLhingexcept ur.6) (11.tr.n arbitra.i6 encodesa and b as well as b.tionis the troublesome one.lla e E. we prrt into the gra.VrrSlSYnll?l 7' TV. M t:tttr:rs a firral stattl.yy 1w. " ): ( q j .4).rolvooo. we include productions WiA + et ( 11 .ncl while V. . Therefore. WdVrtq.'.r W. lbr the second sttp. we introduce variables Vo6 ir.)e for all a. and all rl such that qa e Q.ndVoi6f'or all a € XU {n}. (i1. : kt. '..ry number of leading and trailing blarrks. The variable ydb ()rr(j()(l()tr tlrt'two syrnbols rr.2 UrunnsrRrcrED Gnalruans 285 Tlrt: rxlrnlllete $equen(ie derivations is then of $ 4 qorrl xq. for every ei E F.Vprt . for irll n.7) for a. for cirdr trarrsitirlrr .rringthe secondstep? We solve this Iw enurding strings so that the coded version originally has two copies of u.
lqprrrn. 1.:{a. Carefully check each step in the exa. a(. To rnirkc things work irr this casc. wc rmrst first rrsc (11. Tlris rttachirre acccptsL(uu*).a. )J : {rr. Ttl tklrivc this strirrg with (J.. The extraneous blanks are removed at the erd by (11. We need one more specrial E . T}ris createst}re first terrninal in the string. r) ..6) rrnrl (11. Clonsirler rrow tlte tlttrnuutittiorr esaal aesal aaesl F aqla[ (11.pe rlr:t:rrpicd tly thc input 'ur. I'he Iast sentential fornr is the starting point for the part of the derivatiorr that mimics the cornputation of the'I\rring machine. .6) and (11.mpleto see what the variorrs prodrrctions do antl whv thclv itrt: rrecrlcd. d.b} . 1 2 ) prodrr:tion lre f.. .13). It contains the original . The following example illustrates this complicated construction..14) which acct:1lts tlrt: strirrg na. with qr} I : {qo.\.qo.tr}. c€ EU{tr}. R) d (qo.(q11.o) .7) to get the appropriate starting string.ri) (11 .rt of the ta.kescare of the ca.. tr) !.286 Chopter I 1 A HInn. F) uea Turingmachine n.nrlal Lnr'{cuncos awn Auttlltane f'rrrnll a € X U {tr}.!. CQ.tncIIY oF Fon.by a cWt Wtc for all a. (11. representing all the tape region used.x. S =+ 5'Vll =+ TVaa +'lV""Vaa 4 VoooVooVll.se when M moves outside thnt pa. : (gr. t' : {qt} ' itncl d (so. wtl first us(rnrlc$ of the form (11.13) This last production ta. which tltcrr causes rewritirrgirr the rest. f .7) to generate E. (11. b e f.b. lliHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH mt M : (Q.
10)to (11. By an induction on ttre rmmberof steps.. Thc cxrnstnrction describedin (11.srV11. whercc (z) denotes encoding a string according the givenconventhe of to tion.6) to (11. Next. and of Vo*l/tr0tr WruVaa corning frtlur (11. then e ( r )+ e ( y ) . The construction described suarantees that if rla.uVaa VouVoooVVa * )VooVooVlon + VooV. Then the next steps in the derivatiorr are V. aOa[. which is equivalent to the sequenceof irrstarrtanrxlrsdesr:riptionsin (11. 5 . always rernembering the itrititrl input.8).13) are used itr tlte last steps Vu"VtuVaa 4 VooaVag + V""al + aal + aa.. a1aE. there exists an unrestricted grammar G.nces (11. Finallv.0. The sequenceof the other indices is 0aan. and VoooV.9). aa0!. The sequenceof first indices remains the same...L : L(G). whiclt arc spcr:ifir:insta.13) is the basis of the proof of thc following result.14).z * Voolltr0tr.') " (g) . T For every recursively enurnerable languagc. we apply VoooVoo WoWoo.V.. such that . (11.L.2 UNRnsTRrcTED GRarrrlrans 287 input aaE in the sequellce tlf first indices and the initial instantarreous description esaal in the remaining irrdiccs.I1. Proof.we cirrrthen showthat e (So?.
6 . unre withA€Y. cise. rather tharr a single variable.Ncuacns if and only if q1uri Y' Wc also rnust show tha. 5 . What difficulties woultl arise if we allowed thc empty string as the left sicle of a production in an unrestrictcd grammar? 3 .a$rb bB . In Example 11.ll of whose protluctioris have the fortn 'u+at with z. L.bbbB aStb ' clerive? ffi B .gesa.A aa 2 . m .7. with the family of rec:rrrsivcly 1. or .1. Formalize this concept. thcn find an unrestrictetl grarnrtrar for it using the coustruction in llheorem 11.ltnl.288 Chopter I I Arttr Atlr'ovane A Hrnnnncnv or Fon.hlishwhat we set out to clo. Consider a variation oII gl'alnmars in which the starting point for arry derivation can be a finite set of strings. prove that the constructerJ grarrlrrrar cannot generate anv sentence with a b in it. What language does the utrrestrictcd grammar Jj + 5'rB '5r .ils. a. u € (Y U ?)+ and lzl < lul. Thev show that wittr utrresl.ssor:iirtt:rl etrumerablelauguages.4 +.releft irs iilr cxerconfiguration. Give the details of the proof of Theorern 11. I These two theorems esta. ffi 4 .ricted gra'mma. Slxrw that fbr everv unrestrictcd grammar there exists an etpivalent stricted grammarr a. The cleta. t .rs telate to the unrestricted grarrrfirars we havc usccl here. Construct a Turirrg rrrachineftrr tr (01 (01)).\.rsis idt:rrtitlirl the family of langua.7. then investigate how such gratnma. Give a derivation fbr 0101 using the resulting gramnrar.q.t we can gcrrt:rate every possible sta'rting t:ttnfiguratiorr and that tu is properly rerxlrrstnrctt'd if and only if M enters ir lirrirl whir:h arc rrot too difficult.
there is an equivalent grammar of the other typc.rup Larucuacns 289 8. a E (V U T)+ ancl l z l5 l s l .\tthitffi tl. Show that this alternate definition is llasir:ally the sarne as the one we use.ffi.rir:ty of "srlrncwtrilt restricted" grammitr$ t:tr.nbe defined.'I'.ttention.ype..nguer.nd a€(vu7)"..'s rr. The diffctcncc is that hcrc thc lcft side rnust have at least orre variable. unrestricted grarrlrnar$r ir grt:at vir.q.v(vu7). this alternate definitiorr..3. the productions of In atr utrrestrictetl Hrarnrnar are rerruired to be of the form T+U.15) .lirrc:itrtrourrdcxlirutornata.sour Dcfinition 1.mmarsancl the general.l' A grammar G: (V. among the ones thtlt rlo.I1. in the sensc that for cvcrv grammar of one t.5.S e n s i t i v e o m or s Gr m ond Longuoges Bctween the restricted. Not all c:irscsyit:ltl irrttlrcstirrg rrnultu. C o n t e x t .fi .ted with a.3 Corur'r:x'rSnrqsrrrvn Clnauuans . where ..ge. (il.ssrlr:ia. 9.r$ ha.5.ve relcc'ived considerahle a.P) tions are of the forrn is said to Lrecontextsensitive if all produc where tr. These grarrrrrrirrs gtlrrcrilte lrr. N$ffi$fl N. Somc authors give a definitiot of urrtestrictetl grarnrnars that is not qrrite the same a. a. the cotttcxtst:rrsitivtl grilmma. whic:h we irrtrorhrt:edirr Ser<:tion 10. Show that the conclusion of Hxercise 7 still holds if wc add the further conditiorrs zl < 2 and l"l < 2. contextfiee gra. restricted clirrs of T\rring macltirres.d€(vu").'$$fr.
and we will rely entirely on Dtllirritiorr 11.uacns eNn Aurol"tn'r'A or Chopter I I A Hrnn. s u c ht h a t . we can claim that thc frrmily of corrtextfree languages is a subset of the family of contextsensitivc languages.contextsensitivegrammar$ are a^ssociated a latrgua.tel This definition shows clearly one asper:t of this type of grammar. every contr)xtfrrxl larrguage without . By inclrrding thc ernpty strirtg itr the definition of a contextsensitive langrtage (but not in the grammar).4. Salomaa 1973) that all such grarlmars can be rewrittt:rr irr a rrormal form in which all productions are of the form xAY + YPY.gefamily with the sarnc rrarrle.\ can be generated by a special caseof a contextsensitivc grarnrrlar. so that a contextsensitive gra.4 implies that r + A is not allowerl. both of which satisfy the conditions of Definition 11. in the sense that thc lcngth of successivesentential forms can never decrea^sc. (G)U {A}.L : L ( G ) o r L : . if A language . say by otre in Chomsky or Greibach normal ftrrrn. This is equivalent to saying that tlre prodrrr:tion Atu cirn be applied only in the situation where 4 occur$ irr a context of ihe string z on the Ieft ancl the string 3yon tlrc riglrt.wclHy FoRrr. Definiiion 11. is said to be contcrxtst:rrsitivc thcre exists a contextsensitive g r i r m r n ? r(rJ .290 Lauc. thc forrn itself is of little interest to us here. but it cau tre shown (see. In this definition. . it is noncontracting.It is less obvious why such grammar$ should be called rxlrrtcxtserrsitive.I. wo' reirrtroduce the empty strittg.mmar can never gernr:rittc a lirnguagtl containitrg the empty string. While we use the terminology arising from this particular interpretation. Yet. for cxarrrple. qnd LineorBoundedAutomotq ContextSensilive Longuoges witlt As thc terrrrirrologysuggests.4.
henthere exists a corrtcxtsc:nsitivt: grammar fbr r{A}. + aaabbbccc. At: + fi6"". Tlrc pror:essis very similar to the way one might program a Thring machine to accept tlrc lir.4 is created orr thc luft.ge. + aRbbcc + aaAbbcc+ aubAbt:c + aabbAcc+ aabbBbtcc + aab Bbbccc+ aaB bbbccc. . the corresllorxlirrg T\rring machine ha. Exarlple 11.bles A and B as rnessengers. A ftlw oxa.One sut:h gramma. we see that the farrrily of rxrrrtcxtfree ltrnguages is a proper subset of the farnily of contextsensitive larrguagcs.tttfble t. Iinear bounded autorna. .r is S . Hm6.2 I The language L : {artfirlsn: n > 1} is tr.ge tr. then finding atr equivalerrt grarnrnar frrr it.q. We slrow this by exhibiting a contextsensitive grirrrrilrirr fbr thel la.\. Ab . Frlr every crrntextsensitive language . where it creates another b arrd r:.rrgutr.rnpleswill show that.ngua.3 ColrrlxrSnNsrrrvr Gnntttrrtns. If L is contexLsensilive.bove example is not contextfree. We can see how this works by looking at a derivation of o:lblcil. I Sirrrxl tlxl lir. It therr sends the messenger B back to the Ieft in order to create the corresporrdinga.utomaton M such that tr : L(M\. S + aAbc+ abAc+ abBbct. contextsensitive language.In.tthclaAbc:.11. whenever the language is contextserrsitive.ton. trirvels to the right to the first c. Often the solution is rnost cirsily obtained by starting with a Ttrring machine progra. The linear bounded automaton will have .rqu LeNcuecns 291 . aB +aalaaA. Proof. there exists sorne lirx:irr brlrnded a.2 also shows that it is not an easy rnatter to find a contextstlrrsitivc grirmmerr even for relatively simple examples. We show that derivations in t}ris grarrurrarcan be sinrulatecl trv a.bA. bB +Rb. Arr .t.L not including .spredictable space requirementsl in particular.ngrragein the a.lirrear hounded automaton. it carr trclvirlwrxl as a. Ihe solutiorr clIectively rrses the varia.
$sociated sensitive gramrnar G. All productions generated in Theorcm ll. the computation described in Theorctn 11. and look a. *.derivation of ro 5+11 =+r?'+"'+rn+ut' We can a. A key point of this argumetrt is that no possible sentential fbrm can have lcngth greater than lul.6 can be carried out without using space except tha. Every r:rlrrtext*setrsitivelattguage L is recursive. cotnpleting the argument. The crux of orrr argrrment is that the numbcr of steps in arry tlerivation is a bounded firnction of kpl.7 are rroncontractitrg except (11.13). r Between Recursive Contextsensitive Relotion ond [onguoges Thrxrrr:m 11. one containing the input string ur. I If a language tr is acc:eptcrl by sorne litrear bounded automaton M.L with an tr. thirt is. which is not the case here. the othrtr r:clrrtainitrgthe sentential forms derived r. But this production can be omitted.i for all i t' j. Another point to notice is that a.i t' x. tr*4. nondeterministic. This is rrct:cs$aryitt the argument.rsing G. Theorem 11.t origirrrrlly occupied by ur.292 Chopter I I A HreRRRcHy FoRMAI Lnmcuacns eno AurouAlA oF two tracks. since we can claim that the cclrrect productiorr carr always be guessed and that no unprodrtctive alterrnativcs havc to be pursued.t a. by clefinitiotr.ssumewithout any loss of generality that nll serrtcrrtial forrns in a single derivation are diff'erent. it can be done bv a linear bounded automaton.t in Thcorern 11. Proof: The coustruction here is similar to tha.ineclhy thc corrstructiorr without this umrecessary produr:tion is nttncorttractittg.nguage. that is.10 follows easily frorn tlfs.lincar bounded automatotr is. It is necessaryorrly when the Thring machine moves outside the bounds of the original irrput. The grammar obtir. . contextProof: Consider the contextsensitive la. then thcrc r:xists a contextsensitive grammar that generates tr.9 tclls us that cvt:ry cotttext*sensitivelatrguageis accepted by somtr T\rrirrg rnirr:hitre:arrtl is tlterefore recursively enunerable. We know that l*il5 l"i+rl.7. Therefore.
ritr.Vt.. We check all derivations of lcrrgth up to l'ulrn (l'rrrl).). otherwise it is not. with trl : nL(lrl) u bor. Proof: Conuider tlrtr set of all contextsensitive grammars en 7: {a.} . l .3 CoNTExrSnmsrrrvn GnavveRs ANDLANcuAcr. . definesa contextsensitive {u.)}.Vz.t most ont: corrtextsensitive grarrrrrrarcorresponding to it. any contextsernsitive grarntrtar call be represented rrniqucly by a string f'rom I ((0110).vtr. Furthermore. lz(.. i< l r i 1 . + Jlrrr. call tho grirrnrrritrGi. # L (G. The cxrly tlfttg we need to add is that there exist some rn.11.given length.. A given string ui rrlay ntlt define a contextsensitive gramrnar.I r n .L. If any of them give ur. 1}+. dcperrdirrg otily otr G atrd 'ur. I There exists a recursive langurr.urded function lV u 7l nnd lz. there is a. Since the set of productions of G is finitc. Next.b}. the representatiorr is itrvertible itr the sense that. of follows becausethe finiteness of lV U ?l implies that thcrc are only a linite number of strings of a.the length of a derivation of w E L i s a t m o s t l r u l r n( l r r r l ) .. there are only a finite number of these. l This for all j.grlthat is not contextsensitive. so we ca'nwrite strings in tlre order uJltlrzt etc. Evcry contextsensitive grammar is completely spet:ifitrl by its productionsl we can think of them as written au a single strirrg l U 1+ AtlfZ + UZl . then u € L. such that l z . if it does. we defiue a' la'nguageL tly L: grammar (J4 alrd *. Thcrcfore:.'1.blc f'orm V : {Vo.ls 293 because G is noncontracting. To this string wtl now apply the hotnomorphisnt ft'(a) : 616. h (b): o1zo. set of the We t:tlrr r$() a convention in which each grammar has tr.. Let us introduce a proper orcleringon {0. givcrr arry such string.i: trr4 .):s1+6' h'(V):01i+50' Thus. h () : 61:t6. ' .. This observation giveu rrs imrncdiately a mernbership algorithrn for .
re la... but that there a. Because of the esserrtial equivalence of linear bounded arrtorrrattr tr. 1.nthen a.lgorithm. there would exist sornrr rl. * 1 . beirrg gr:nerated by contextfree grarnmars. As various examples show. suclr that L : L (Gi) We ca.rtfir'txrt4rrt ] l. (ur)} 3. and pushdown autornilta ilnd contextfree la. they arc ir proper suhset.L.xtsensitive.ndcontextsensitive languages orr one harrd.L is ttot mrntoxtsr:nsitive.. t The result in Theorern 11.nguageson the othcr.t. (u) {an+rbnr" 1:ri}1}. lf *" .testhat linear bounded automata are indeed less powerful than TLrrirrg urachines. art: ir. But L: L (Gi).llrshdown automaton is also accepted by stlrne linea.grammar. Show that the farnily of contextsensitivc languages is closed under reversal.t) is rer:ursive. so wtr Irave a corrtrrrrlir:tion.bekrrrgsto.s.L.w €{a. check it to scc if it defines a contextsensitive we grammar G.if we assurnethat.'incl contextsensitive granlmars for the following {anguages. If the strirrg rlclrlsdcfine a.L (Gj). then by definition wi € L and we havr: irnother contradiction. Tcrseethis.. tlrcrr by clefinitionuj is not in. we construct a membership a. ( b ) Z . We must therefore conclude that .ui # L(Gi). then u.L is rrtlt rxlrrto.: rn {e. Contextfree languages. Show that thc family of corrtextserrsitive languages is closcd under union. z " :n . ( u . I. If it were. Conversely. : { t t " h " a . Larucuncus ANDAuroMNr'A. ) : n . then ruaf L.r bouncled automaton.and we can use the membership algorithm of Theorem 11.11 indica.srrhset of the contextsensitive languages. is well defined and is in fact recursive. since they accept only a proper subset of the recursive languagt:s. then L((. If it is not.b}+} ffi * 2 .')< n. 4.rn. Find contextserrsitive fJrammars for the following languages.r*L tlrat trri ( L(Gj). ( t u ) } (b) t : {w t no (r) : rro (u. wu see that anv Ianguage accepterl by ir. (a) I: { t n : n * ( ' u r ): 7 r . ffi . > 1] {ww. > 1 } (c) I: (d)l. Elut . Given ?o?. If not.nguagesaccepted by sorntl Iinear bounded automata for which there are no pushdown automata.10to find orrt if rui € I'(Gt).254 Chopter I I A HtnnencHv or FoRvel.skif uri is in . It f'ollows from the same resrrlt that linear bounded automata are rrrore powr)rfirl than pushdown automata.
3) exhibits the relalionship clearly.nguages (Zcs). This original tenninology lns persisted and one firrds frequent refererrcesto it.ra. type ? consists of the As contextfree la.miliesr:rr.3 and 11.3 7. Figure 11. Figurc I I. Type 0 languages are those generated by unrestricted gramurars. provided an initial classification into four language types. tho rxlrrtcxtfieer (.ndtheir pltx:c:irr Figrtrc 11. the corrtextsensitive la.' .4 'I'HE CHor"tsxv HIURARC:HY 295 5 .11. a founder of fonnal language theory.ge he sturlirxl.'p).10.rrivea. the recursively enumerable languages. Frrr m.nc).3 shows the original Chomsky hierarchy. Type I consists of the contextsensitive languages. altirough thcir relatiorrships do trot always ltave Lite neatly trestecl structure of Figures 11. as a function of lrrrl and lv u71.4. We have also met several other language families that can be fitted into this picture.4.t the extended hierarchy shown in Figurc 11. (Ln.n defined tr. each larrguage farnily of type i is a proper sulrset of the family of type i .ges (I'nac).hle languages (Lnu).ngr. b}+} ffi E The Hierorchy {ffiffiffiffiffiChomsky We have now encountered a nurnber of language families. and recursive la.ngua. type 0 [o type 3. in Theorem 11.t11t. fa.nguages. Including the families of deterministic contextfree languages(L nc p) . show that therc cxists a contcxtfrcc grammar for the language 7 : {uul i 1t. that is.nguages and type 3 consists of the regular la.rmera. lir. A diagrarrr (Figure 11.ngrrages and the regrrlar lilngrrargers Orre way of exhibitirrg the relatiornhip between these farnilies is by the Chomsky hierarchy. Noarn Chorrrsky. the relationships are not cornpletely unclerstood. we have seen. but the numeric types are actually different rlarnes for the language families we have stuclied. give explicit bounds for m. In sorne instances. we a. Without explicitly constructing it. arrlong them the recursively enr.4 Other la.{a.L6.1.
This indicates that the relationship between regular. linear.. the language 7.5 . but not linear.Lnmcuacns enn AuroMArA Figure 11.4 ll\]i$'$.ilWil$\$ language we havepreviously introd'cedthe contextfree L: { w : n " ( w ) : n 6( u ) } and shown that it is deterministic.296 Chopfer I I A HrpRaRcuyol FoRraer. f l'igrrre 11. and rrondeterministic contextfree languagesis as shown in Figure 11. On the other hand.5.: {a"bn} U {a'b?'} is linear. but not deterministic. determirristic corrtextfree.
11.re more powerfirl thtln lirrcar botutded autouata.3) of languages that are Iirrear but rrot deterministic contextfrcc. I . propcr subset of the contextsensitivela. we established a hierarchy of languages and classified autornata.ter. At this time. These in turn are more powerful tha. We can now ask the question we asked in r:onnection with other autorrrata: What role does nondeterminism plav here'l Unftlrtrrrratclly.5.4 are indeed propcr ones.ngtrages. deterministic contextfree but not linear.there is tro easy answer.At thc bottorn of the hierarchy are finite accepters.rsxv Hlpn. Collect examples given irr this book that dernotrstratc that all the sub. In doing so. by their power ir^slanguage accepters.n pushdown inrtomata.3) of larrguagesthat are . $ec:tiorr10. 3 . We introduced thu trrrx:cl1ltof a deterministic linear boundecl automaton in Exercise 8.set relatiorrs rlepit:tetl irr Figure 11.mple I1. Firrtl two exalrrples (excluding thc one in Exa. it is not known whether thc tarnily of lattguages is accepted by deterministic linea.r bounded automrr.with which we begarr our study. .crrv 257 There is still a. ir. To slrmmarizc.n unresolved issue.4 Tsr: Clsorr.an. Ttrring machines a. 2 Firxl two exarrrples(excluding the onc in Example 11. w() have explort:d thcl rclationships betweeu several langurlgclfarnilies arrd their associatedautomata.
.
w() now look at what they cannot do. This question is clearly of prrr. then by definitiorr thurc is no membership algoritlrrn ftrr it. If this were a.t we mean when we say that sorrxlthing cannot be done by a.loccasions that there coukl rrot uxist a.ted(but nol yet proved) that we there exists no algoritlmr to detc:rrrirut whether a contextfree grammar is rrnambiguous. if a langurrge is nonrecursive.mrning languages. we hitvc c:lnimed on severa.lked a.crtical significancein the study of progra.ges have little pra.lgorithmsfor the solution of ccrtain problems.ll there was to this issuc. hirve sta.meabout quite sirnply. But the problem goesdeeper. it would not be very interestingl norrrec:ursive: lirrrgrur. Thrirrg urar:hine. Although Thring's tlxnis letadsus to believe that thcre are f'ew limitations to the power of a T\rring ma.L i mt s o f A l g o r i t h i c i m p Com ulqtion rrvirrg trr.nv a.chine. We first define the concept of decidability and computability to pin rlown wha. We then look at several classical prcltrk:msof this tvpe. FYornthis follow a nuuber of rrllatrxl prohlems fbr T\rring machines and recursively 299 . Sorneof tlxr results ca.ctical value.bout what Thring nrachincs r:iur rlo. For r:xa"rnplc. alrrong thern tlx: wellknown halting problern for Thring rnachirres. Now we rnake more explicit what we mean by this claim.
rt which are known to lltl rrttsolvableby atry computer.r.t its rlomairr is. A firnctirin is rrrx:rlrnputableif no such 'I\rring machine exists. WInt is more plrter that wc carr now cottstruct or even pla.brrt cleir. We see fron this that. when we cla. the set of all contextfree gramma. with irrt apparetrt possibility of an a'lgorithmir: solution. l'or example.t computcs t]re value tlf / frlr all arguments in its doma." 'I'he probletn is to it is fa. unfortunir. certrr. sitrlrkl "yr)$" r)r o'rr).rs. we consider the sta. while in the second case one tha. thc:rc are Ito algorithms. we rnust be clear on wira. the langua.in.t we do not know what the correct' . (G) is rr." lrr this case. becausethis ma. This may seenrlike a fa. each of which must he either trtrtt or false. a rkx:idablc. Them miry be a Turitrg rnachine that can compute . ftrrrt:tiorr rrs cortrputable or not cornputable. We say thtlt a prublc:rnis decidable if there exists a.mwe will understir.nda set of relatecl staterrents.rlywc rnust have otte or tlte other.lse.ya.city of arty cotnforerstx:. decide whelthrlrthrl stirtcrrtetrtis true for any G we are given. Specifically.bltl there exisls a Tlrring ma. Hcrc we find quite a few importa.f rlrr pnrt of its dorrtairr.mbigrrorrs. After this. filr Whetr we state decidability or: undecida.lways answers "firlse" is appropriate.Tlrrirrg rnac]rine t]ral computes the frrnction on ther wtxlltl of its dotttaitr.nt ltrobletns for which. T S o m eP r o b l e m s h o t C o n n o fB e S o l v e db y Turing ochines M The argurnent that the power of met:hilnir:irl txrrnputations is limited is not surprising. wrr stated that a function / on a. but we call the function comprrtable only if there is a. wc look at sotne questions relnting to contextfree langua.chinetha.geI. there is iln rrnrk:rlyirrgtlorrrain.rsibly is that thcrc arc questiotrs that can be irrterestirrg to computer scientists clearly and simply stated.tement "Fer il t:ontt:xtfrtxl graIIIInaI' Fur sttrtreG this is true. sirrce the allswer is either single instance of a problem is ir. ond Computobility Decidobility In Definitirlrr 9. By a proble. fhe fact tha.ges. Again. In the first cirtrcra TLrrirrg rnachitre lhat always answers gives ttxl rxlrr(x:t irr$wer.coRt'ruMIC CoMpurATIoN cnumerable languages. wc rntrst always The probkrrow whal the domain is.Tirring macirine that gives tlxr txrrrcct irnrJw()r t:very statetnetrt in the domain of the problem. lem rnay be decidable on some rlomairr llrt rrclt ort atrother.lwirys "ttue" true or fa.tr:ly. Intuitively wc krrlw that many vague and speculative qutrstiorrs require special insiglrt and reasotritrg well heyond the captr. Our conceru her:e will be the sorncwhat sirnplilicd setting where the result of a comlrutatiotr is a.300 Chopter l2 Lrurr$ oH Ar.cetiollrtrn$w()rr tnrt it ernphasizes atr ittrporlatrt point.ssify a.bility results.indornitirr is saicl if to be c:omputrr. for others G. we talk about a problem being decidable or undecidable.t a. br.4.flectthe:c:onrlhrsiotr.lse.
br:(:ir. we arc looking for a single T\rring marchinethat. the prol.cHtvt. )_1. Here q.rrse there is no limit orr the length of l. r'Rfi$.ur.If M en[ers an irrfinite loop.8Nllili Let up1 be ir. A solution of the halting problt:rn is a T\rring machine H.s 301 alnwcr is makes tro differerrcc.tion.we ma.tthe same tirne givc: rrs a.Ni'. Thring machine M irrrd rr.f.string that describesa.fiinnrnmtmW. Tlrc bestknown of thr:sr:is the Ttrring rnachirrc: halting problem.yr.t can detennine thc r. What we need is an algoritlrrn tha.or simply (M.what rna.t if M applirxl to zr does not halt.4.tr' applied Lo .c:omputation that t:vcntrrerlly halts? Using irrr tr.ninput 'ur.he cornltrrta. . then no matter how long we wait. TheTuring Mochine HoltingProblem Wt: begin with sotne problurns tha.ssuggestedirr Secltirln10.:11'w if .. But as wc now show. n.l.rlernis: given the description of a.d. we statc:rlsrlrnewhat looselv above.12.4. given thu rlt:scription of an arbitrirry M and tl. 'fhe dorrrairrof this problem is to bc tir.nv ?rA.t rr. F). atrcl let.ll ru.'u bc a strirrg in M's alphabet. we rrskwhelther M applied to .ven ey TuruNc.nclru are encoded as a string of 0's irrrrl l'$.t we mean by tlu: ha.T\rring machine fuI : (Q. A. We cannot firrrl the iruiwer by simulating tlrc a.vcry long cornputation.bhreviated way of talkirrg aborrt the problem. It rnrry simply be a caseof a. are both fitral states of ff .u). wlfch firr a..nclthe input. tha[ is.lting problem.1 Solrn Pn. will predict wltctlNlr or not the cornputatiori of M applied to to rvill hrrlt. halts or docs not iralt.rand ?r.EMS THAT Carvruor Bu Sol. For subsequent discussirirr. no such algorithm exists.c:tionof M on tu.otl.l configuration {6ro.c.whcn startecl in Lhe iritia.rr rrl u'. M. anrl qo'tlt It) i. w() (iirrrrrever be sure Lhat M is irr ftrct in a loop. perform rr. llcrforms the computatiorr i *rqo*r. precise idea of wlra. say by perfbrming it ol a urrivt:rsill Thring machine. Q(Ju.urhalts.q a.ke a specific tlcfinition of whal. Sirrrply stated.l signific:ance arrrl thir.rlrrect answer for any M arrrl ur by perfbrming sorne arrirlysison the machine's clesclilltion a. arrrl g".it is convenient to hilvr: ir.t have sornr: historica.ttersis that tlxlrc exists some'Ihring machinc that cloesgive the corr(x:t re$ponse. a. We will imsllme tha. r1o. for this rcilrJorr.t'ur.nd a. starting point frlr clevelopinglater results.kenas the set of all l\ring machirrcs ir.does .
TIte intetrt of this dia.t n lltQ. the action of 11' is descrihed by qilt)Nt'tu irr. s&y.tl w^ill htr.grarrtis to indicate that. Proof.nsitiorrs tretween state qo atrd tlrtl rrtlw states q* and q6 are to be rnade.!!2. and (1.nbe visualized }ry a block ditr. With thc a. the rnodified machine fll will t:rrter att infinite loop. We imsurtte the contrary.t.1.302 Chopter l2 CoururNt'lol Lrlurs or. Thc wtw this is done is stra.t.chine ff . The: rcxpiretnent is then thtr.T\rring rnitt:hirreiI' with the strrtctrrrtl showrr irr Figure 12..1. Formirllv. Figure 12..n algorithtn. f if M appliecl to tl does mrt halt. it will eventually ha. if M applied io 'u. The siluation ca.halts. givctr any wMwl the T\rrirrg machine.tIc Thurc cloes not exist any Ttrrirrg rnachiue Fl that bchirves as required by Definition 12..ddeclstates itr Figure 12. Comparing iI and II' we see tha. and qDwMu.c+orrtrtIr..iglrtforward.lt in ttnt: of two correspottding firrrrl states. As reqrrired by Definitiotr 12. namttlv. or 4r.. thrrt solves the halting protrlcrn. irr situations where fI retit(lhtl${rr alrd iralts. wermodifu fI to produce a.rn like Figure 12. regardlcssof the tape syntbol. if M applied to ur halts.*. {ri or qr.gra. Next.2 wc warrt to colrvey that the trtt. in srrt:hrl way that lhe tape remains unchanged.Ar.1.lt in state q. that there exists a. The input to ff will be tlte string u. and consequently sorne I\rring ma. qow u) i rt. we want ff to operate according to thc following rules: Qowtwuisngyn2.14ru. The haltirrg problem is therefbrc urrdecidable.'!!2.2. We achieve this trv askittg that 11 ha.U't M if M applied to ru dot:s rrot lrrrJt..1 .lt with either a yes or no arswer. if M is startsl irr state {s with input rit14tr.
also can be used as input string. it behaves exactly like I/'.l says.[y'.F is applied to fi.1.bove.. After that. 1}*. if M applild to wp1 does not halt. "I1 had to start and end in very specific configurations. ending in its^irritial state qs.Az. and qofr n AtenUz. ri. if fr applied to fi halts. in additiorr to being the description of .q. This is clearlynorsense. It does not preclude solving the haltirrg problem for specific cases. Tlfs string. I One rnay ohject to Definition 12. if JI. and qowm ifi qouMlilM f p Afl. and that essentially the same reasoning could be used with a"rryother starting and ending configurations. since we required that./ applied to w71halts. must be false. Ff. This new rnachine takes as input ur14 and copies it. Then the ar:tion of fI is such that qow*t l ii Q11wx4u)14 *. not hard to see tlnt these somewhat arbitrarily chosen conditions play only a minor role in the argument. It is important to keep irr mind what Theorem 12. but this does not affect the conclusion.12. to solve the halting problem. and hence the assumption of the decidability of the halting problem. contradiction to not The tells us that our assumption of the existence of 11.identifyirrg M with Il. i if fr applied fi does halt. say. It is. Flom the ir.2 FYom f/'we construct arrother Ttrring machine f/.rCaruruorBp Solvno ny TuRrrrrcMacnrNns 803 Figure 12.often we can tell by an analysis of M and ur whether or rrot the T[rring machine will halt. however.. Now fI is a Ttrring machine. so it has a description in {0. We have tied the problem to a specific definition for the sake of the discussion. we get * gsfi l6 oo. What .1 SoMEPRoBLEMS Tg. We can therefore legitimately ask what would happen if .
Problem Another lo Reducing Undecidqble One The above argument. If H says "ycs. Let 11 be the T\rring machine that 'We cotrstruct from this the following proceclure. X. making L recursive. thtlre is no algorithrtr thrr. The argumerrts for proving Theorem 12.If . both being a version of diagonalization. But we: already know that there are recrrrsively enumerable lattguages that Arc rtot recursive.'no. The stateentry problern is as follows. d. The1." thett by definition rrr is not in . from the decidabilitv of B. The only different:c is that i1 the haltirrg problem we do trot distinguish between haltirrg in a final and rrotrfinal state. f .L. and let M be a Ttrring mac:hine that accepts . w € E+.mplesto illustrattr this idea.F1says 2. thrrt the halting problem is undecidable.1 vrcre given becarrsc they are classical anrl of historical interest. t The sitrplicity with whit:h the halting problerm can be obtaincd from Theorem 11.t catr make a correct decision for all toy and ur. the halting protrlcrn is undecidable.5 (via Theorem 11.304 Chopter  2 LIvIrs oF AlcloRl'rHr. so it will eventually tttll us whether tu is irr . then (lvety recursively crrurnerable language would be recursivc.[.L be a recursivelv errurlerable languageon E. Consequentlv. illustrates the very important teN:lrrrique reductiorr' We say that problem A is reduced to a problem B if the decidability of A follows a. whereas in the membership problcrrr we do." then apply M to u. This problem is urrdecidable. Apply H to w14tt.roclosely related. F) and arry q € Q. The proofs of Theorems 11. The corx:lusion of the theorem is actua. But M rnust halt.L or not' This constitrrtcs a trtembership algorithm.1a. Let us do a few exa. Giverr atry Ttrring machine M : whether or not the state (8. if we ktrow that A is urxlecidable.lly inrplied irr previous results as the following argument shows' If the haltirrg problem werq tlecidable. we carr conclude that B is also undecidable. q is ever entered .L.3) and J. Iet . The contradiction implies that I1 cannot exist. that is.5 is a consequenceof the fact that the halting problettt and the memtrtlrship problem ftrr recursively enumerable languages artl rrearly identical.tIcCotueurrrlom the therlrem says is thrrt this caunot always be done. '. 40. <lecide wherr M is applied to tr. Proof: To seethis. 1. t:ttrrrrecting the halting problem to thc ntembership of probletn.2. solves the halting problem.
The constnrc:tion in Example 12.. Given a Ttrring urachirrc M.JrJume exists.. But this is impossible. an algorithrn for the blarrktape haltirrg problem can be converted into an algorithm for the haltirrg problem.. the starteg is entered. We apply the stateentry algorithn eto ('rt.will halt on a blank tape if arrd only if M halts on ru. determine whether or not M halts if started with a blank tape. I The construr:tion in the argurnents of thesc two exelmples illustrates an approach cornurorrirr cstnblishing undecidability results.nswers yes. The conclusiorr tr:lls us whether M applied to tl will halt. Putting the two together yields an algorithm for the haltirrg problem. the stateentry problcrn must also be undecidable.rr) is rmrlefined. Tltus.u. ru). and we can rruncludethat A cannot exist.use some d(qi. . we use the algorithm for solving the blanktape halting problem. that is.u. Given any (M.. writcs ur on it. We can do this simply by looking at tlxl transition function d of M. A block diagram often helps us visualize the process.' M and some tu. After that. Because the halting problem is undecidable. we first construct Mr. If M halts. To get fr. We first rxtnstruct from M a new rnirr:hine Mthat starts with a blank tape. which wrr a*. tlrcrr (M.rrr) ha. the same must be true for the blarrktape halting problem..R ) . then (M.l SoME PRoBLEtMsTHnr Calrlror Bn Solvnt ey Tunrlrc M. If A says no. To show how this rtxhrr:tion is accomplished.il The blanktape halting problem is anotlrcr problem to which the halting problem catr be reducerl. we first modify M to get M irr urrcha way that M halts irr state q if and only if M halts.*) does not halt. Sirrcc the latter is known to be urrdercidable.regiven sornt. it cloes$o beca.12. Clearly M. .lts. Next. configuratiorr qoil).. Since this can be done for any M and ur. For exanlple. thon apply the blanktape haltirrg problern algorithm to it. that diagrarn. We could then uscrit to solve the halting prohlem.).11. assumc tha. where q is a Iinal state. Supposenow that the blanktapo halting problem were decidablt:.q.2 is summarized in Figure 12. then positions itsclf in ir. we first us(] an rrlgorithm that In transforms (M.teentry problem is decidable gives us an algorithm fbr the halting protrlcur. a . the assumption that tlrt'sta.1. we chrr.nge every such undefined d to . suppose that we have an algorithm A thrrt solvesthe stateentry problem. l ( q n a ) : ( q .crrrlus 305 Trr reduce the halting problem to the stateentry problem. This is undecidable.t we a. given a. srrch an algorithm clearly exists. If A rr. M.ny M and .w) irtto M. T Bxfiilnpls t fi. acts like M.
or M applied to ir. Before we set out to dt:rnonstrate this.rnctitlrr question. that i$.Il of the nstate machintls. ir true or false ilnlJwcr. 1. Of a. n}.rrurn's Figure 12.llltl funclions we must Iook a little firrther. It is easy ttr modify M to produce M in sttch a way that the la.vrlorrly final sta.lso at rnore genertr. Becatrscof Tbritrg's itr thesis. is not cxlrnputable. we ttr.chine M and positive numbcr ttl. We cau loclk ir. Considor t}rc function / (n. All we hirvc to do for this is to hirvr: M count its moves arrd termiuate when this r:outtt exceeds rn.ktrthe Iargest to give 'l\rring Tirkc irny ma. Most exa. there are some that always halt. This is because Q arxl 1' are finite' so t)iras a finite domain a. for exarrrple rnachines that ha.lt with one of two answers: M a. I'his in turn implics that there are only a linite rrurnber of different dts and Lherefore a firrite rrurnber of difl'ercrnt rlstate T'uring machinrrs.lfurtctions to to stx: if they are comprrter.) whose value is the mtrxirrrutn rrumber of moves thilt catr be made by any zstate Ttrring milchirre that halts when started with a blattk tapc.tesand therefore tnake no mov()lt. hlarft tape makes mtlrtl tltatt ?Iumoves. Iet trs rrtake sure that .sstlciated with tr.chinethat does halt will execrrttl a certaiu number of rntlvesl of these. we expect that functions t:rrcoutttered prar:titlal circurnstanceswill he crtrnputable.reorrlY a finite number of T\rring machines with n statcs.f (n) is dclirred for all rz. as it turns out.pplietl to a blank ta'pe htrlts irr rrd more than rzr nrovcs. Assrurlc rlow ihat / (tz) is <xlrnputable by some T\rrirtg I (n) . but they do not enter the definition of /. Every ma.rnllltls of uncomputablc ftrtrctions are a. This function. A decision problem is cffectively a' fitnctiorr with a ra'nge {0.306 Chopter l? ou AlcoRlrHMI(t CoMPUTATIoN I. 1}. do so. we f'olltlw the established trcthocl and rerhrr:c the halting problcrrr (or any other prtlblern knowu to be rrntlecidable) in to the problern of computitrg t.he fr.5 Algorithrn ftrr halting problerri.blc.ttcrnpts to preclict thtr bchavior of T[rring rrrirr:irines. Some of the zstate tnachineswill rxrt halt when started with a blank tape. Let f : {0. so for exermplcs of uucotnprrta.nd ritrrge. Noticg first that there a.ttcr will always ha.
n / (lQl) steps.chine 4 .lts. Show that the following problerrr is undecida. depertding on whether M applied to rl ha.1 ca.ke if it is to halt..r MAcHlNns sOz Figure 12.chinefor extx:rrtiorr. tu) halts. If we find thal . Mdoes not halt in rn steps rrtirr:hineF. W .12.. We carr then put fr ancl F together as shovrrrin Figure 12. We can relax this gencrality. Show that even in this restrir:ted setting the problem is unrler:idable.rrr € X+.use the definitiorr of /.ncla clescription of ffi is givt:n to a./ iu not cornputable. Irr the genera. we ask for an algorithrn that gives thc corlect answer lbr arry M and u.il.te of M.ltsor not. Given any T\rring ma.eexarnirre proof the of Thcoretr 12. g. qow*tw i I o.ximum number of moves that .1 to require that qo.n bc modified to prodrrce Ii' Suppose we r:hangeDefinition 12.t.This tells us whether M applied to a. . We say that such a problern is decidable iffor every u there exists a (possibly different) algorithnr that detcrmines whether or not (M.4.rrrnr. and.lhalting problern.u q. where Q is thc ster. I I' Dcscribe in detail how H in Theorern I2. M.blc.then trtx:tr.iot does rrot affect the proof in any significarrt way.fapplied to a blank tapc makes rnore tharr /(lQl) rnoves.o. by looking for an algorithm that works for all . detcrrnine whether or not thc symbol a is ever writtcn when M is appliecl to 2.trla.1 to show that this diflererx:e in the clcfinit. the irnplicirtion is that M nevc:r ha. The value wc get is then usecl as rrr ttl r:onstruct fr as outlinecl.1 sorvrsPRonlnvs T'rrn*rcaNNor Bp solvnn nv Tun.lxc.41 }lrt only a single . Thiu tells us the set mir.tu. R. 3 . lirst we compute /(lQl).\4 r:arr ma. a.4 Algorithrn Iirr blanktape halting problern.. Thus of we have a solutiorr to the blank tape hrrlting problem.4. for exarnple.nktape halts or rloesnot halt in lcss tha. r'he irnpossibility of tlrc conclusion forces rrs to accept that . a E f.universal T\rring ma.
ffi Considcr the set of all tlstatc Trrring machirtes with tape alphabet  : {0. ffi 1 1 . Deterrnirre whethcr or not the following statement is true: Arry problem whose dornain is finite is decidahle. 6 . Show that this set is recutsively cnumera. t}re nutnber of distinct Tbring machines with this f. the t:ell under the readwrite head at the beginning of the cornputation)?" Is this a decidable questiorr'i D. 7 .D}. Show thatr thc problern of determining whether or rrot niu u is undecidable. there is little we caII say about these la.GoH. ffi L How is the conclusion of Exercise 7 affected if Ml is a finitc automaton? 9 . that is. but rather is the gcrreral rule. Slxrw that the problem of tletermining whether a T\rring machine halts orr arry input is undecidable. !} and let b (n) be the maxirrrurn nurnber of tape cells exatnined by any rzstate Ttrring rnachine that halts when started with a blank tape. Let M be any T\rring tnachinc and r arxl gr two possible instantaneous descriptions of it.3.) is not t:onrputable.tIc show that there is no algorithrn to decide whether or not an arbitrary Tirring machine halts on all input. can we always predict whether or not the automatorr will halt on input Trr? 1 0 . Corrsider the questiorr: "Does a Turing trarhine iu the course of a computation revisit the starting cell (i. Thc lack of an algorithm to decide on some propertv is not an exceptionirl state of affairs for recrrrtrivcly enumerable lan. in e$$errce. Show that there is no algorithrrr for dcciding if arry two Ttrring machines M1 arrtl Mr accept the sarne language. 1ti. 1 3 . Recursively erlrrnerable languages are so general any question we arrk about them is undecidablet. 1 5 .l LItr. In Example 12. Lct I' : {0.e. Itrvariably. 14.rrguages. Lct B be the set of all Ttrring machines that halt when started with a blank tape. ffi ffi f P U n d e c i d o b l e r o b l e m so r R e c u r s i v e l y o E n u m e r o b l[e n g u o g e s We have cletermined thrr. Show that b (rr. I.2. that.t there is no membership algorithm for recrrrsivtrly tlrrurnerable languages. Give an expression for rn(rr. Is the halting problcm solvable for deterministic pushdown automata. As we now show.rrttr. 1.tIrson Al.ble' brrt not recursive. givetr a pda as in l)efinition 7.308 Chopter l2 Coururun'tol.). give the values of / (1) and / (2)' 1 2 .lrages. .
. Clearly.E LaNcuncns 309 whetr we ask a question about recursively enurrerirble languages.u. Wu conclude therefore that the sta..t there exists an algorithm A for deciding wlrt:tlrcr or not tr(G) : E. Suppose we are given a Thring rrra(:hineM and some strirrg zr. L(G) not empty ueL(M) eL(M) . carr alwtr.rr. M first savesits inprrt on some special part of its tape. Let G bc an unrestricted grailrmar.): L(M) n {zr}.7.ysbe done. whcrrerver enters a final sta. srrch tha. we would have a mernbership algorithm for any recursivcly cmrmera. We give here sonre examples to show how this is done and frorrr thcso examllles derive a.) is nonernpty if and only If w e L(M).5.5 is a T\rring rnac:hine whiclr for any . We can modify M as folkrws.it checksits it saved input and accepts it if and only if it is ur. we find that there is sorneway of reducing the halting problern to this question.I Figure 12.nindication of tlre seneral situatiorr. we tlrcn construct a corrcsponding grammar G. Figure 12. creating for r:irr:hur a machine M.rallyclear is that. 2 UNDECTDABLTI PRonlnus FoR R. Eqr.44 arrd .ECUR$IvRr. irr dirs:t contradiction to a prcviously esta.L(G. b l e .y Er*rurrrnRasr.If we let 7 denote an algorithm by which we gerrcrateG.ble language.12. Proof: We will reduce the menrbership problem for recursivcly enumerable languages to this problem. If sur:h ir T\rring machine existed. thc rxlnstruction leading frorn M and tl to G.t L(M".r tells us whether or rrot rt is in L(M).ted problem "L(G):0" i s n o t d e c i d a .. Usirrg Theorern 11.blishedresult.5 Mernbership algorithm. whethcr or not Then the problem of determinirrg L(G\: O is undecidable. We carr do this by changing d in a simple way. Assurne now tha.te. Then. then we carr prrt ? and A together as shown in Figure 12.
(fr) is finitc. final state for all input. the the newly crcer. Therefore.sked.rc clnttged so that if any one is reached. it will be modilied to accept the two strings a and b.L(M) contains two different strings of the samc length" is undecidable. is imrrraterial. If we trow assulne the existence of an algorithm A that tells us whether or not . all input is accepted hy MThis can be done by having any halting configurtrtiorr go to a final state.I'r'ulr. the specific nature of the question a. Show that for an arbitrrrry Thritrg machiue M with X : {4. the problem . First. Therefore no algoritlrrn for deciding whether or not tr (M) is finite can exist. I IE Notice that irr the proof of Theorem 12. tht: irritial input is saved and at the Figure 12.u) tloes not halt. If M halts in any configrrrrrtion. For this.6 . thgp fr will tralt in a final state. then frwiil not halt either a. the original mat:hirre is modified so that the new machine ff first generates tr on its tirpe.*). If so (M. In other words. Sccorrrl. using atrd some otherwise urrused space. we can construct a solutiorr to the halting problcm as shown in Figure 12.colt. M either accepts the infinite language X* or the finite Ianguage 0. if (M.chir. b}.tIc Lct M be any Ttring machirre. ur) halts. thetr performs the sarrrecomputations as M.310 Chopter I2 Coututnrlot't LIuIrs oF Al.4. except that whert M reaches a halting configuration. From M we construct arrProof: Consider the Blting problem (M. the halting states other of M a. M will rea. Then the qrrestion of whether or not L (M) is finite is undecidable.narnely "Is L (M) finite'i". T\rritrg machine M that does the following. ye use exactly the sarnc approach as in Theorerrr 12. We can change the nature of the problem without signifitnntly affecting the argrrrnent.4. To show this.ncl will accept nothing' In other words.6.teclur moves rrrade by M after it has written u on its tape are the salne as would have been tnade by M had it started in the origirral configuration gnrr.
lizing this is known as Rice's theorem. Argue (a) frorn Rice's Theorem and (b) from first principles.2 UwpBcTDABLE PRonlu.rgument then procxrercls as irr Theorem 12. The adjective ((nontrivial" refers to a property possessedlly sorrre but not all recursively enumeratlle Ianguages. LeL Mt and Mz be arbit.geis undecidable. ffi Show that the problem 4 .r'y T\rrirrg machines.[f in Th"o. Thcse questions. Does there exist an algorithrn fbr determining whether or rrot .ncuRsrvr:uy ENulrnRanl.L(Gr)n L(Gz): is urxlecidable.rrystring of length five?" or "Is L (.f) regular?" without affecting the argurnent essentially. show that the question "Is tr (G) : L (G)*?" is undecidablc.4.redwith a arxl h. Thus.L (M) contains any string of length five. we can substitute other questiorrs slch as "Does L (M) contairr tr.L (G) : f (G)R? 6 .rs non R. that the problcrn ."rrr I2. . otherwise fr wiil accept rtothing. ffi a Slrow Show that the question in Exercisc 6 is undecidable for any fixed G2.A precise statetnent arrd a. are all undecirlrlble. as long a^rtr (G2) is not empty. Let G be any unrestricted grarnmar.12.w) hirlts.L (G)E is recursively enumerable? 5 .4 is constructed. L lbr an unrestrictecl grammar G.cceptingonly these two strirrgs. (b) .proof of Rice's theorem can be fourrd in Hopcroft arrd llllman (1979). Does there exist an algorithm for determining whether or rrot . as well ars similar tluestions. M will accept two strings of equal lcngth.rcuAces Bl1 errtl of the cornputatiorr compa.L (M) is regular. The rest of thc ir. namely (a) . " L (Mt) C L (Mz)" is rrndecidable. ffi . 1 .pLer. and Gr any regular grammar. a.ra. This tlreorem states that any nontrivial property of a recursivcly enumerable latrguir. Let G be arry rrnrestricted grarnmar. I In exactly the sarrrc ilIirnner. Show that the two problems mentionetl at the end of the preceding section.4. if (M. Show in detail how the rrrachine . are undecidable. Let Gr be any urrrestricted gramrnar. A general resrilt fbrma.
7 1 ul u2 I .S Let x: {0.. for any (A. If we take ?01 Ul = 00..B)... there exists a PCsolution as Figure 12.312 Chopter l2 LItr. say A: Ld B : UtrI)2r.u2:11. and it is corrvcrrient to establish $ome intermediate rcsults that bridge the gap hetwecrr the halting probkrrn and other prohltrrns.PCsolutiotr' Eifrrlplq t{.?s:1I For this case. 0 0 v2 "J I .h..1} and take A and B as : r u r : 1 1 .1 u. The Post correspondence probk:rn is to devise an algoritlun that will tell us.ue:001.7 sltows.l interelst.ltirrg problem dircctly.Il)n we say thnt there exists a. The Post corrcspottdence prohlem t:itrt be stated as follows. One srrchintermediate result is the Post correspondence problem.. : 1000 urs :0. particularly in tlrc area of contextfitNl languages. such tltat U ) i ' t . These intermcdiate results fbllow from the rrndr:r:idability of the lrtlltirtg problem..LDzr. brrt are more closcly related to the protrlerrrs we wa.tIrs oF ALcoRI'I'UMIC CoMPUTATIoN Problem fiWilffiThe PostCorrespondence The undecidability of the halting problem has many (toruequencesof prirctica. ' : UiUi''' 1...i.)1r. whether or not there exists a. Given two sequence$of z stritrgs on som{} alp}rabet E. But irr rrratry instanrxrs it is cumbersomrr ttl work with the hrr. Post correspondencesohrtion (PCsolution) for pair (A.us:011 Figure 12. ? r 2 1 0 0 r u t r: 1 1 1 ur:111. t .nt to study atrd therefbre make the argurrrerrts easier.rlz : 001. B) if there is a nonelmpty sequenceof integers i. t " i' u k .rI)r1 u)Lt.
nceswe rray be abk: to show by explicit constructiorr that a llair (4. Cl . we iutroduce the followirrg r:onstruction. to it.Ia.. The Post prrlllklm is therefore undecidable.xTIffi ilTlil'll.8. B) has a modilied Post correspondencesolutiott (MPCsolution) if there exists a sequrrnr:e of intrlgursi. .F.:K'J#liJJT Exom$lf I f. Brrt in general. we introduce the modified Post correspondence problem. then there is also a ltCsohrtion. as we did abt)vc. Wc ua. Wcrwill demonstra.f.5. " In the modified ltost corrt:sllorrdence problern.restion under irll r:ircumstances.I2. be arry PCsohrtion simply becausc a"ny string composed of tlltlments of .5.rt. Iet us illustrate it with rr simpkt example. that no such solution can cxist. B) as shown in Figure 12. We want to claim eventually that rp e L (G) if and only if the sets A artd B txinstructed in this way have an MPCsohrtion.aABblBbb. Supposewe are giverr arr unlestrictcrl grammar G: (V.9 =+ is to be taken as'u1 irrrrl tIrc string F as u1... c.lsor.b.tely obvious. we break it into two parts.'l'..ythat the pair (A. In Figurc 12. but the converseis not true.ir (4.oeLEM 313 there cannol.f.rndecidable.P) arrrl a targr:t string ur. such that 'W1'Wi'Ui "tttk = U11)i1)i'''U1x.tJ*Ti. R) admits an MPCsolution.te the undecidability of thc modified Post correspotrdence problerrn by reducing a known undecidablc llrotrlem. Itr tlrc first pa.i.Posr CoRRnspoNDENCD Pn. AC + uat:. BbC.lgorithm for deciding if arr artritrrlrv pair (4.:fi ". The order of the rest rlf the strings is immaterial.". This problem is a. P) with productiorrs S .keof clarity. The rrrotliliod Post correspondence problern is to devise an a. there is tro algoritlun ftrr dor:iding this qr. we create thc pa. Since this is perhaps trot irnrntrditr. Ttr this end.d Let G : ({A..} . ilirr. . ()r we may be able to argue. For tlrt: sa. the first elernt:rrts of the se an MPCsohrtion. B. B) permits a PCsolutiorr. With these. the membership problem for rrlcursively enumerable languages.4 will be lorrger thnn the corresponding strirrg frrlrrr B.fr.8. the string . I In sptx:ific insta.ll THr'. corresponderrce To show this is a sourcwhat lengthy process.
9 T u.a. for every *.e.c.C \ B Fis a symbol not in FU 7' a foreveryaET v tl for everyVreV E JroE E is a symbol not in I/ U ? ti x.L(G) and lus a derivation S+aABb+aAC+a.' and u.9. used to create the string. vi I FS+ a tj t F tl 2 3 + 5 6 7 8 9 10 11 72 13 l4 b t A B c .314 Chopter l2 LIIrllrs ol AlcoRtrHMIc CoMPUTATIoN Figure 12.ralleled by an MPCsolution with the corrstructed sets can be seen in Figure 12. respectively.(Lc in.s A B C s s s E d'{Bb Bhh U ailt j addcb Bb AC .10. *1.rrdB obtained from the suggested is con$tructiolr are given in Figure 12. How this derivation is pa. The integers above and below the derivation string show the indices for z. The string ru : a. in P } D and take ut : aaac. Figurc 12. The sequencesA a.wltere the first two steps in the derivation are shown.e.8 d F.
indicate tho lines along which the rrext rcsult is established.rtreduce the rrretrbership problem for recursively enurnerabkl lrr.linductive argument based orr the outlined reirsonine.. We will omit the details..10 f wl %to s ?10 a d B b il! ur F .S +. Let (A.a t s = "10 il14 b v12 + ilt^ v C '13 E . I Let G : (V. Therr thc pir.S. Irr this instance.ngua. The only exception is the last step. Tho c:onstruction. I Witlt this result. .ir ('{. so to rnatch it we have to use u1s or u11.3 THE Posr Con.8.2 l vg Exarnine Figurc 12. lending us to the secondstring in the partial derivatiorr.12. we ca. The complete MPCsolutiorr is shown in Figure 12.n.10 r:a. Proof: The proof involves a forrrrir.mmar.this brings irr'rr.7 *10 1J).11.refully to see wlnt is htr.. B il'rz b = rt C vz Figure 12. We want to construct an MPCsolution..rur.pspoNt)ENcE Pnoer. f. we is always longer than the corresponding string a1uiaj. with u any string irr T+ . so we must sta. that is. we u$e tr10.11 F vI 1 u1o a i D ur4 *z s a @r2 ur4 u2 a il u.rt with tu1.ur1.ppening.ges the rnodified Post correspondence problem and to thereby dernonstrrrtethe undecidability of the lattcr. where uo rnlmt he applied to let the ustring catch up. together witlr the example..'16. Looking at several rnore $teprJl see that the string u. This string contains 5.T. B) be the correspondencepair c:t)rr$tructecl fiom G and trr be the process exhibited in Figure 12.nlr S15 Figure 12. ^ u2 tus u12 4 s ?10 d n il.t and that the first is exactly one step ahead in the derivation. B) permits an MPCsolution if arrd only if w e L (G).P) be any unrestricted gra.
5.ttxlfrorn trri by appending I to cac:h l..il.s16 Chopter l2 LItr.Aft.r algorithm. r'1: lua In words. . By Theorem 12..P) and ur € T+.12 Membershil. a.n. wt: construct the sets A and B as suggestctlabove.nnot be arry G and u.. li : zi : wilQu)i2Q''' LDi.' does not.i$. now we assume that the modifiecl Post correspondencleprobSuppclsc'l Itlrr is decidable. We must therefore c:clrrclude probletn' r algorithm for deciding thc rrrodified Post correspondorrt:tt With this prelimina.nrs on Al(loRIrttNatc CotrrlpurerloN Figure 12. wc are rlow ready to prove the Ptlst corresporrdenceproblem in its originirl ftrrm. Proof: Given arry urrrestricted gra..\)i1 on lrorrrealphabet X. .B) Iras an MPCsolution if and only if w e L(G).2. respectively.. . and B : 1.1'ttn. An algorithm fbr constrttctitrg A front B from G and tu clearly exists. the modified Post correspotrdence problcm woulcl be decidable.4 and B as in Figure 12. We can then cttnstruct atr algorithm for the merrrbership problern of G as sketched in Figure 12.S. defincd as follows. where ru4i irrrtl u4i denote the jth letter of u). Construct..8. but a membership algorithttr for that there ca.nrl rni : lu. We then introdu(le rrew symbols g and $ and thtl rrtlw sequeilces C:Ao.. .T.ai1Qui2Q. . For i : 1. l.12. The Post corresponclenr:cprotrlern is undecidable.ry work. gri is creir. .. arrd ?i.t. thc lrair (A. We argttc that if the Post corresponden(icl)roblem were decidable.mmar G: (y. A tu* we Srrppose are given sequence$ : 'u)71Lu2r.n. tZ n l t . Lti. Proof....tUn*tt D : z o t Z t t . ueL(G) vte L(G) The modified Post correspondenceprohlem is undecidable.
t algorithtrt for decidirig thc Post rxlrrosp{nclence 1 .1 1 1 . B) irave a PCsohrtion? Does it havc an MPCsolution'l ffi Provitle the rletails of the nroof of 'l'heorcm l. D).urra 317 Figure 12.2 L u . 'Qw. . such a solution must have g/0orl Lhe left rurd Un1l olr tlrtl riglrt and stt must look like ' 'Qwtt(.n N{PCsolution. j ' ' " l i l 1 r ': U1'11. We can trrrrr tht: ilrgrrrnerrt rlrclund to show that if there is an MI'Csolution for (4. Consider rrow the pnir ((J.rJv tltx:idesthe modified Post correspotrcletrce problcrn. B) therr tlierc is a PCsolrrtion for the pair (C.12. t . I 1 .i''' Ulrj so that thc pair (. . . r  .+I $r f() * 3lr 2". This mar:hineclea.3 THE Posr ConnnspoNDEN(iEPRosr. MPCsolution No MPCsolution charat:ter. Q E . 1 1 1 . B) pcrmits a. q \ : g z t 1 1 t l u 1. To cornplete the deflnition of d and D.1' . . Does the pair 1 } (4. while aa is obtainecl by prefixing each chirrnr:tr.. ( . . with f .1111Qw12' . Brrt tlrc rnorlifiexlPost prohlern is undecidable. 1 0 1 a n d I J : { 0 1 .2. we tako A 0: g Y r ) : 3/n. 0 1 0 } .4. and suppose it has a PCsolution. . D).13 MPC algorithrn. Bcr:arr$el of the placernent of q arrd $. consequently.'rof u. we see tha.13. We Assurne now that tlrrl Post r:orrespondence r:an then construct the rnachineshown irr Figunl 12. Q1. i . L e t A : { 0 0 1 . Ignoring tlte charactcrs r/ arrd li. problem is deciclable.t this inrplies ' W 1 U l . we camrot havel arr cortesprlrrrlcrrtxr problerrr.+1* #li. 0 0 1 . .5. u t .
. . .l Now look at the contextfree gramma. (a) There is an MPCsolution if there is a scqucnce of integers srrch "'utNwT = uiuj "'utrur.i . j. . aiai. S r a . .'5. . Suppose we restrict the domain of the Post correspondenceproblem to include only alphabcts with cxactly two symbols. &jilil .ka.k such that wnwj .wtuth&ka4. S U C hh a t .... AlcoRrrHurc Conrur. there is an algorithm that can decide whcthcr or not (4. .Show that the problem of deciding whether or not j. .&r}flX:O.I. that is. ffi that tuatrri (b) There is an MPCsolution if there is a $equenceof irrtegers such that ultrr2zriur "' u)k :'Dt'Dz'tti'uj'' "D hr. Show that the ftrlkrwing rnodilications of the Post correspondence protrlerrr a. .. new set of distirtct syttrbols & I r & 2 r .t1r)tr.B) on a singlclctter alphabet..r G : ( { S . Is the resulting correspondcncc problem decidable? 5. for Undecidoble Problems ContextFree ffiilffiffi [onguoges The Post correspondence problem is tr convenittrrt tool for studying undecidable qrrestions f'rrr txrntcxtf'ree larrguages. B) has an even PCsolut..qrtox 3. 1)luke. 'r.3L8 Chopier l2 Lrvrrs or.ion is undecidable.wk : Itn'u . Show that for lll : 1.. and qlnsider thtt twcl larrguages Lt1 : and Lp : luitt. an arbitrary pair (.or.. Choose a.4. B) hasa PCsolution ftrr arry given (. . (wt. the Post correspondeuce problem is decidable. B) is sairJ to have arr euen PCsolut'ion if and only if there exists a nonempty seqlren(:eof everr integers 'i. n } ..'ur) over some alphabet E. The corrcspondence pair (4.a r . a n }. P ..u2..1J)2. } J U { a 1 . {wiwi .....ndB : Proof: Consider two sequences strings A: of (ut.. . a 2 . S ) 5 . There exists no algorithm ftrr decidirtg wltetlter any given contextfree grammar is amhiguous.. . t {or.re undecidable. ffi 4. . We illustrate this with a few seler:tedre$rrlts.4. j 6.
.14. we could adapt it to solve the Post correspondence problem as shown in Figure problem. If a given string in .CoN'rEXrFRueL. Conversely. then the Post correspondence problenr with the pair (4. Now take G a : ( { S . S ) . S n } . ^ 4 * wiwi " 'wtrak " 'eiai : 111 and 5 =+ ^9gg ui]Bai 4 + uiui ' ' 'ukek'''aiai : v1.. the second set P6 has tltc lrro<1rr<:tions S __+ Sr.4 UuoncrDABLE PRonlnlrs FoR. . S a } . B) has a solution. a 2 ...2. . Similarly. . 1.if G is unambiguous. wi]eatlwiai. 12. X U { a 1.. then the Post correspondenceproblem cannot have a solution. if G is ambigrrourr it mrrst be because there is a ur for which there are two derivations J :* Jg ) wi5nai L (Gs).L (G) ends with aa.12...ncuncns 3L9 wltere the set of productions P is the union of the two subsets: the first set P4 corrsists of SSa. Then clearly Ln:L(Gn).. Consequently. i : I. Thus. a n } . we can tell at any later stage which rule has to be applied. . G n : ( { S . Ln: and L(G): Ler Ln. I \ ) { a 1 .q.n. Sn + uiSnailuiat. ..2.. But since there is no algorithm for the Post correspondence we conclude that the ambiguity problem is undecidahle. S. then its derivation with grammar Ga rrrust have started with 5 + u)isai. If there existed an algorithm for solving the ambiguity problem.& . if G is anbiguous. . I .n.q.. . a 2 . P e . It is easy to see that Ga arrd G6 tly themselves rlre rrnamhiguorrs. S ) and 'i: I. P n .
c+oR.320 Chopter l2 Coupuranron Lluns or Ar. That there are many undecidable problems connected with contextfiexr languages seems surprising at first and shows that there are limitations ttt computations in an area in which we might be tempted to try an algoritlrrrric approach. i . whilc otlters are more easily solved by establishing different intermediatc rc'sults first (see for example Exercises 6 and 7 at the end of this section). r There is a variety of other known results along these lines. Srrppostr common element. u k e k" ' e j a . We conclude that . or if two different specificatiots of a language are in fact equivalent.i' .14 PC algorithm. For example. B) hasa PCsolution. We will not give the arguments here. Keep in mirrd that this does not rule out the possibility that there may he ways of getting the answer for specific . There exists no algorithm for decidirrg whcther or not r (Gr) L(G2): n n for arbitrary contextfrcc grarrllllars G1 and G2. Converscly. and it would tra a waste of titne to Iook for an algorithm for either of these tasks. i and S 6 1 u i u i . ' u t r a k ' . then L (Ge) and I (G6) cantrot have a common element.L (Ga)n L(Gu) is nonempty if and only if (. that is S a 3 t r t i t u. . Then the pair (A. But the results that have beerr established tell us that this is not possible.if the pair does not have a PCsohrtion. ' a j a .4. B) has a PCsolution. defined Proof: Take as G1 the gralnlnar Ga and as G2 the Hrammar Gr a*s that tr(Ga) and L(G") have a in the proof of Theorem 12. This reduction Drovesthe theorem.rrrrrurc Figure 12. . Somtr of them can be reduced to the Post correspondence problem. it would be helpful if we corrld tell if a programrnirrg language defined in BNF is ambiguous. but poirrt to sorrreadditional results in the exercises.8.
GB. For any such comnutation gourF fll F 12 F ." [. we can then construct the strins qot F *fl F rc. We can as$lrme without loss of generality that everv computation involvcs an even number of rnoves. t 3. * 6. Use this to show that the problem "I (CJ) is regula..4 UNDECTDABLE PnonlnrulspoR.rJ is urxler:idable for contextfree G. Let Gr be a contextfree grarnmar and Gz a regular gramnrar..'REE LeNcuecns 321 casesor perhaps even most interesting ones. Gr.ff. Show that thc problem of detertrining whether or not * ? 1.(Gr)c t(Gr) is trnrler:idahlefor contextfree grammars Gr. * 7 .e by therrrselves are unambiguous. F :r. Let M be any Tlrring rrrachine. (Gr ) n L (Gr). I.f  . Show that for every M we carr cortstruct three contextfree gramrnars Gr . Let Lr be a regular larrguage arrd G a contextfree grammar. What the urrdccidirbility results tell us is that thert: is no txrmpletely general algorithm and that no matter how many dilTerent cases a rncthod r:an handle. the problem "L (Gr) n L (Gz) is contextfree" is undecidable. Is the problem I ( G r )n L ( G z ) : o decidable? .8 is regular. (h) the rret of all invalid comprrtations (that is. * 5.'l Ihis is called a valid computation.Cor'r. the complement of the set of valitl t:ornprttatiorts) is I (Gl). Gz.12. xn.. frrr arbitrary contextfree grammars Gr and G2.XT[.8 that Ga and G. * 4. then it must be empty. such that (a) the set of all valid rnmprrtations is /. Show that. Prove the claim made in Theorem 12. Show that the problem "lt C L (G)" is undecida]rle. Use the results of the above exerciseto show that "L (G) : X*" is undecidable over the domain of all contextfree grarnrnars G. Show that if thc languagc L(Ge)nL(Ge) in Theorerrr12. there are invariably some situations for which it will breah tlowrr.. * 8 .rr.
rrrnrrc CoNrpurerroN * L Let Gr anrl G2 be grammars with Gl regular.c. (c) when Gz is regular? .on. Is the problern L(Gr) = L(Gz) decidable wherr (a) Ge is unrestricted.322 Chopter l2 Lrurrs or Ar. (b) when G: is contextfree.
No t:rlrnrn(:r(:ilrl rxlrnprrterswere availahle a.A. T\rring's work was published in 1936. M.t that time..O t h e rM o d e l s of Computqtion Ithough T\rrirrg rnar:hinersa. lhe whole idca had bccn rxrnsideredonly in a very pt:ripherir. In fact.re the most general models of cornputation we carr corrstnrt:t.t tha. contribuled to it.thematics inr:rea. other rnodels hirvt: llccrr proposed.re not tlre only ones. but orr rnatlxlrnaticu ar a. With the discovery of diffr:re:rrtiill irrrtl integra. some of which at flrst glance seernedto tre raclically different f'rorn Thring machines.whole.lcrrlus Newton and bv Leibniz in the seveuteenth and eiglrteenth c:etrturic:s.l goal was not to provicle a fotrrrhis da. all thr: rnorlcls were fbund to be equivalent. thev a. Although T\rring's ideas eventually becarnc vcry importattt irr cornputr:r sr:iunr:e. irrtelrcst in ma. To understand what Thring was trying to do.thematicsa.tion for the study of digital r:ornputcrs.l r:ir. Eventually. The results that were fourrtl slrrxl light not only on the concept of a mecha. however.nical cornputation. At variotrs times. origina.t tirne. Ttrring among them. A mrmber of and 323 . we must brielly look at tlxr statc of ma.sed the discipline entered an era of explosivr:grclwth.l way. Much of the pioneering work in this area was tlorrr:in the period hetween 1!130 and 1940 and a nurnber of rnathernaticiirrrs.
Becarrse pra.rart(ltt O'r'truR. by Consistency is indispcnsablc irr rnathernatics. one rxrrrldgivt: rrtclredetailed reasoniug.ble. Ihe arguments throw light on the subim:t attd at least increaseour confidencethir.r sct of assutned givetrs. This krd to a concern with rigor in reasoning and a consequent examiniltiorr of the foundations of rna.l proof itr just about every book and ptr. it rnust couany interesting consistent systeiurmrrst tltl irrrxlrrrlrlctel . or make wrong infi:rcnt:tls.rison that required a more ca. at !'indirrg such lortnal systetns was a ma. Whetrever we see arguments like if this. aud atrything cleriveclfrom an inconsistent $y$torrrwoultl tltl cotrtrary to all we agree on. prec. Often there is no way of telling. Rut this hope wrs rlir.Frlr sorne tirntr it was hoped that consistent and complete systomtr ftrr rrll of tnalhernatics could be devised thereby opening the door to rigorous but cotrtpletely mechanical theorem proving. this type of reasoning is a. then iL is considered false.nother.tiorr carr be checkeditr a routitre atrd completely mechanicalway. By the end of the nirxlteenth century." Such phrases a. Of course.l of mer.nbe seen casilytt irrrd ttit follows from this. tvpit:a.t. A sequenceof plausiblc cla. Two concerns immediately arose. One alternative to such "skippv" rnathtlrlrirtitls is to formalize as fa. We sta. called axioms. by which we mea. if challengedto do so.rt with er.jor goa.shtxl by the work of K. Itowever.Ir that a.isely definecl nrlels for logical inference and cleduction.324 Chopter l3 Motnls on Clolrlru. Gcidel showed that tlnt is. circlt of whiclt takes us from one proven fa. By this wc rrx:irrrtltat tirere should trot be atry proposition tha. Godel. A propositiorr is irr considered proven true if we can derive it fiom the ar. and as possible.thema.ptrr dealittg with mathematical subjects. irrxl what one Ineans by them is thtr. use faulty hidden assumptions. In his famous Incornpleteness Theorem.nypropositiorr cxpressible in the syst.n be proved to bc tnxl by orre sequellce of steps.pplictr. The rules tlre of used in a $eqrr()n(:o stclls. different areas were studied.re (pnv()rrtiorritl. But to those demanding cornplete reliability. the body of mathcrnatical knowledge hacl become quite la.t ca. a.rgrt.ct to a.inlerspersedwith phraseslike "it ca. If thr: propositiorr corrflicts with another proposition that t:iln b(: llroved to be true. Ttr see why this was llecessary.tical knowledge in the proce$$. A secoud concern wils whctlrtlr a systern is complete.eurcan be provecl to be true or fh.t tlrc rt:sult is true. Matitettnticiatrs also had becromeurrfficienlly sophisticated to recognizc'that sorne logical difficulties had a.t:ticalliruitatiorrs.txxrptcd of by most rnirtherrraticians. Tlrrl first was that the system should be consistent. this is very dangerous. Thc: nrlr:s rnust bel such Lhat the correctnessof their a. considtlr what is involved in tr.t]rcrrratic:s t]te crrd of the nineteenth century.lse irrxlthtlr ctpirlly valid argumeut.refulapproach.nrl sigrrificatrt aclvauceswere made in alrnost all of them.lse. then shown to be ftr. irrrd lortg ancl involved proofs have heen published and fbr.xiorrrs a Iirrittl sequeltce of logical steps. since it is possiblc to overlook things. we cannot help but wonclelr tlrtl llroof we are given is indeed correct.rnclerr*one()uiJ orrly after a considerable amount of time. they are unacceptrr.irnsis rrrade.
addressed. In order to studv the tprcstiott. It was this problettr Lhat T\rring and other mathematicians of the time. We will give onlv il vtlry bricf presrlrrtatiorr.t gives iln explicit nrethod for thtlil cotrrputation.thematit:stxltrld be tnade precise with mechanically verifiable proof's. Prorrrirrc:rrtirIIIoIrg them were tht: rccursive futrctious of Church and Kleene a. This is very broad and generrnlirrrd itnmeditltis association. so that sta. called the domain of tlx: firrrc:tiotr. Recursive 1gffiffifi$ffiffi Functions Tlrc rnrrcept of a function is firndeurcrrtalto rnuch of nta. a. C.nd Qualitz (1978). Kleene.t there are futrctiotrs that cannot bc cxprcssed itt atry way tha. and the combined notion is sotrretines called the thesis.sthat arosc out of tltese studies. In tiris cha'pter we briefly review some of the idea.tion. whilc not provable. gives $trong e:virlt:trce powerful models cart lle fcrutrd. Dennis. I'he clainr is tif t:tltrrsevery closely related to Thring's tk:sis. Itost.1i1.hle rxnrld sornt:ltowbe distinguished from the provable oncs.trelcrlcrrt of otre a st:t. Gcidel's work left rrnanswurudthc tluestiorr of whether the unprova.y$irr whidr furrctions can be definelcl.blished.ninherent limitation itr lhis atrd thir. The utodels of computation we study here.flnction is a rulc tlnt assigusto a. arrtl E. S. as well as others tha. have diverse origins. while a good disclnsiorr of vitrious other rewriting systems is given in Salornair(1973) and Salomaa (1985). ilhe spirit of this observation is genera.ntltlrt:rset. Gtjdel'u rnvolutiotrary conclusion was published in 1931. A qr. but there arc rnarry other such systems tha. It also irnplies lhal there is a.referritrg the reader to other ref'ercnctlsfbr detail. ately raisesthe questiorrof how we can explicitly reprtrst)rrt There are many wa.llyr:a. It provides a genertll prirrciple for algorithmic: ChurchTlrring that uo more cornputation and. if they are sufficiently hroa. while otht:rs are less colnmolt.ncunsrvn !uncrroFls 3?5 tain soure unprovable propositions.rrrust be eqrrivaient.ll fnmiliar wilh functional rrotatiorr irr wlticlt we write expro$sions like + I Ui: rz2 l. prr. Church.ntl Post systems.1 R. Sorrte of thetn we use frecluently. unicluevalue in a. We are a. This thesis statcs that all possible models of ulrnputir. called thc range of the function.1.tl.l rrxrtlels of computation were esta.t ha.l here thnt wc carrrtot cover.rticrrlarlyA. Brrt it was evetttually found tha't thev wcrel all equivalent in their power to c:itrry ottt cornputations.lkxlChurchts thesis. a.tementu thcrc was still sotrtehope that most of ma.t havc br:crr sttrdied. There is a wealth of materia.thematit:s.rite firrrt:tiorrsarrtl Post systernsca. a variety of forrnir.nbe firrrnd irr accessibleaccount of recursiver Denning.As surrrmilrizcd in Section 1. .vt: bccrr proposed.
r from defined functiorrs gt. .we can compute its valucs in a strictly mechanical fashion.a ) . a ) ) frorn defined functions gL)gz. we will consider only functiorrs of one or two variables. Since ma.u)).yrrof building more complicated functions from these: 1. If a function is dcfined (that is. we rnay well ask to what extent the notatiorr is universal.h.i. the set of all nonnegative integers. 3 .y.ny very complicated furrctions (:an be specified this way. To cornplcte the definition of /.2. (n.'. There are two wa. then the range of / will be some subset of the set of positive integers.Tll ..rgrrment n. Composition. gz. multiply that value by itself. and whose range is irr 1.v). and therr add one. hy which we construct ( I @ . together with nrles for building from thenr solne rnort) cxlmplicated ones. by which a function can be defined recursively :'. for exarrrplc.i. a ): l t ( n ( * . 2. Sirrce the function is defined irr this explicit wa. we take the domain to be the set of all integers.. or I x I. we must first clarify what the permissible forms are: for this we irrtrodrrce some basic functions. we know the relation bctween the elements of its domain atrd its rartgc).xt in sequence to r. Primitive through recursion.. and h.r z ) : r k . MoDEr. we also rmrst specify its domain. Thct projector functions whose value is the integer ne. in the rmual notation.soF CoMpuTATroN Tlfs dcfinrls the function / by means of a recipe for its computation: given any value for the a. If. P n ( n 1 . we start with the basic: firnctions: 1 . The successor function. Primitive Recursive Funclions To keep the discussiorr sirnple. 2 . can it be expressed in such a functiorral fclrm? To answer the question.326 Chopter l3 Orrrnn. for all r € 1. s (z) : z * 1. that is. Tlre zero function z (r) :0. In this setting. s z r .s(r). . whose dornairr is t ither I. h : I.
iu which h is itlrrntilicd with Lheatld.13. adrl. To adcl 2 and 3. Y ) ) .r.the uerxrndstr:p is arr applicatiott of pritnitive recursion. 0 ) 1 )+ I + :(3+1)+1 :4*1:5. we rnust deflne it. I ) + I ) : ( a d d ( 3 . t :.1. we cirrr rr()w dt:fintl rrrrrltiplication by mult(r. ta. I Substraction is not rpite so obvious.g) is the projector ftnction Pt\r..0) : .! ! : .lly. we apply these rules successively: a d d ( 3 . m u l t ( r . (r. function.3 .0):0.lltxl thc: monus. it defines subtraction so that its range is 1. A kind of subtra(:tiorr is dcfined from usual subtraction by : t .g l f x : 7 A .king into rr. Forma.rlcrloNs 327 We illustratc how this works by showirrg how the lrasicoperatiotu of integer arithrnetic can be constructed in this fashion.t/) + 1. Additiorr of integers r and E can be implernented wiih the function add(r.tl). :t"u:0ifc<E' The operator . defined by ttdd(n. ( r n u l ' Ln .3/ + 1) : add (r. Y + 1 ) : a d d ( n .ccorrnt thnt rrcgativtl nutrrbers are not perrnitted iu our system. First.1 Rncunsrvr Fr. . and 92(r. 2 : a d d(.is sornetirrxlu cir. I Using the add function defined in Example 13.y).
. A firnction is called primitive recursive if and only if it can be constructed from the ba*sicfirnction$ s? .(subtr (5. and from it.328 Chopter  3 Ornon Monnls or.t.t/ + 1) : pred(subtr (*.irfr. y)) . Conrpureuorr Now we defirre the predecessor function P r e d ( 0 ): 0 ' Ttred(Y*I):'c. It follows from this that everv primitive recursive function is a total function on I or I x 1. fil i.il$' lrr . To prove that 5*3 : 2.l . With the algebraic operations precisely defined. We call firnctions that can be constructed in such a manner primitive recursive..pk. .e.d(B) _. we can define integer division. gz.2)) pr ed(ytr (subtr (5. but we will leave the denronstration of it as an exerci$e. and very complex computations built from the simple ones. the subtracting function subtr (r. and lr are total functions. i'r.0) : r. we see that the basic arithmetic operation$ are all constructible by the elementary processesdescribed.3): :  pred. we reduce the proposition by applying the definitions a rrumber of tinres: subtr(5.pred(pred$)) : pre. by successive composition and primitive recursion. other rnore complicated ones can now be constructed.0)) )) pred(ered(ered(5))) . 1)) ) ed pr ed(pred (pred(subtr (5. subtr (r. Note that lf h. T In much the same way. then / defined by composition and primitive recursion is also a total function. If we accept this as given.
proving tlte theorem. Proof: By the argument of the previous theorem.1 RncunsrvnFUNCTToNS 3?9 The expressivepower of primitive recursive functions is considerable.its the following argument shows. The actual construction of an example of such a function is a much more complicated matter. g is well defined and is therefore in F.. not only are there firnt:tions that are not primitive rccursive. there are in fact computable functions that are not primitive recursive. f t. This contradiction proves that F cannot be countable.2. r The nonconstructivc proof tlnt there are computable functions that are not primitivc recursive is a fairly simple exercise in diagonalization. Then there is some Proof: Every primitive recursive function can be described by a finite string that indicates how it is defined. say. Let C be the set of all total c:omputable furrctiorrs frorn 1 to 1. the function g differs fiom every ri arrd is tlterefore not primitive recursive. Let us denote the functions in this set as rr.. the demonstration that it is not primitive recursive iu quite lengthy. However..g is r:onrputable. this goes even firrther. We will give here one example that looks quite simple. Such strings can be encoded and arranged in standald order... not all functiorrs are in this cla*ss. f2. however.gby sQ):rift)*By construction. Therefore.. I Actr. We next construct a function g defined as s ( i ) : f i ( i )+ \ i : 1. function in .. Clearly. Let f' denote the set of all frrnctions from I to I. the set of all primitive recursive functions is countable.13.and define a firnction ..rz. But clearly . g differs from every fi in the diagonal position.. Suppose now that the set of all functions is also countable.rally. We can then write all functions in some order.. the set of all prirnitive recursive functions is countable. Combining these two observations proves that there must be some function in f' that is not primitive recursive. . Then there is some function in C that is not primitive recursive.F that is not primitive recursive. but equally clearly.. and most common functions are primitive recursive.
fbralli:n.p . Of course. It is not hard to seethat . The situation here is similar to the one we encountered when we tried to prove that a language wa$ not regular or not contextfiee. In fact. we cannot argrre directly from the definition of A. How this is related to the limit of growth for primitive recursive functions is made precise in the following theorem. For primitive recuntive functions.'CoNtrurnrtotl Ackermqnn'sFunction Ackermann's function is a function from 1 x 1 to 1. . 5 3 4 ) . But in spite of its apparent simplicity. We need to appeal to some general property of the class of all primitive recursive functions and show that Ackermann's function violates this property. which is tedious an<l technical.1. t 1 * 1 ) : A ( r . 0) A ( u . . : A(:r . Its proof. it follows easily that Ackermann's function is not primitive recursive. Even though this definition is not in the form reqrrired for a primitive recursive function. ExercisesI to 11 at the end of this section. one such property is the growth rate.4 is a total. see.nn's function grows very rapidly is easily demonstrated. r E If we accept this result. it is quite elementary to write a recursive computer program for its computation..i. computable function. There is a limit to how fast a primitive recursive function can grow as i + oo. Proof: For the details of the argument. defined bv A(o's):3/+1. and Qualitz ( 1 9 7 8 .330 Chopterl3 OrHnn Mounls or. That Ackerma. Dennis. 1) . and Ackermann's function violates this limit. i ) . ): A ( i . Then there exists some integer n such that /(i) < A(n. see Denning.. Ackermann's function is not primitive recursive..).n+1. Ackermannts function is not nrimitive recursive.4 (4a. will be omitted. Proof: Consider the function s ( i .1 . it is possible that an appropriate alternative definition could exist.A ( t : . y ) ) .. Let / be any primitive recursive function.fbr example.
for r: { 3 P'a(g @. the minimalization operation opens the possibility of definirrg partial firnctions recursively.ll) is a total function. accordingto Theorern13. we must add somethirrg to thc rulcs by which suc:hfincltions can be constructed. We see from this that everrthough g(r. I As the abovc t:xamplr: rthows. Orrclway is to introduce the p or minimalization operator.A)) mav only be partial.1 R.i. If u { J.ncun.3: 0.n. In this definitiorr.3. which is a total functiorr.u)) : 3 . But then. w(l iL$$rrme that g is a total function.?)) : snrallcst 37 such that g (r. But it turns out ihat it .)..there existsan n suchthat s(i)< A(n. : unclcfined.13. I Recu 11. y) : O.then so would g.rsive Functions To extettd thc idea of recrrrsive functions to covcr Ar:kermann's function and other computable furrctions. fhsn U:3r is the result of the minima. Thercfrrrc. ftrr all rl.for r ) 3. wt] get the contradiction S(n): A(n. then therc is rro y € I srrt:h that z I y . pA(S@.).srvp Furrcrronrs 331 If A were prirnitive recursive.n) < A(n.'a):n*u3.$* g(x. proving that A cannot be primitive recursive.lization. but if r ) 3. If we now pick i : rr'. defined by pA (g @.r.. Ler llHr*sNE[U*il*.
rrdorrly if it is cotttputalrle.seqrxlnr:t: irpplicatiots of the poperator and the opera. Agaitt.33? Chopter l3 oF Orul:R MoDELS CoMpurATroN also extends the llrlwt:r to defitre total functions so a. Proof: Fbr a proof.I and lll. we tnerely quote the major resrrlt witlt references to the litcrra. :0 1 I x t ''c.turtlwirere the deta. Usc thc definitions in lixamples 2+3:6.2 to rrrove that 3 + 4  7 and .ble fi. Chapter 13).l.a..recursivera. v ) : 1 i f r > U :0ifr<U Show that this function is prirnitive recursive. Nhft'ilrii\ A firnction is sa. g r e a t e(r* . is .titlrrs primitivc rct:ursiort.idto br: precrrrsiveif it can be constructed fiom thc birsis of functions by a. 4.rrtlQuiilitz (1978. I I'he precursive finctions therrrftlrc givc rrs itrxrtlrtlr rnodel for algoriihmic computation. Derrnis. of compositiotr and if A lunction is trr.s if r : u . .rrirnitivc rccursivc.frtiffift.he luttctiou (t:.ffi 3 .'g) : I rtyt. see Denuing.ffiRq.tt):n :0 Show that this futctiot ifr*u. lili. ir. Show that this function is prirnitive ret:trrsive.rn<:tiorrs. Let / be defined by 'f(n. t.$to int:hrtlt: all computa.ilsmay be fortnrl. i fr : U. Ctrnsiderl. Define the lunction Ill.
2). 11. . does rrot strictly adhere to the defirrition of a prirrritive reclrrsive function. (a) . s = s + 2 f f i ) (b) A (2.tl in Example 13. Ilse it to evaluate A (?. Show that the furtction g (n.37)).g) :u mod(s*1) 16. Cat you explain what vou observe'l 15. W 8.:3). and tletertrine its domain. L Prove the following fbr the Ackermann lunction.u) : intescr of (* .5).u ) : 2 ' + 3 * 3 10. 14.1 RncunsrvnFrrrucrlons 333 rk 5.13.(t.5) and A (3. wlrere n is the largest integer such that x j rrg.3 ffi (c) s (r.i. :2ur3 ffi a) ( c ) A ( 3 . L2. lbr cach g below.nA. anrtl rem. The definition of pre. ''. Use Exercise I to compute A (4.ua:rd re. ( a )A ( 1 .):2"' r u . g). 7. Write a cornputer l)rogram for computing At:kermann's function. 6.'g): n.4 (5.nr ate prirnitive recursivc.u) : x is primitivc recursive. Integer division can be defined by two functions iCzu ancl lern: diu (x.s (r. Sltow the sequencc of rccursive calls in the cornputation trfA (5.1)I fu +t7 part (d) g(n. 13. Show that Ackernrann's functiorr is a total function in 1 x 1.11.q(". Clive a gerrcral expressiorrfor A (4. Show how thc definitiorl t:an be rewritten so that it has the corret:t form.'a): n . althorrgh intuitivcly clcar. Tfy to usc the plograrrr t:orrstrrrctedfor Exercise B to evahrate . Show that the functions d. Show that f ( r t ): 2 " is primitive recursive. :a) : :ra (lr) . r:orrrpute ltakl@.2). I) and A (4.
A Post systeur II is defined by fI : (c.. Wi EV.ffi .. called the nonterrninal constants. s.rxlsorne production nrltls by which successivtl strings can be derived. and C7'.. The prodrrctions irr a Post system mrrst satisfy certaitr restrictions.. where C is a finite set of constants. cir. But there are significant difl'crettces in the way irr which the productions arc applied.. yt € C*.*ytWtUz. . A is a linite set fiom (J*.sf.t eac:hvirriable on the right mrrst irppear ou the left. sutrject to the requirerncrrt tltat atry variallle can appear at most on(:c orl the left.1.Wm.lrres tlte W's ou the right of (13. that is i=t U*rU" t=l we Srrppose have a string of tcrrnirrals of the forrn rru)trzl:tz' ' 'rDnfrn+tt wlrere the substrings rt..1) where ff.334 Chopterl3 OrunR Monnls or Covpurnrton MM PostSystems A Post system lotlks very much like tln utuestricted grammar' colsistiug of an alphabet rr.r.blt:s.l$. thtl st:t of terminal constants. consisting of two disjoint sets (.P) . v. fi ilitftilll\l... Thcy must bc of the form fr1V1fr2.'.'trz : V2. Tz.ritr..!lmlt. and Va.. and th. V is a finite set of va. (13. P is ir"fittite set of uroductions. so that Vfviforilj.lledthe axioms.VrT.'.A. We can then rrrirkethe identification u)1 : V1. rrrirtchthe correspondingstrittgs itt (13'1) arrd'ur1€ C*.1)' Since every .'1. for and substitute these va.
V : lVr. . and production Vt This allows the derivation A =+ ab + aabb. Arnlt.Antt.1) with the identification 11 : A.2 Posr Sysrpus 335 W is some yd that occurs on the left. it is assigned a unique value.u..b}. If you continue with this. Cw:A. irn C 7 : { 7 . In the first step.tedby the Post system II : (C. As for a grammar. we reidentify Vt : ab. . A. Vr : tr. We write this as I1W1I2W2. V. . A : {I}. At : &.Vz.* .r Post system is {attbt' : rz } 0}. we carl now talk about the larrguage derived by a Post system.VsI. I aVrb. leaving everything else the same..13. we apply (13. 4 irrt C7: {a. A:{1*1:11}. and we get the new string lltwiUzwj . rt : A. (II) : {w e Ci.rs u for some zuoe .: . Wt : Vt. : } . and 3r2: b. frnlt + !yUa!2Wi. . The language genera. you will quickly convince yourself that the language generated by this particula. Cu =4.a} . V : {Vr}.. In the second step. P) is I.
mmar G.tcsit.Cr : T. remember that auy recursively enumerable language is generatod by some unrestrit. for every production fr i A of the grammtrr.tiorrs integers. as tlte next theorem shows. t . havirtg productions all of the form t+lt with r. y e (V U T). it can be carried out on a Thring machine. P11).6 illustrates in tr sittrple manner the original intent of Post systems a^r a rttecltanism for rigorously proving mathematical statt:rnents from a set of axioms. Proof: The argurnents here are relatively simple and we sketch them briefly' First. of Irrterpreting the stritrgs of . wc create a Post where Vn : {yt' Vz} . any langua. Tht:refore. since a derivation by n Post system is completely mechanical. derived fiom the axiom 1 * 1 : 2. A. even though they are t:urnbersome for provirrg complicated thtloretns' are general models for computation.['s as ullary representer. C. t]re derivation can be written as 1+1:2=+2+1:3+2*2:4. and with productions V1rV2 + VrUVz.Cw : V. It also shows the inherent awkwardness of such a completely rigorous approat:h and why it is rarely used. It is then an etr$Ymatter to show that a ur carr be generated by thc Post system II if arrd only if it is in the Ianguage generated by G.ted grarnmar G. {S}. A : system II : (Vn. _l Example 13. Given any unrestricted grs.336 Chopter l3 OrImn Moont s ot'' CoupurartoN and productions Vt*Vz: Vt*Vz: V 3V 3V r 7I V z : VtIVzI: Vrl' Vrl' The system allows the derivation 1*1:11+11*1:111 + 1 1+ 1 1 : 1 1 1 1 . The language gcrrerated by this Post system is the set of all identities of integer additiors.gc getrerated by a Post system is recursively enumerable' Frrr the converse. But Post systems. A language is recursivcly enumerable if and only if there exists sotne Post system that generir. such as 2 * 2 :4.
Give the details of the proof of Theorem 13.hine can he viewed this wrr. Ftrr X . These observations carr ller formalized irr tlNr concept of a rewriting system. ffif. since its instantarreorrs description is a string tha. For X: {u.b}}. Show that for every language. Even a 'Ihring rnirt. The program is then just a set of rules fbr producing one suc. what language rloes the Post systern with axiom {a} ancl the following production generate? Vr * I4Vr W 4.e. in addition to the usual reqrrirements. find a Post systern that generates the ftrllowing languages (a) L(a. Rewriting ystems S The virrious grammars we havcl studied have a nurnber of things in common with Post systems: They anl all based on sotne alphabct fiom which one string can be obtained frorn irnother. What language does the Post systcm with VnaVV and axiorrr set {ab} generate'l 8.1).e {a. startirrg from the axiom 1 + 1 : 1. 3.t completely defirrcsits configuration.. i. a rewriting system consists of arr tr.L gerrerated by some Post systern. t . there exists a restricted Post system to generates Z.hstring from a previou$ one.ysterrr that generatcs t: u.{o}.b 1a['*c) (b) r: (c) l. Generally.. What language tloes the Post system irt Exercise 3 gencrate if the axiqm set is {a.13. ab} ? Find a Post system for proving the identities of irrteger multiplication.lphabet E and a set of rules or productions by which .c}. A restricted Post system is onc on which every production r * g satisfies.: {urtu} ffi\ lanb"t!'j . Find a Post s.$ 6 . the further restriction that the number of variable occurrcnces on the right and left is the safire. n : nr in (13.h.3 RnwnlrrNc SYSTTTMs 337 1 . {***.y.6.
But. T Matrix grillrtlnars contain plrrasestructure gramttrars as a special ctt*se in which each 4 coltains exa. . For matrix graIIIIIlars.t rnatrix grammar$ trrrtl plrrasestructgrtl graurrnodels of computa.nd allows atty numtrt:r of specific ca$osirr adtlition to tfie orrcs we have alretr. For dotir. ea. applied. the set of productious consists of subsets P't.. P 3 : 5 ' 1.titlrr of the productiorrs.l strings generated try this matrix gramrnitr is 7:{a"b"c":n}0}.pply the secondorte to the string jrrst created.Zt+'AIt12+!2r.tion. This rnakes it ea. Consider thrl rnatrix grammar P1 : .338 Chopter l3 O'rseR Monnls or.ctlv one production. P2..9 + . sornetimes the use of with arr uurestricted phrasestructure grammar.strilg in E* can produce another. and so on.sy to see that the set of termirur. 9 2 .ils..they are governccl by Church's thesis. thart we cau a.14rly set can also be applied.s a. we must next ir. since tnatrix grrlrnm&rs repre$crrt algorithmic prot:t:sses.chof which is an orderul sequellce 1.. AIso.t r . Note that whertever the first rule ttf 1'2 is used to create arl ar the second one also ha^rto be used. The iderr.. is Whenever thc first production of some set .t r .chieve .ll otlter productions in tltis cannot tr.7 mars trave the same power ir.91.rrtatrix grammar givcs a much simpler solutiotr showtl.pplied.tly etrcountered. producing a corresponding b and c. Pn. We cxlnclude from this thir..lomeur Mqtrix Grommqrs Matrix grammars differ from the grarnrrtars we lrave previously studied (which are often r:alled phrasestructure grammars) irr how the prodrrctions can be tr.' Cotrpurnuon a.nqtht:r is the nature of X atrd restrictions for the applictr. Htrrc we briefly intrt)thrce sonre less wellktrown ones that are interesting arrtl also provide general tnod(1985)' els fbr rxrrnputatiou.is cluite broad a.see Salomaa (1973) and Sa.92. as ExamJrle 13. then the third otre. P2: 51 + 0. A derivation with this grammar is S + S1S2 + a.5I' Sz + bSzt:.P.9rbSzc+ aaSlbbS2cc+ &abbcc. We the first production of 4 unless tr. What distinglishcs one rewriting svstern from a.
I .t u ) : " o ( r ) } .' 'Fr" t*qmFld lS. Every step in the clerivation a.ls identified. In a tlcrivation. productiorrs a. ba .$ Consider Markova. tlNry will be showrr a.ndcontinues urrtil either a tcrminal produr:tion is used or rrntil there arr: rro a. . I Lct M be a Markov a.1} I . A derivation utarts with sorne string . Furthermore. Some of the productions rnay be singled out ir. Tlrcn the set L(M): is tlre language accepted M. by {".sterminal productions.it_.s r + . so L (IV): { r r € { o . Starting a is with a terminal string. the first applicable prodrrction must tre used.! t R.lgorithn the with X *T : {4.\.pplicableproductions. : r t .!.6} and productions ab4.re applied rrrrtil the empty string is Drodrrced..5.t. the lcftmost occurrerrt:eof the substring r must be replaced by g..(. e f. b } .nnihilatesa srrbstring ob or ba.. .I3.ur € x tr.3 Rnwnluuc Sysrnlrs 339 MorkovAlgorithms A Markov algorithrn is a rewriting systt+mwhose productions n+ U are consideredordered. For languageat:ceptance. set ? C X of terrnirra.lgorithrnwith alphabet E arrd terminals ?'.
gcl 11 a certain uense.tlt:rivatioll' everY syrtrbol has to bc Lsystem must bc: rcwritten. = {trLfirl : rz > 0] .Markov a"lgorithms are simply phrasestructure .starting point for a proof of thc ftrllowing theorem that cha'ra'cterizcs tlx: power of Markov algorithms.S.L. This ctr. Lsystcrrrs are essentially Ttarallel rewritirrg systems. An arrswer is ab. grailrrrrars working brr. otre such productiol nust be a..yer.rrgua.rkqv tr. we gc:t a contextf'rt)(t gralnntar that gcrterates the la. 35). tz5b + S.q"ttotl Wnd a' Mtr. But the observation does prtr viclq ir.+'u) (13.r I.thtt llroductions of tr.rkov algorithm f.2) where a € X a.lgorithrn for it.bleif arrd only if thertl cxists a Ma. Bv tis we mean thnt irt each step of ir. Proof: SeeSalomaa(1985.rrdz € X*. Wht:rr a stritrg is rewritterr.rrguageis recrtrsivt:ly errumera. If irr this last examplc we take the first two productiorrs and reverse thtr left nnrl right sides. since it is nqt rlc:irrwhat to do with the Iast produt:tiotr. Lindentrir. r LSystems The origins of Lsystems a.\. 5 i .340 Chopterl3 OrHnn Motlpl. Their clervcloper. . For this to rnake sense.rr of the form 0.pplitxl to every symbol of the string bcfore the new strirrg is geuertrtetl.rc quile clifferent f'rorrr what we miglrt expect' A.p.rrrtotbe takett totl literally. used them to rttodel the growth pattern of certairr organisms. I A ltr.s op Courur.t:kward.
9+S1..o.11Ruwnruuc Sysrr:MS 341 Letx: i\iHl$[Nt$[$W.a.r for 7 : {uzn: ut € {a.t group of procluctions to P 3 : . Wlut language is generated by thc matrix grammar P1 :. It is known that such extended Lsystems are gerrcrell models of computatiorr. Note agairrhow such special rewriting systemsare able to deal with prohlems that are quittl difficult for phrase structure gremrnars. 9 14 . 3. IxEncrsES IFind a matrix grarnura.toi. we carr rna. For cletails.0. m Pr:5'rtr. Supposc that in Examplc 13. What language is generated lrythis matrix grarnmar? 4.a.7 we changc the la^. The set of strings so derived is clcarlv 7.13. productions irrel of the forrn \ixlatA) + u1 wlrer(ra e E and 'J:1y E E* .ke the derivation a + aa + aaaa + a. 9 . . sec Salomaa(i985). with the interpretaticln that a carr bc replaced 1u by z only if it occurs as part of thc string rag. {a}anrr define an L*system. An cxtension of the idea provides the necessary generalization.2":n>0}.g not accept abab'l . Pe: Sr a S L b . In an extenrlrxl Lsystern.S'r*tr.S2..2) are rrot sufficierttly general to provide for all algorithmic r:omputations.b}}. 2.a. 5 2+ .:{u..a. I It is known that Lsvstems with productiorn of the forrn (13. 5 2+ h S z a .a. Why does the Markov algorithm in Examplc 1B. startirrg fiom the string tr.
342 Chopter l3 OrHon Monpls oF CoMPUrerIor'l 5.0. + aa1 o. What is the set of strings generated by the Lsystem with productions o. + o. 7. Find a Markov algorithrn that derives the language L {a"hncn : n } 1}' ffi * 6.1 when started with the strirrg a'l ffi . r z l} 1 } .o. Find an Lsystem that generates tr(oo). Find a Markov algorithm that acccpts L: {a"b"'un* i n} 1 . 8.
tecl.ningin this chapter.n carried out with reasonabkr be efficiency.rd.re can be crca.clical world of software dcvelopment. we rrt)tlrl not only to know that a.srr(:tras processor tirne and mernory spa.we are cuncernecl with thc efficient use of thc r:omputer's resources.nd computirtions. but we also rnust be rrble to construct algorithms that c:rr.id Iittle atterrtiorr to what actually can be expcctcd when we apply these ideas to rual compulers.problern can br: solved in principlc. In the pra.or how relirrble it is. Problerns Lhat can be solved effectively a. For actual c:ornprrtations. we may be rnore concemed with how quickly softwa. At still tlther times. we have so f'ar pa.l significant:c. All we can do is to ftrcus on sorne of the more tangitlle issues and create the 343 .. This is an appropriate starting point firr a theory but cleirrlv of limited lrrat:tica. we may ernphilsize the efficiency with which a us(lr'$ problems can trtl solved.An I n t r o d u c t i o to n co mp u t q t i o n o I co mp l e x i t y rt strrdying algoritlrrns a.ce. how effectively it ca. a descriptive term that will Lregivcn a nrore precisc mea. Wu have been alrnost exclusively corrr:ernecl with questions of the existt:rrc:eor nonexisterrr:e clf aleorithrrrs frrr certain problems. AII this is mrrr*r too complicatcrl to be captured by any abstract thcorv.nbe maintairx.s many facets. At other tirnes. Sometimes. efficicncy ha.recalled tractable.
the primary colcern is the efficiency of a computir. there are a number of sorting rrrethodsso that selection of thc algorithm ilr irnportant. that are simply statecl and errsily appreciatQtl. In this set:tiotr. The nodel fbr our study will be a'ftrring mat:hitre. clearly.. Most of thc tesults tha. computational complexity theory is ilrr extetnive topic.How long will it take to do this task'i" we see If we now ask the questi0il immtxliately that mrx:h more inforrration is needtxl before we (laII answel it. proofs a.t btlfore you can evctt tttake a rorrgh guess of the tirne requirements.little rnore accessible. Fclr the most pirrt. E f f i c i e n co f C o m p u t o t i o n y I. The exact type of Ttrring machirre to be used will be discussedbelow. 2. arrd we must corrcentrateon those that are most fundtr. . The size of tht: problem will be denoted by n. computatitllr. however. For this reason we will allow ourselvesir great deal of latitude. most of which is well outside the scotretlf this text. trut timecorrrplexity is a.rtrtlifficult ald we will clispense with thern hy rcference to appropriate sources. but there irre other f'actors. say.we givera brief overview of some compklxity results. There is thc question of what computer we use and how we write the program.metrtal. we will rnake the folIowing simplifving assumptiorts.et rrs start with a ctlncrete examplc. Tfuere are probably a few more things you carl t]rink of that lleed to be looked a. hoth irt the selectiorrof topics rlrrd in the formality of the discussion' we will lirnit our dis<Irssion here to issues of timecornplexity. Givetr a list of orre thousand integers' wc want to sort tltem iu som() way.344 Chopterl4 CoMPLEXITY Arv lurnonuc:rtot'tto Couru'lA'l'IoNAL appropriate irbstract fiamc:work for thesc:. Our intent heru is to present the flavor of the subject mirtter and show how it relates to what we knqw about languages and autorrrata. in irscendiug ordtlr. If wc have arry hope of producing sorne general picture of sorting. There are similar results fbr spacecotnplexity.t have been tlevelopecl arlclressthe sllirt:c and time cfficiency of ir. Ieading to the important topig of complexity theory. the numtrerrof itenrs in the list plays arr itnportant role in how much time will be taktlrr. 1. Although thc: size of a . Wt: refer to this irs the tirnecomplexity and the spacecomplexity of algorithms. and tha. Sorting is ir sitnple problcrn but also onc that is verY furrdamental in cortrputer science. Also. Thcre are some rt:sults. rnost of these issueshavc to be ignored. In the stucly of t:ornplexity. For our sorting problem.tioll as met$rlred by its time arrd space requirerttettts. rl is obviously the number of iterns in the list. For our rliscussion of computational complexity.t throw further liglrt on the nature of languagesarrd compUtations.
lyze algorithrrn try writing explicit progrrrm$and counting the nurnber of stcps involvecl in solvirrg the protrlem. set txlntrr. First. Orrr first attempt at rrnderstanding the resour(:e requircrncnts of an algoritlun is thcrefore invariably an ord. 'I\rrirrg Wtr think of a ma. we are less intcrcsted in its perforrrrarrr:rr on a sper:ific case than irr its general behavior. wc often get very useful itforrniltion.m and so iltay depcrrd strclrrgly on the prograrrirrr{)r..r:tica. irr r until we either firrcl a.riety of reasons. there is sorrrc vtr.best we can hope fbr is that thc T\rring machine analysis is representative of tlxr major aspects of the reallif'eperformance.. we coukl a. this is rrclt ovrlrlv useful.set of rz numbers ir:r1r'21.tch or we get to tlrc la. we rrrearrthat the computation firr any problern of sizc n can be tlclrnpleted in no rnore thirrr 7 (rz) moves orr rrome Ttrring rnachine. which we corrrtr)ar(r successivelyagainst :r:11.tr:h tlte first cornprrrisorror on the last. 3. specific typc of T\rring rnachirre as il computa.irrs r. we give some mea. Our inrtrretliategoal will therr llc tcl characterizt:the time requirement of a.na.rge.14. frorrr a pra. By saying thrrt a. TIrc. can geuerally relate it we in some way tcl rr positive integer. we carrnot predict on lxrw much work is involved.chine as making one rrove per tirrrc rrnit. ma. but we know thtlt.rclruNcrr CoMprr.. tsut. In spite of the apparent inforrnality of this approach. Unlcss the set is organized in some way.Ftoru 345 oF problem is rrot rr. problern il^s fhnction of its size.rch.. which may difftlr considelably fionr how it does orr a T\rring machine. We are particularly conctlrrrttdwith how Lhealgorithm behaveswltt:rrthe problem size irrr:rerlres.T21. the number of operatirlrrs perforrntxl may vary with the srnelll details of the progra. in the worst case. rrsing a Thring rnalchineas the cornputer a model. O. we wtrnt to study how the cornputational rrxlrirements grow with the size of tlrc problem.cterizecl.r'A.rofmagniLude analysis irr which we use the O.lstandpoint. After scttling on a. Irr analyzing atr algorithm.ningt. for cxa. Second.e. nnd () notation introdrrr:ed in Chapter 1.. Norrnally. wc irre itrtercstcd in how the algoritlrrn perfbrms in the real world. the sirrrpklst nlgorithm is just a l'inear seu.tion is the nurntlt'r of moves nade.riation.lwaysso easily c:harrr.1 Epr. in the set of all protrlems of a given size.stthe resourrxlreof quirenents grow as rz becornr:sltr. so the tirne taken by a courllrta. Becaust+ this.. lbr a va..st element of the set. Since we may find a rrrir.tional model. As stated.the asyrnptotic growth rate of the time complexity. and a key mrrnher r. Givr:rra. First. Here w(r irre irrterested orrly irr the worst case that hils the hight:st resource requirclrncrrts.mple. determine if tlrt: r. computation has ir timecorrrplcxitv 7(rz). tire prirnirry qrrestion is with how fh. wcl hirvel .othe concept of time for a Thring machine.
I LL supposc you are givetr a set offl. Irr Example 9.n tlx:n say that the titnecomplexitV of this Iirrear search is O(tl). In ilraking this analysis. or even bt:tter. But. with a twotape machine w() carr use a different algorithm. the equivalence btttween the various tVpes of T[rring mAchifles no longer holds. 2. rrutttbers l.insarry duplicates. But whetr we tnake cornplexity an issue.irD alld are asked trt tleterminc whether this set conta. Is the algorithm as efficient as possible? 3. this tirne dctcrmining if the set containrt arry triplicates.1.ltclrt what machine tlfs is run on t)r how the rrlgorithm is implernented.mple 10.gesto avoid some of the tedium involvtxl in using thc standard Thring machine model. Review how the r:hoice of algorithrn afl'ects the efficient:y of sorting.rgrtedthat the various types of T\rring machirres were equivalerrt in their powt:r to solve problems. A look at that algorithm will show that for ut : unb" it takes roughly 2n stepu to rnatch each a with the correspondirrg b. ffiffi o T u r i n gM o c h i n e s n d C o m p l e x i t y In Chtrpter 10 we a.ssurnptiousa. we rnacleno specifit: ir.riscltrs. as we later indicated in Exa.346 Chopter l4 Ar'r Inrnonucrrlow ro Coururhl'loNAl CoNrRLuxtrv to rriake rz comptr. We first copy all the a's to the second . singltltape guage 'ftrring machine for the lan 7={anb":rr. Repeat Excrcise 2.^r1frz1.We ca. Therefbre tlte whole comprtation takes O (n2) titne.}1}.4 wc t:onslructed a. O(n). (a) Suggest an algorithrn and find an orrlerof*rnagnitude exJrression for its tinrecomJrlexity.'. This allowed us to take whattuvcr type was most t:orrverfettt for an argurnent and even use programs in higherlevel r:orrtputer langua. (b) Examine if the irnplernentation of the algorithtn on a T\rrirrg rnachinc affectrr your conclusions.
14. then the exhaustive st:arr:h method has compklxitv O (n M) .lgorithrnfor this problem proceedsby sirnply guessing which sequenc() productions is applir:d in the derivalion of rr.2 arrcl6. true or firlse. The simplest boolearr oprltltclrs are or.\productions. T'he nore elflir:ientCYK algorithrn him urmplexity O ("'). we have an O (n) algorithm.rlrl.rnsta. then rnatch tlrcm tr. Both the copying arrrl the rrtatr:hing can be donc irr 0 (n) time atrd wc see that a twotape malt:hine has tirnecorr4rlcxity O (n).3 we cliscussecl rncmbership problem for t:onrextthe free larrgrrages. I f.1.n expressiorts. where M depends otr thcl gramma.instthe b's orr the first.1 Tape 1 Tepe I Tape 2 (a) Initial tapes. If we of work with a gralrlllrirr th.tionis essentially l. which we will denotc: tly I and 0. Botlletr.n operators are thcrr used to combine booklan rxrnstantsa1d variafles irrttl bclolea.tumpl* l4.r. A nondeterrnirristica.rrt variablc is one tha.nt role in complexity thexrrv.. Both thcse algorithrns are cletcrministic.t has no unit or . so I Fr(qffiFl$ l4.l:.. tirpe. which plays an irnporta. respecl. A logic or boolcla.ga.S Irr Sections 5. Tape 2 (b) Tapes aftcr copying of a'. The situation before and afher the copying is slrown in Figure 14. denoted by V a.t can take orr exar:tly or twtt vtrlues. the Ierrgth of the dtlrivtr.ively.2 TuRtNc MncHrruns AND CoMpLEXrry 347 Figrtre 14. .s.rrddefined by 0V1:1V0=lVl:l 0V0:0.If we take tlx: Iength of the inprrt string tt as the problr:m sizen.n .4 We uow introcltrc:c the satisfiability problern.
titlrr a variable.rcha.gat'ion... t:0..hm. make e2 A dr:tr:rrninistic algorithm for the satisfiability problcxn is easy to disftll cover.348 Chopter l4 Cotr. that is. lclok irt rt.ltrcof citch r i and then evir..tplnxlrv Ar'{ IrurRonucrIoN l'o CoMPITTATIONAL by tlrtl rr.V..lltcs the variablesrr .blt:btlcause every a.9pstands for somtl variable or the tregtr. this t:xhiruslive approa't:h the exprrlssiotr..V.riahlels that will tnake ttrtl value of e true. rrr.he variables flr irlrd z2 will fa. . 2 ) of wlrere each El. c2: (r1Vr2)AzrArz lbr is not satisfirr. 0: 1. Rtr rr specific case.ti. Tlre terms ti..nd artd operation(n) definecl : 0A0:0n:L 1A0:0.r) trr() t:reated by oring together variables tr.rtl2t' such choices..s..hrir. 1z\1:1.. is there an assignrnent of values to tire va... If e is satisfiAgaitr. we createexpressions (::ftAtrA.ssignrnc:rrt l. fi2 : 1. iu exprt:ssitlrts corqiunction normal form.. The satisfiability problem is therr simply sta'ted: givcn atr expression e in coniurrctive norma. Therassigntnentrt : 0. As irr Exarnple 14.lsc.rt frotrt variabl€s 1r2. by dettotedby a ba. dtl rr<ltktrow of any norr()xl)()rtential deterministit: algclrit.srr. is essentially irrr itble..l fbrm..$pr ( r4 .ilrl their Ii :. t3 : I na.. (14. t'r. approaclt simplifics rrtatlers. iras r:xporrentialtinre compk:xity..3. starting with fi1.V.Tz...fk nrrgation. the nondeterurirristic: 'flils e. Wu take:all possibleva. However.tttllle.kes e1 trtrc:so that tllis expressiclrr is satisfiable.. alrd evaluate Siuce there a. In We considerrurw boolea.lgrlritlrrn whose complexity is exponetrtial and a lirrear uotrdeterwe rninistic orrt:.r. On tht: otlter haucl.At7..t' e 1: ( 7 1 V r 2 ) n ( r 1 v r 3 ) .. urrlike tht: llrtlvious exa..and definecl Also neededis ne.tc: O (n) algorithrn. we hirvt: ir deterttriuistic exhaustive sea. ff2. I .sm. this fbrm. . we guessthe va..
5 .wotape rrrachirre in timc O(Z(n)) catr be perforrnerl orr a standarcJ'l\rrirrg machinc in time s.n wc. nrultitape T\rring rnachinrl iLsorrr model for stuclying cornplexity issutls.. For the tlxur(:ise$in this set. we will rrse a.b}.t is thc Lrest. offline 'llrring machine in titre 0 (T (rr)) also r:an be perfornred on a standard Turing rnar:hinein timc O (? (n)). 6. bc morc precise and rdairrr that T (n):(. lihow tltat any t:ornJlrtation that can be performcd on a t. I .1 suggeststhat a. Rewrite the boolcan cxprcsslon (rr Arr) Vnil in conjunctivc nornral lbrrn. Exa. Wha. Find a.rnple 14. For this reason.). lineartimc algorithrn lor rnernbers]rip in {tlur: ru E {a. o ("' (n)).14.) (tl2) ftrr the first t:ase. Dctertuine whether or rrot the expression (rr vr.2 'I'uRrNcl MecHrrues erur Col.lgorithrns for a rnultitapc rnirr:hine may be reasonably close to what wt: uright use when we prograrrr irr ir rxlrnputer language. ()a. Show 4. 7.') antl the $et:onrl O (n. you could expect on a onetape macirinc? a) Show that any cornputation that can be performed on a singlctapc.t the issue of deterrnirrisrrr v()r$llti noncleterminism is a partir:ularly r:nrr:ial one.2? .nd tha.and T'("): O (n) for thc scconcl'lIIow this strcngthen the argutrretrt in Exarnple I4.2 we claimcd that thc first algorithtn harl tirne trrmplexity 0 (rr. that any txrrnglrtation that ca. Irr Example l4. vrr) n (r1v:r2 vn 3) n (nr vnr vn3) is satisliable.tprnxrlv 349 These exarnples suggc:stthrlt complexity cluestions are affectcrl try the type of Ttrring tnar:hirrt: w() ll$e a. assume that the Thriru rrrirr:hirrcsinvolved are all deterrrfnistir:.n be pcrformcd on a standard Tfring tnachitre in tirne O Q (n.} using a twotape Turing machine.)) also can be performcd on a Turing machine with one semiinfinitc tapc in tiue O (T (n)).
): O ( T . Whtr. is acceptedinO(T (z)) moves.r:xIrv Longuoge omilies nd Complexity losses F o C language In the Chomsky hicrirrr:hy for language classifica. DTIME (ri (")) c DTIME (Tz(n)).nguirgc.reobviolrs. To do so.ir ritsof'1.tion. tlris implies tha.ss arttomir. wc first clefitre the time complexity of tr language.L in time T (n\.ssNTIME (f (n)) if there t:xists a rrorrdeterrninistic rmrltitape T[rring mar:hine that atrepts L in tirne T (n).r.350 Chopter l4 Aru IN'r'Rot)uc'I'Ior'tro Co\.ssifyirrg typc but corrsidertiute complexity to usc a Thrittg tnachiue of a 1ra. rxlrnplc:xityclassessuch as Some relatiorrs between theser DTME (r(')) and T 1 ( n .t a.L is said to be a member of the cla. Another way of t:lir. . .til is deflned by latrguages is the natrrre of its tt:trrporary storage.t for ()v{}ry 'u € L. If M is trondeterministit:.. we assctc:iate of of fa. 2 ( n ) ) irnplies c NTIME (7(")).milieswith clir.rticrrlar rr distinguishing factor. we take in progressively morr: languirgc:s. n.t wt) r:arr say is th.Lis said to be a tnember of the cla.L Wu sa.stlrtl order of T (z) increases.ssos inrtornata.ge irr tirrre T (n) if every ?u in. wltere each cla.L with lrl < r.IIUTATIONAL Corvet. ir lolllll.ssDTIME (f (")) if there exists a deterrrrinistic mrrltitape Tirrirrg rnachirre tltat accepts .1ini. A lrrrrguage.ythat a Thring machine accepts a langrrrr. t}rere is at lea.nrx."f'i. llrt frorn here the situation gets obscure quickly. A lir.st one $eqlrenctlof rrrovesof lcrrgth O (Z (ltul)) that leads to accepta. .
and so on. namely the existence of a computable / (z). does not accept .trzt. IslL. Whcn wr: have i. it operate orr to fbr /(lr.3). is in L. must be false. we the is find M.l) steps} .. no matter how rapidly the cornplcxity firnction 7(n) grows. r:onsider any ur f tr and compute first / (lurl). etthough this may be of thcorctical interest. this is posuible. p.r1t2. wit}r all strirrgs in X+ arranged in proper otder'u1.AIso. so is recursive.nd14.l T\rring computable function. 1}.L is recursive.L? If we clairn that'ur1.14.3 LalrcuecnFauu.. DTIMn (t**t)..rli in f (lur. that there is a language in DTI M E (na) ttrat is not in DTi M E (nB).rps enn CourlExrrrr Classns 851 For everyintegerA > 1.1 rr. To see this. thert: is some M6.L is not irt D'I'IME (/(")).l Ttrring computable function / (n) srrt:hthat every recursive larrguagcr in DT I M E (f (n)) . for example..'in the sequence101.. there is no limit to this. This will tell us whether or not zu is in L. The irurbility to resolve this issue is a typical diagonalization result and leads us to conclrrde that the origina. Mz. Since.. Wt: ca. # (14.we get a contradiction if we assume that ** # L. then M6 accepts trrp in /(ltrr6l) steps. There is no tota. Supposeit were.n then define the Ianguage 7 . But this corrtradir.{tut: M..n be accepted in tirne z? for which there is no linear merrrberslrip nlgorithm. DTIME ('"*) . that there are languagcs in DTIMH (n3) that are not irr DTIME (n2). is Proof: Consirlt:r the a. This is becauseL € DTIME (/ (")) and every w e L is accepted by M6 irr time / (ltrrl).lphabet X : {0. there is alwrrys sornething outside DTIME (f (")). By assuming that / is a tota. This gives us iln inlinite number of nested com* plexity classes. I Theorern 14. Proof: This follows from a result in Hopcroft and Ullman (1979... Conversely.l) steps. it is not clear that such a result .t.ln in.L is recrrrsivt:. 299)... Assume now that the function / (n) irr the statement of the theorem exists. In fhct. and let.2 allow us to make various clairrn. We next find the position i of u. But we carr n()w show tha. assumethat we have a proper ordering for the T\rring rnachinesit M1.3) We cla.im that . r The conclusion we can clraw frorn thiu is that there are sorne larrguages thrlt ca.ts(14.l assurnption. This is also possibk: bet:arr$e sequence in proper order. such that tr: L(Mn). We gct cvcn more if we allow exponerrtial timt+ complexitv.
ngrra. 2.llvgut:ss thc rnirldle of this string. I It follows fiorn Exarnple 14.ch syrntlol orr tape 1. Thert:ftrre Lanc. as we calt recognizt:strirrgs in this langua.ngrrilges. e * o m p tie .). (n:]) . Clearly all of tlre steps r:an bc dorrein O (lul) time. Tlrc argtrtnetrt given there r:tlrr lltl rrsed for eveu rnore complicated la. Compare the symbols orr tape I and ta'pe 2 one bv tlrrtl. C DTIME (rt). We havc rrlready establauguage {a"b' : rr. Ilvery regular language can be rerxlgrriztxl by a det. We leave the details tls iln t:xercise.mplc:s tlrrrt give sorne of the mrlrc cltrviousresults.} is fi NTIME(n. Ctlpv the secoud llart to ta. 3.352 Chopter l4 Arq INrnotrtrc'tIouro ClonrurlTtoNAIColtplnxlrv ha. Ct4ry tlte itrput from the inprrt filc to tape 1. Brt. € noncontcxtf'ree by This is straiglrtftrrward.> 0} can be Iislxxl in Example 13.la. .b}. At this point.) if wtr c:arrdevise au algorithm for flnding tlx: rnitltlle of a string in O (n) tirntr.r {n.2 that [. I f *innpidlilUildlt The language7': {wu: rr.ctical sigtrificance.ge thtr algorithm 1.). $ .s any pr:r. DTIME (n) includes mtrc:hrttore lhatr L6a6.gt: DT'IME (na) might bc. we have no t:hrtl wltat the fu r:Irirracteristics a. Actua. TIds can be done: we look at ea. keeping a count on talrc 2.lly. so L e NTIME (n). we cirrr show tltal L € DTIME (n. We will krok irt sotne simple exa.cp { DTIME and Lar E NTIME (n).l to tlrc lcrtgth of the input. but couuting only every seconrl syrrrbol.pe 2. Nondetertninistictr.7 that tht: c:ttrrtextfree rclcogtrized in time O (n. We can get a little of rrroreinsight into thc matter if we rela'tethe c:omplcxity classificationto tht: languages in the Chornsky hierarchy.ermittistic finitt: autornatorr irr titrte proportiona.
lready seen. hrlwever. . say.M. tha. I Frorr these exa. 1. Note.t we ca"rrnotclaim from this that Lcs c DTIME (nM) becausewe cannot prrt arr upper bound on M..4}+} is in DTIME(n).ges that are not in NT'IMfl (2"). Show that 1. L : {wwdw:w € {u. . : {www : tr € {a.5.nxlly Cl. Exhaustive searc. Show that thcrc are langutr. such as that betweert DTIME (nft) and DTIME (rzh+l). where M clcpcnclson the granurrar. limited number of productions are applicirtrkr. and it is not a priori clear which model is most appropriate frrr ir real computer.. lct us ignore some f'actors that are less irrrportirnt. Therefore.R1G.Lcrp are in P.sses nrul NP P 353 Consider now the family of contextsrlrrsitive languages.14.mples we note a trrlrrrl: as 7 (n.lr parsing is possible here also since at every step only a.l. 4. Lcs arcl txlvered.. and . This leads us to consider the famous corrrplclxity c:lir^rs pUDTrME(nt).rchy and the corrrplcxity r:lilssesis tenuous and not verv clear. S h o wt h a t .vea.timecomplexities witlt differcrrt growth rates a. Lcn. ftrr example by removing sorne urrirrtercsting distinctions. since some of it depends on the specific modtrl of TtringE machine we have:(rr.L466.ppearsto lre unproductivrl. 2 .) increases. DTIME (n) and DTIME (n2) is trot fundamcntal.ttempt to produce meaningful hicrarr:hicsvirr. We (rfirrargue that the difference between. more and nrore of thc fermiliesL. rl) I This r:la. without arry rcgard to the degree of the polytrotnial.ny tapes).ssincludes all languages that are accepted by some deterministic T\rring machine in polynornial tirne.how ma. But tlte connectiorr between the Chomsky hiera. 3 .4 THn Coupr. b}+ } is in DTI M h) (n). every string of lcngth n can be parsed in time n.g. As we htr. Cotrplete the argumcnt in Example 14. TheComplexity P Closses ond NP ffiffiffi*ffiffi Sincc the a.
ctnbkt.l prcibletns.lls Iem tra. but whtlt is rrot known is if this c:orrtainment is proper.ppear$to bc fundamenta.too)f While the CookKarp thesis t:a. To explore it.particularly in the clilss P. no one has yct fourrd an exanrple of this.c:table.tirrgproblems we c&n frotn l. and onc thnt is not is said to be intra. thrrrc is rro realistic hope of tr. Ohviously PgNP.lling a. c:ornputerscientists have ir. In the CookKa. To utrdersLandthis bettelr.ble.dcirr what is generally called thc CookKarp thesis. The jrrstificntion for tlte CookKarp thesis sccrnsto lie in the empirical obscrvatiorr tlrat tnost practical problems irr P are h DTIME (n).a clear distinction exists between llroblems in P and those not irt P.ttt:rnptedto put the itlea of intracta.rrguirges NP that are not irr P. rrny rxrrnputation that is not in P hirs tirtre complexity that grows faster tha.roduce NP : ! I'rzrua ("t) . Certain comprrtir. this will be excessive fbr large ?r.l.But what a.bility orr ir formal basis.n ilrry llolytrornial. while in prirrciple cornputable. or DTIME (rz3). say rz ) 1000. DTIME (rf).tiorrs. firnctiort like 20'1".354 Chopter l4 Couplnxtrv Ar'r lrutRoouctIoN To CoMPITTATIoNAL Since the distinction bctween deterministir: irrrd noldeterministit: txtrnplexity classesa. At the root of this is the question whcthtlr or trot P:NP. cxrmputer scientists have introducerrl a variety of .1ly cut. and wc rrriglrt feel justified in ca. I'he interest in these complexity classtltt.bout problcrns suclt a probtlrat are il D'I'IME (n. prirctical algorithm.nnot'l The answer is not clearwork with rea.prohlcrtt with this complexity intra.rp thesis. Among practica.ctable.have such prrtatiorrs. Even for a. This is one of the funda. irs well as on supercomptrtt:rsyet to be designed. One atternpt to defile the term intractable is ma. Is the CookKnrp thc:sis a good way of separir. Such problems are sorrxrtirncscalled intractable to irrtlicate that.tempt to distingrrish between realistic and urrrcalistic cotnalLhoughtheoretictr. while thosc outside this class tetrd to havc exponential crrtrplexities.hose we ca. P The study of the rela.llvpossible. wtl irlso int. come$fiorn arr at. rlrrc srrrely cannot do much with it even for small n'. Whilc it is generally in believrxl that there are sorne ltr.Iistic:a. Obviouslv.mental urrsolved problems iu the theory of cornputation. a problem that is in P is called [racta.tiorr between the complexity cltt^sscs and NP has generated partir:ular irrterest alnoltg computer scientists. high resourcerequiremrlrrtsthat in practice they mrrst tlc:rejected as unrealistic on existing computtrrs. and its require:trcrtts will increase very quickly with the probletn size.
rl a.chineby which an1' u1 irr thr: alphabet of "L1can be transfonned irr polynomial time to a tu2 in the alphabet of . an NPcomplete problem is one that is as hirrcl as any NP problem and in sorrre sen$e is equivalent to all of thern.T tinte algorithm f'or any NPcornplete larrguage.14. Florr this we see that if . The staltementtha.T. Orre of them is the idea of an NP<:omplete problern. We e'cocle specific instances as a string that is actrrpted if and only if the expressionis satisfinble.sa language problem.lplnxrry Cllessns P aNn NP 355 related concepts and questions. Definition 14.Llis said to be polynomialtime reducible to sornelirrrgrra. therr cvcry la. I In addition to the satisfiability problern.L. Similarly. This problem is NPcomplett:. rr. if Lz e NP. a' be vir:w. This puts NPcornpltlterrttssin tr.L1 is NPtxrmplete ruljn".ngrragein NP is also in P.L2 if there exists a determirilstic T\rring ma.*. It follows easily frorn these definitions that if sorne .central role for the study of this tpcstion.'1lH:til.t the satisfiability problem is NPcomplete is known as Cookts theorern. llM A ltr"nguage. For all of thenr we carr lirrd txponuntirrl . Wha. Loosely speaking. therr ."*j#l"ll''.ltime reducible to .'r'J::'::.4 THU Col.ge .L2 in such a way that w1 € L1 if nnd only if w2 €.4 A Iangrrage /.discussion of which t:irrrbe fcrund in Hopcroft and Ullman (1979).t this rrreans ha. tha. a largc rurrrrberof other NPcomplete probltrms have heen found. arrd if L2 e P.t is P:NP. thcrr trr E NP.Lr is polyrromialtime reducible to . is said to be NP*corrrplct<: if I e NP and if every .L/ e NP is polynomia. L2.L1 e P.sto be explained.L2.
3. and compile a list of problems that are NPcornplete. the question remains open. These failrrres ltlad us to believe that prohably PINP.nxIrv ArrrlrutRopucuoN ro CoN4r. until someone proves that no such language exists. Consult books on complexity thcorv. 1. Is it possible that the question P : NP is undecidable? . then L. alternatively. trut until someone produces an actual language in NP that is not on P or. but for none of them has anyone discovered a llolynomialtime algorithm.Lr is NPcomplete time reducible to tr2. Prove the statement that if a larrguage . ** and polynomial 2.356 Chopter l4 Cotr.lpr.urATIoNAL algorithms.: is also NPcomplete.
Let' Z : : 51 U 52. Then. which means that r catrnot be in 51 or in 52. that is. This can be proven by an induction on the rrumher of sets.. Z :5.+r. 6.. By the stattdard U DeMorgan's law M:E)8.91tl 52. Then r f 51U. 5z completing the ilductive step.92.n..1 5.. Suppose n . 5'.+t. n Fr rt .+r Z l) 5.q Solutions nd Hints for Selected Exercises Chopter I Section 1..if z € . U 51".+t.€ 51 U Sz..5'z..9r U. With the inductivc assurrrption.9*. that is r € . 357 .+l : Sr f'l fi "'15' I F. wc mrrst show that an eletnetrt is in the first set if arrd only if it is in the secorrd. Conversely.9r t'l .U Sr.92. thc rclatiorr is true for up to rz sets.. Therefore ZnE. that is r e 5r U 52. To prove that two sets are equal. then r is uot in 51 and r is not in 52..
J. the result holds for all strings.: ?'tyTL2 where hoth factors are less than n.. Take now a string of length rr { 1.. (c) Since 21.14 were ratiorral.::_" By indrrction then. n! : O (n"). If z is a prirne. Suppose that 2 . Many string identities can be proven by induction. contradicting the fact that V2 is not rational..92. a rrecessaryand sufficient condition is that the two sets be disjoint. Srrppose now that St f Sz and that there is an element r in Sr but not in. ):r#. By the inductive assumption..51U Sz) .. Thcn r e Fz so that SL1E1 + fr. nl nnL n n n n n n is the product of factors lesstlurr or equal orre.n n1.T. it carr be writterr as the oroduct R. L2. By induction.. they both can be writterr a*sthe prochrct of primes. Therefore. Suppose that every integer less than z can he written as a product of primes. Suppose 5r : 52.. An arguilrent by contradiction works. If r is in .then u is not in (._ ..Sz.5z. 27. Supposethat (zu) : ufuft for all u € E* and all u of length rz. there is nothirrg to prove. if not.2 R 2. Thc c:ornpleteexpression carr then not be equal to the empty set. Then * (.. . 1 5 .358 Alrswnns 8 . Section 1.. Ttren n 2rt: WL gives \/t = 2 m . Becauseof this..91and u is itr .say ur : ua.il". Then 51 l'l$2 : ^91 5"2: ^9rfl^9r = O and tlte entire tl expression is the empty set. and so can rt.
baa. 11. (ru. The secorrdcan generate any number of ats and b's in atry posititln. This shows tha. t:ilrr eaclr of which is iri L. 13.then add an arbitrary nurnber of a's ancl bts arrywhcrtl. (a) Generate one b. Sitrce abaabu.uba. Tlrc first iu cxrvered by St + aaaStl(t t thc sec:ondby Sz aaaSzlao.crsus S59 4. baaaaabaais in L*. bu decomposed into strings ab.e.o.e.. finally a. Similarly.'s.)} 3. S+aA+abS+o.\ 14.u.Solu:uorvs nNn llrnrs pon Snlncrnr Exlln. However. (d) Wc first go'nerrrte three c. I'his leads to the easy solutiorr SA aaaA aAbl.aAlbAlA The first productiorr grlrrcratcsthree a's. b}* rrs klrrg ir. there is no possiblclrlt:r:ompositionfrrr haaaaabaaaab.aa. .ndI'rirlmod3 :2. bAlA 13. S + AbA A + aAblA B .ab.t the grammar can geuerate atry string trr € {4. 10.baA+abubS fiom which we see that L(G) : {(ob)" : n.> 0} . (d) The arrsweris ca. thc string is in tr*.sn. S +AaAaAaA A . this so string is not in I*.lrulmod 3 : 1 rr.srrrarry more b's as needei. then an equal number of ats and bts. TIte two carr be t:orrtbined irrto a singlc gramrnar by 5 51152.sic:r stxrif yorr notice that to Lt : {1""l:1b"' : ttt } 0} . (b) The problem is simplified if you break it into two cases.
Section 1. The a*s label of the state is then produ<:erd output later. integer + sign magnitrrdtr sign + + llA magnitude . We remember inprrt by labcling the states mnemonically. Remembering t:an htt done by labeling the state with the appropriatc irrforrnatiotr.360 ANSWERS ro. Consider then a $tring u € L. Most real crompilcr$.$'r. This situa.If it starts with a b. The following solution is partial.digit I digit magnitrrde digit + 0l1l2l3l4l5l6l7l8le it Tlds can be considered an ideal version of C. If this string start with an a. The automatort lus to remember the input for one time period so that it r:arr bc reproduced for output later. then it ha. . When a set of three hits is dene. it can be derived hv S r ^9r5.13.L1 be the language in Example 1. Let . \)uh 10. but the completion shtiulrl trc obvious./ Y . though. 7. (a) We can use the trick and results of Exantple 1.3 1.sthe fbrm ur : o:tltt where trrl e .Lr.13 and modify that grammar so that the start syrnbol is 51. /)uo . place a Iimit on the number of digits. ( ^) +\ r Vi\ b/a 7q \ l )\.tion can be taken care of by S + rr. t ) /\'' . ir^s puts no lirnit on the Iength of an integer.*. w{: produce output and return to the beginning to profft$$ tlx: rrext three bits.
pcteD ExERcrsEs 11.the transducer must rememberthe two precedinginput symbols and make transitions so that the neededinformation is kept track of. In this case. .SolurroNs ANDHrNTspoR Spl.
The solutionthen is quite simple. putting the dfa in an acceptingstate if the suffix is bb. 7. t)tltl a. A { lf+il A ll+ilA ti+t \/o o. \/" \y'" \y'd \*J l (a) The first six symbolsare checked. the string is rejected. . three a's. (c) Break it into three caseseach with an accepting state: no o'il. A fourth a will then send the dfa into a nonaccepting trap state. A solution: \t +ti n' \n' \a' \n' \n"o . t t I t ]t+il . If the prefix is correct.362 ANSwERS 2 Chopter Section 2.If they are not correct.1 2. (a) UsestatesIabeled with ltulmod 3. two a's.we keeptrack of the last two symbols read.
(a) Count consecutive zeros. (}OrO:{} Then put in additional transitions to keep track of corsecutive zeros and to trap unacceptable strings. 9. .Soluttor'rs aNn Hnrrs FoR SELECTEI ExnRcrsns 363 (d) For this we usenine state. with the first part of eachlabel no (ru)mod B. The transitionsand the final statesa. to get the main part of the dfa. part rz6(tr)mod3.re the second then simple to figure out.
arrrl so every accepted string ha. Iirrite trumber of stcps.:4}.tt:.helanguage is finite. (h) AIso by cortttaclictiou. Suppose G1a has no cycies in arry patlt from the initia. But to generatean arbitrarily long wa. 2 1 . The solution is a little long but rrot hard.14ha.l sta.sto berof firrite length. (a) By rxrrrtradictiorr. with atr acceptedstring. then complerlt:rrtthe solution. A uartial sketr:h of the solutiou is below.berlrxl a finite language ca. But this irnplies that t. Ttrl tlirsiest way to solve this probll:rn is to cotrstruct a dfa.n \ 1 3 .lk ha*sa.tc:to any final state. Assutne that (J. {.nnotc:ontairra.lkLr.s$otrc c:yclein a path 'Wt: carr then use the cycle f'rornthel irritial staLeto sotne accepting utir. fbr L {u"' : n. .rtritrarily lotrg stritrgs. Then every wa.364 ANSWFTRfi (d) Here we need to remembcr all contbinations of thrcc bits' This requires I states pluu sorrrcstartup.
A foursta'te solutiorr trivial. r. but it takesa little expcrirnernting get is to a tlrrr:rrsta.rnr FixnRcrsns 365 24. T'he strirrg a. Section 2.tj'uor'rs ano Hlr'r'r's noR Snlnc.2 4 . q r } . a ) : { s o .rting sLatep6. d(*Q r . Introduce ir sirrglesta. Tltcre il.Sot. 17. Then add a transitiorr d ( P 6 ' .te Herr:is one answer: one. This is the kind of problem in which ylrr jrrst ]rave to try diffcrcnt ways.lent to the origina.re many diffcrr:nt solutions. rerrrtlvcster. d * ( q o .L A a / *V\'_/\i\ / r n \ No.s three diffi:rent symlrols arrcl tlxrre is no way this r:iln he accepted with f'ewerthan tlrrcc utates. 1 5 . Prohalbly rnost of youl tries will not work.e t . Here is one that doos. Next.l one. \ )= g u . t r ) : { q u .hclia. Here is one of tlxlrn. . e z l t . It is straightfrlrward to see thnt the new nfa is rxlriva. 7.rting state status frrim Qe.
Introduce a nonaccepting trap state and make all undefined transitions to this new state. it cannot be made for dfa's. It is a sirnple matter then to argue that if d* (qs. Since this construction requires trtrarrsitions. Solution: Section 2.r) : {pf } afber the modification.3 2. Then make ?7 the only final state.366 ANSwERS 20. then d* (qo. Introduce a new final state p1 and for every g € F add the transitions d (q. as can be seen by constructing dfa's that accept {I'o}. Just follow the procedurenfatodfa. . Generally it is impossible to have only one final state itr a dfa.w) e F originally. This givesthe dfa 7. so both the original and the modifies nfa'$ are equivalent.A) : {prr .
..... Getting an answer requires some thought.= + { tfr acceptsw. Then the nfa .Sol.ecrpo ExeRcrsns 367 8. This is not easy to see. For example. This can be done by doubling the rrumber of states and adding O or E to the labels. \r' tl accepts L. One stllution is _ffi a.b h lL.. Suppose that . The trick is to use a dfa for tr and modify it so that it remembers if it has read an even or an odd number of symbols...w^}.uttolrs er'ro HrrrTS ron Snr.wz. so the language is regular...L : {trrr. 14. if part of the dfa is ..
. originaldfa accepts eaen(L). It should not be hard to seethat the new nfa acceptschopz(L).\aa'.. you shouldbe able to convinceyourselfthat if the With a few examples the a4o.\): Q.368 ANSwERS its equivalentbecomes Now replace all transitions from an E state to an O state with . reading any twosymbol prefix u. Although the constructiorris plausible. (b) introduce a new initial state p0 and add d (Ps. Suppose have a dfa that acceptstr' We then (a) identify all statesQ ttrat can be reachedfrom gs. and therefore we 15. that is Q:{SeQ:d*(go.u)*qi.2&3o'4. new automatonwill accepttra2...\transitions. a complete answer requires a proof of the last statement.
denotes arrystring of 0's ancl1's.SoLurroNs eNn HINrs rron Sunc. 6 .o) # F tr.2. tnffi. By contracliction.uu) e F.is not minirnal.) f F and d* (q. % f F and qa € F. $o does(0 + 1).fbllowed by the requisite nunrberof b's. Ncxt.use ((o+ t)(0+ 1).ndd* (qo.t'b"". are indistirrguisha. Solutiott: uaa(1. d* (Sr.a) € h. Sirnilrr. with either rz { 4 or rrz> 3. Assume that il[. T}ren use the procerhrre marh to firxl the indistinguishable pairs (go. (A + (r:) The complernerrt the language 5(a) is harder to find. (Ienerate or rrrore 4 a'$.gr).d* (qr. Chopfer 3 Section 3.* b + bb+ bblt). qo ancl g.3. Yes. Continuing this way. But this dfa is smaller than M. First. 5. 1. we see that all statcs ir.r.ssumptionthat M is minirnal.ndq.nd q6 are indistirrguishable and indistingrrishability is arr equivalence relatiorr (Exercise 7).treca.. so (3 and ga are distinguishable. brrt . 1 0 .rly.gr) arrd (g3.1 2. By contradiction.hle. must be irrdistirrgrrishable. sr) e2 and Qa are distinguishable.4 2' (c) _@_"*64__r_@___@*G) Tlris is rninimal for the firllowing reasorr. rerrrovethe inaccessiblestates {2 and 4a. This thcn gives the rrrinimal dfa. Assume that q6 ir. ur) Q1and {3 are distinguishable. contradictiru the a. cornplerntrntthe final state set to give a dfa fbr .(1.D Exnncrsns 369 Section 2. uu.L.Since go a. Then we can corrstruct a srnallcr dfa M that acceptsT. A utring of irr is not in L If it is of tlxl fbrm o.).redistinguishable and therefore the dfa is minirrml. (a) Separate irrtrlruNes ???:0.l.
nbe rcpeated as often as desired and thcrefore denote arbitrarily long strings. S p l i t i n t o t h r e e r : i r s e sn t . (c) The staternent ( r ' 1* 1 2 ) . The tcrrn (a + b * c)*a (a * b * c)*b(a + h+ c)*c(a + b + rNwill do this.t. The expresSiorrfor an infinittl latrguage mgst involve at least rlrre starred otherwise it r:arronly denote Iirrite striugs. Ea. = ( r ' 1 * r ' 2 * ) * (I(r1)u is true.t tlenotes a nonerrtpty string. bb. * : 9 . ) . (r') and nu (r) : na (r).. We rmrst also ta. The answcr. For all the completc solution you rilust genera. (c) You just have to get in each symbol at lcast once.kel the strings in which a b is followed by an a. (c) Create two 0's. r t ( r 1 . that is the set tlf all strings ( r 1 ) a n r l L ( r r ) . r 2 * ) * d e n o t e s( ( r . 2 3 . etc. 1 6 .)* + 1*. (a) Create all strirrgs of length tlree and repeer. then this string ca.A short sohrtiorr is ( ( a+ b + c ) ( a + b + r ) ( a * b + c ) ) . m e n t so f . starrecl sullexpressiontha. 7 2 . Bv thc given rules (r1 +rt)* denotes the langua. o. Sohrtion: (1*01.llyrrot hard' 1 5 . L which is the same set. .* b)* ba (a+ t). interspersedwith l's.strrrightforwarrl solution.370 ANSWERS irr this cloesrrot txrrrrpletelydescribe Z. z ) 3 . m . A closecl gorrtour will be gcntlrated by an expressiou r if tr. givirrg six terms that cau be adrltxl. n Z 2 . B r . 1 4 . Enumerate irll caseswith I'rrl: 2 to get (a * aa (a * b)* aa I ab (a f b).c:h 1 2 .ge of of arbitrary cont:er. a n d n . If there is one subexpressiorr.b ba.01. but is not enorrgh since the a. is conceptua. . although quite lorrg.: l .( I ( " r ) ) . ( " r ) ) . then rcpeat' But don't fbrget the casewhen there are no 0's at all.tcrrations eletr ("r)). will precede tht: b. Solution: (. 2L. r t t ') 3 ' case has a. : 1 .\ + a + u(r + aaa)b* + u*bbbbb* (4. 18.nd only if n1 (r) : n". . .te permutations of the three symbols. 1 b)* ba + hb(a * b).
Solution: 4.2 S. The bit string must be at least 6 bits long. then the solution ( 1 1 1 1 1 0 1 0 1()o + 1 )( 0+ 1 )( o+ 1 )+ + + 1 ( 0 + 1 )( 0 + 1 )( 0 + 1 ) ( 1 0 ) ( 1+ 0 ) ( 1+ 0 ) ( t + 0 ) . (a) Start with Then use the nfatodfa algorithm in a routirre rrranner. but gives a more cornplicated answcr.c. Section 3. . This can be solved from first principles. so arrything will clo. then either the second bit fiom the left (16) or the thircl bit from the left (8) must be 1. Notice several things. If you sec this. If it is exactly 6 bits.Solurlons .ND HrrrrsFoR SELECTEo ExeRcrsns BZl 25. The latter will of course work. If it is longer than 6 bits. + readily suggests itself. its value is at least 64. without going through the regular expressiontonfa construction.
the edgesin the general case created by such a path are omitted. gotten by looking at aJI possible: paths. This is easily done. one of which is qs f F. (b) First.2.a*(a I b) ab) . One case is l l t \ ir'\ }{ ( rr )"*( I ) \+/ 11 n Since there is no path from qj to qa.372 ANSWERS 7. .L (r) where r : a* (a + b) ab (bb * ab* o. 10. E. The result. we have to modify the nfa so that it satisfies the conditions imposed by the construction in Theorern 3. is \ / t / Ou.o qlL **19 The other case can be analyzed in a similar manner. Removing the middle vertex gives bb+ah The language accepted then is .
t irrrd introduce a "parallel" autorrraton with states Q0.onSnr.rt of the original dfa looks like tltett the dfa with its para.. removclstate 4.. ) 17' (a) This is a hard pr..4t.blem until yorr see the trick.SoLUTroNs nr'ro HrNrs r. Then arrange matters scl that the spurirlrs symbol nondettlrministically transfers f'rom any state of the original automaton to the corresponding state irr the parallel part. start with a clfa witlr states eotett....L if arrd only if the corrstnrcted nfa accepts i. For exarnple. . aa+b Next..nsert(L). (ab) + (aa + h)(ba)t bb ai}ro \l \/ 0 The regular exprelssion then is r' : (ab * (aa + tr) (Oo)OA.ncrnl ExnRcrsns BZB Therr rernove statrl 3.llelwill be arr rrfh whose corresponding part is It is not hard to nrakc the argurnerrt that the original dfa rrccepts .. if pa.
(a) First constnrct a dfa fbr L. 6( q r . Split this into two cases: (i) z and rn are both even and (ii) rr and rn with are both odrl.r: S + Abbb A'+ AblB B +aaU C i a(.d (qu. But 8 contairrs the rule A . If wc now apply A + Bu. then w 1 Btnttl.1. (qt.R . Right linear grarrrrrrar: SaaA A ts C Left Jinear gramma.tetl with left lincar derivations.uRB. with A € V andrur e T*.b ) : g r d . o) : t1:1.S Section 3.374 ANSWER.J1. This is straiglrtforward and gives transitious such a^s d (qu.3 a.wfuRB : (Buw)R completing the inductive steP.5 b) : qr d d (qr. it Inust have the form ur : Awr. The solutiorr then falls out eir"sily. o) : (1o.\ t . aAlB bbbG bClA we (:arr show by irxlrrction that if tu is a sententirrl form derivtxl with G. By the inductive assumptiorr wR : u.a ) : e 2 . l o .b) : . d (qr. 5 + aaSlA A + bbAll tnkirrg care of casc (i). thcrr t tR can be <lcrivedin the sirrrrenumber of steps by G. rZ.FA can be derived via G.b) * Qo ( g s . Beciilrse ur is cretr.(q2. so we (larl tnake the dcrivation ll.o) : q1.
1 2.\ qr + bqnlaqo qz Qzaqtlhrlo tulzlbQt 16. .teand final state 91. Flom this we find ) d ( ( s o .a ) : q r . b d ( ( q t . b) : Ft. p r ) . r .) is given by d (po.b) : qt.L(baa.) n L (baa*) : baa*.f (pr. p o. A dfa for 6 (h. p o .6 (p1ra) : pz. Therr thc' construction of Theorcm 3. d ( ( s r . \ Ff^f"'L(qtl The condition that V1 ancl V2 should be disjoirrt is cssential so that the two nfa's are distinct. a d ( q o .6 (pz.pr) and that L ((u + b) a. but tedious.4 gives the answer qs _. a ) : ( q r .te p2. with q1 a trap sta. When we complete this construction.SoLU'r'roNs nr'rn HrNrs poR Sur. aq1 lbg2l. l i ( g r . p t ) . ) ) : ( q r . We can show that their union is also regular by constructing tlrc f'ollowing dfa.6 (po.a) : pzl6 (pz. we see that the only firral utate is (qr. p t ) . Chopter 4 Section 4. p r ) . p z ). Obviously. a ) : ( h . ) : qr. L ((a + b) a. A dfa for . p z ).eflret ExnRcrsns 378 with qs the initial and final state. with Iirral stir.b) : p. . 51 is regrrlar as is 52. a ) : ( q t .n) : pt. (rr) The construction is straightforward. etc.) is given by d (qo. . l r ): g r . 1 ( ( q .b) : pl.
usitrg the constnrt:titlrt suggestedin Thcorern 4. theu L n L ((u. L1 tt l'2 is finite atrd therefore reElrlirr for all tr2.) = a. Therr. Notice that nor (I'1. by A .we calt a"ssume that yl arrd V2 are disjoint.by A+:rSz. The Ieft side is regula.376 ArrswHlrs 7. cotrstruct the tifa for Lft. final state. 5 2 .Pr) genertr.L2. Suppose Jr :1yt. By closurc urtcler reversa.L contains Ilo even length strings.W i t h o r r t l o s so f 26. Htrrc you need a little trick.L.2 1. Then make every tllrlrrrent of P a fina. fbr aIV tr2. The resr. We cim use the following construction. 5.tt:.$o we can use Theorem 4.tcsP suclt that there is a path fiom tlrt: ittitial vertex to somt: elertrent of P' and fitlrn tltat elenent to a. 2.2. which is regula.u1 ab * ba I bb). $ince by Exarnple4.L2 is regular' 18. Sint:cL1l L2 is regular and wtl have att algoritlrrn fbr set equality. replace cvury production of the flrrrn A+ frt with A € y1 atrdr€?*. is Tlre key observer. . Frotn the clfa ftrr. If Lt c . Lz) : L1l) L2.r. Find a. The rest ttrett follows easily fiorn the known closures unrlcr uttiott antl compl(lrrxlrrtatiotr' 14.lity. X*. Use trr : X*.tiorr that sitrce tr1 is Iirrite.7.llsta.ndz € 7*. (c) In Pr.mmars aild (a) Make 5 the new sttr.Tlrt: 16. .rlt tht:n follows fronr closure untler cottcateuation. P 2 ) . there exists a' rrtetnbership algorithrn for it.Lft is regular. replar:t:every production of thc forrn A+ rt with A e V1. The an$wcr is yes.r.'l L1 . Tlrtl result then follows frorrr closure under irrtt:rsection and complcrntltrtir. ir.rt sytrtbol and add prodrrt:tiorrs.t :r.6. Corntlirre the two gra. If . 12.51 lS2' (b) ln P1.l. 12.l sta.thetr L1l L2: Jlz.ny. Then use thc rxprality algorithm in Thcxrretrt 4. Section 4.5r. T .r:Sr.tiorr.7.9 . werillstt have att algorithm for sct irrclusion.Lz)n Zi) u (r' 1rz) . ( a n d G z : ( V z . LylL2: given statement would then imply tha.L2 is regular. It can be olrtairreclby sta'rting fiorn the set identity Lz = ((Lr u.
Suppose that 1. tr is regular.l choices.keL1 : {a"b* : n. (a) Givcxrrn. n.. is not in /:. which is regular. If we chooseu : a(Laauaba. > 5. Now ig is a. If this is longer than thc rurrnber of sta. .cryclelabeled E in this walk. Brrt Z : {to .qxo Hrw. As a courttc:rtlxilmple. (a.ge.r.rly this cycle can be repcatrxl a. : a. directly. 9 . By contradiction. (a) Ibke p to bt: tlxr srna.s r.*h*). ta. we carr thclrrviolir.'rrthrr.o(w): no (u)} whit:h.tesin the clfa.t has two consecutive symbols the sarne is in the languagc. t h t .ge not regula. For the dfa ftrr L to l)r(x:e$rJ the middle string u reqrrirt:s ir. 1 ) f u : p ( h + 1 ) i s c o m p o s i t ea r x l E . wa.rnt O2rn.) The Iauguage is regular. . rrsis ea"rily shown. A regrrltr. 5... r rp + ( i * r Irtplr is nclt irr ttn lirngrrir. bot}r of whic}r arrl rulrrrcgrrltr.ryrmrst then he aa and the putnpcxl strings will be 'ut.rrrl But LrtlLz : {att6nt . n.nd : clroose'Lt) trP.onSelncrrnn Exnncrsns 377 Section 4. pir:k rt. (a) The larrguagc is rcgular. I f w e t a k e i .ngua. 1 S . r t s cottstnrr:t rcgrrlilr expressions.t: ettzb't'a2"'. is not regular. L(a. 4. Lherermrst bc ir. If y consists of orrly o's.nguages.. But r:leir.The strirrg.i)ft > rn. Then by the closure of regular languages under c:ornlllrlrnonttr.r.s severa.} * L2 tr. (e) It does not seerneasy to apply tht: prrmping lerrrma.r known closures fbr regular Ia.soften as desired wilhout charrgirrg the acceptability of a strirrg.Z would also be regular.r. firr which one can ea.r expreusion for "L is (a + h) (a * b).L is not regular. since anv string tha. If we ta.m't (i'* I)k r.(aa + bb)(a + b) (a + b). h : 0 . wert: rrlgrrhlr.so lhat wi : aP+\it)h.llestprime number greater or tlqrrirl to rn rr. If the opponent choosesgr as consistirrg of b's. Tlre prolrosition is false.tion. This is nxrst qilsily seen by splitting the p r o b l e mi u t o c a s e s u c h a s l : 0 .lk in the transition graph of lt:rrgth lul. so we prcltxlerd indirectly.sily (lr) Tlris lir. our is opporx. then u. 1 m. We see this frorn L : h rf .3 2.Lf rrnd tlx. > rn}. we use i : 0 to violatr: thc txrndition n ) 5.1 : p .tothe condition ft < L 1 1 .string of a's of length fr.Soluuons .kei ) 2 then nL+('i .
r languir. Take .2. l u l l < z } .take any string. u f .Li is the set {ttrtrrft}. Since neitlrtlr of these chatrgesthe nurnber tlf a's arrd there. We claim that the rrrriorrof all the . such as y : (.Li : tlb' ti : 0. As the final step we must slrow that for each i. r f : l u d < z } . steps this corrdition holds. Altcrrratively. ) ti. L 7 . ..a5b... In the sary llow has severir. With the second choirur. take any strirrg z : lttu)Rt with ltul : n.ct that for circh i tltere are onlv a' finite mrrntttlr of substrings ui. is irr the uniou of a.. we carry out an induction on thtt lcrrgtlt of ihe derivation.\ or 5 .La. Since the ouly possihlt: idtlntification is ururft = hta.ht: Li.pplv S . 1 .f . L a . t h t r nz e { t t i u f ..thc lettgth of u0 is odd. For ea. t}re prefix conditiorr holds after one step. either.o. This ndds arr extra a attd an extra b. if tlrtl prefix cotrdition holds after n z steps.. To justify tlris. Chopter 5 Section 5. r r r l t } r c r e f o r e i n . If rr.ll t. As cotrttterexantple.s (:irrrrlotbe in {tizuon: ltr. the prefix condition txlrrtirlrt:s b's or the location of those irlrt:ir. 2 0 . it will still hold a.lforrn irt rl * 1 steps.378 ANSwER. .r. . tlrt sirrce the addecl a is to the left tif tht: adtled b. To get a sententirr.gc tr : { a ' b " : r z> 0 } ..55.. No.r of lt:rrgtlt rtl that is in a'll of tht:. luil < i}. 2 . Obviously. . Tlte adver: r:hoit:cs. To show thtr. z < z .n m.rly to hold. so we havc a trirsisarrd the induction succeeds.fJer { 1 stt'ps. 1. so t}trt it has the form tutlft. If it we take i greater tha.1 a. Liis regultr.t: (ab)'"'aa(ba)"'.Take '7.. we apply S .. so it r:irtrnot Lrein 1.l = i} trec:ause is not Iong enough. . ri. is firrite atrd therefbre regtrlirr.S (b) The language is uot regrtla.ch t. it is not. : l t r i l= i } u { r .This follows from the fa. It is qrritc otrvious that atry string genera. .texlby this grarnmar has the silmc rrurlrber of ots as b's.take the lir. brrt the utriotr of all the langtrilgcs is the notrregula. e z: {0.Consequently... . : l o i l = i } a . Corrversely. 1. then a E If r { u i u .l first case uro = (nb)k aa(ba)"'' . 1 5 .r. Suppose that for every sentential fbrm dt:rivcd frotrt 5 itr rt.ab)kot . the prefix r:orrdition will still be satisfied.. Thrru.ltl'tl6 is trot in 1. . It must tht:rcfrrrctbe in {lr1u.} ilntl so also itr L. we ca'n a.'tJ fub)k'a.nguirges Li: : l { u i u u f .t thtl llrelix conclition no (u) > n6 (u) holds.
L2.L directly to get a grammar for Z.L. (a) If S derives.Sz.aSrbltr.bDclE E . This can be done by S ttaaA A + aAblB B . 1.Ecl). It is normally not lrossihleto use a grammar for .SoI.a.ke care of the casesz : 0.\ In the second case.\ l51l Sr .:h*m and rn: h+rz. then . (a) For the first c&se : ?n arrd ft is arbitrary. Then add rnore b's.. = m * 3.BlA D . solve the ca$e n. hopefully recursivedescriptionfor .+B b l I But this is incomple. (e) Split the probltlrn into two ()ii*..Je$: n.te since it creates at Ieast tlrree a's. 1 5 . Finally.AlaAlaaA (d) This ha^s unexpectedly arr simplesolution S + aSbb la^9bbblI. This c. wt: add 5 . we start productions with S t Srl. Sz +BD B + rr.rn be achieved rz by St: AC A + aAbl\ C +Ccl. (a) First.2.urroNseun Hrrurs poR SslucrHD FlxgRcrsEs 379 7. Theseproductionsnondeterministicallyproduceeither bbor bbb cach for generatecl a. so we rreedanother. The first caseis solvedby 5 .9c . Here we use n.5r  55 derivesL2. Trr tir. is arbitrary and m 5 h.
possibly lead to But this scrrterrtial form has the suffix abc.ab.ceon the right.380 ANSWER.lh B C b](rla]b aCa bCblA.nnot aubbabba. There are two lef'tmost derivatiotrs for w : aab.l(E)llls. Suppose we hrr..t is rrot of the form uuft.S Z. The first two productiotrs geuerate the u and u. 19. A solution irr S +aA. A +a.2 2. One obvious subset of 1. tlrcrefore be of the fortn utturRbu rsr ubwwRau with lul * lul.ts Note that the more obvious grammar 5 + a^9rB b. the senterntlc 22.EIE. at some point wc get aII a otl tire left and a b 'I'he string must in ther rxrrrcspondingpla.ASAIB A + a. trrrt this is not all. C)nesohrtion is S . ttris is a little harrl to stle here. 5r + a5rBltr Bb is rrot arr sgrammar.rsfor these typtls of strittgs. E + * E lE. 5+aaBlaab S+AB+AaBlaaB+a. or vice versa. compare corresporrrling symbols. atrd the last the innttrmost palitrdrome. The only possible derivations stirrt witlt S + aaB + aaAa + aabBba + aabAabu.nbe of tlte forttr wwR . 6. . E Section 5. Orrct: we see tlris. we can then corrstruct grailtlna. Working fiom thc r:crrtcr to the left and to the right sirrrultaneously.veirrr cv(llt length string tha. While some part irroutrd lhe center ca.ABlb. thc third the two disagrecirrg symbols.so it cer. contains tire strings of odd length.
. * uhab.rting with A +AA. It is not ea.li .fi S + AlulaBB u A .rrnrnirr i$ 5 . The only nullable variable is A. consider the two tvpical situations..1 3 .t this grammar is unambiglr()lrir.lont rrrril. Sirrce the dfa never has a choice.syto see tha.1 to substitr.bAblhh.aAblabl AA.\.Al. C B is the only unitprodrrction rr. whitfi carr otrly be derived stir.mmilr is obviorrs fiom thu rklrivatiorrs S+aSb+ah S +.\ A .aaAlau B +hClhbC C8.a B +b}lbbl C +bClbb1.2 thc two grilrnmirrs are equivalent.9 + abS+ ab.rr orrly be derived by starting with .4. Solution: S . be Then B becomesuseless and the associatedprodtrt:tiorrsc.ilrr retrtoved.kt:it plansible.d \. whictr r:ir. Chopter 6 Section 6. Use lhe rule in Theorem 6. ut = aahb.Ala. so they can be parsed only in one way.Exnn. To rnir. there is never arry choice in the procluction that can be applied. An orltivir. 20.rtefbr B irr tirc first grantrnar.SELECTET 381 L Ftorn thtl dfa for a rcgrrlilr limguagc w(] cirrl gct a rcgulirr gritrnrnat by the rrtt:thorl rlf Th(xlrcrn 3.5. Thcl grirrnrnirr is itrr sgrittrtrnar except for {l . By Theorems 6.ndllrnovitrg it results in 'i*hl h ' r  f i A "*\> '8 yl i".1 and 6.s b a? uAlalaBB fi + A + ae.4 n aAb.rnlligrttlrrtt gri. . r4. But this rulc rlotls rrlt crclatc irrqy arntrigttity. and ti. Arntriguity of the grir. so removing Aprorlx:tiorrs gives '? ..aAlaAA A . Morc cxrrnplicated strings are built frorn these two situations.crsns Souluoxs ar'rl HrNrs FoR.
An exa.V. B + V7. The grammar ^9 + aA.2 5. A + a does not have any useless productions.6. 1 6 . B and d ere useltltss. Whenwe remove trproductions get we SaAla A .bbAlbb.aa.aaAlaa.AB lbbAlaBlbb A .aa is an equivalent grammar. of of is S . The language generated hv this grarrrrrrar is I ((aa). Brrt it is not minimal gince S . Section 6. Removal this unit*production easy.Al V"BIV.aBbl\. Tltis has introduced a unitproduction. we get so S .aBblab. A +V.BBIB B .382 ArqswnRs Finally. We can now apply the construction and get S + AB lVhV. any unit pr()ductioruJrer any Aprodrrctions.V6AIV6V1. 14.rV6V1.a) .mpleis S+aA A. This is obvious sincxr the rernoval of rrseless Drocluctiorrs rrever adds anvthing to the grammar.uAl(r A .b B + bbAlbb. . 2 1 . This gives S+ BlBlaB A Aaab B .BB B . which is not acceptablcin the construction Theorem6. First we must eliminateAproductions.
wc catr use the yij colrtputed there.VnV. Then use a. .tr.nd A atVr. aBC. Vt .. but after substituting filr B..AIV.(trn Vz +QB Bb1b2.blelu Vz attd productions A .AlWBlVbVb A VaVa g . Fbr A .Srlncrno Exnn. Va b. . until no terminals remain on the left.bAV and V + BC.V. 15. V' . I/r. so we introduce A .a.r I)ro(lessto rernove terminals on the right. .. Only A +\ABC is not in the required form. Considerthe generalfonn for a production in a linear grarnmar A a t a z .b.. irrrrl so rlrr.3 2.V.11. simila.ABlV. . wtl hirvc SaSA AbAV VbC C Section 6.aVtVz.anBbft2. W * a Vnb' 8. Solrrtions:5' +aV6laSluV"S.4 +BC a. create new varitr. a. The latter is not in correct form.nd S .crsns 383 a.. Vz .br.. 9. the string arlb is irt the latrguage generated by the grammar and can thercft)rc be parsed. itttroducitrg V2 alrd :..A.. 12. W a.V. tr/l . Since S € Vrir. Productions of the form . n Irrtrodrrce a new variable Vr with the productions Vr + az.. b r u .reperrrnittr:d since a : . This normal fbrm r:arrbt: reached easily frorrr CNF.\ is possible. Sirrce aab is a prefix of the string in Example 6. (lontirruc this process.Souluoms eNl Hrr'rrs poR.V. aB b 1 b 2 . .
u sS e A €.nnot he reached.. a. we see tlut we carr get to this tlorrfiguratiort orrly if at tlris point the utrread lrirrt of tltc irrptrt is r1:r2.ll to thesecan thcrr bc usetlirr thc parsirrg S+AB+aB+aAB+aaB+aab.. 1 1 a ).np.z. ( q 1 .. Chopter Z Section 7. 1 " ) .switch is made at some other point or if the input is not of the ftrrm ur?rrft.384 ANSWF]Rs Frrr ptr. d ( q o o . I ) } d ( .a . z ) : { ( q 1 .1 1 a ) } a .. 1 1 . 1 1 ). 1 rt. u ) : { ( . V11 b e c a u s e A B €V23 becauseB A € V 2 2 becauseAAB. .trse + b.rp. ( q r . a.By exarrrirrirrg thtr transition functiotr.a ) } I d ( q o .e). t ) : { ( S t . 1 1 ). 1 i . 1 1 1 ) }.. a ) : { ( q r .iln a. that is. a ) : { ( q r . or three tokens by d ( q o . The key to the argument is the switch frorn q6 to q1.r$ing. Suppose tlre ccrrrtcrrt thcl stirck ir.b . with A € Vzz. .. Solution: d ( g o .\.1 1 1 ) } d ( q r .B e Vn a B e V r r bet:a. 1 ) : { ( q 1 .teone.r .we determine the prodrrt:tions that wttrc trscrl irr jrrstifirrg5eV13: S e V r r b e c a . ( q 1 . if thc original input is of the forrn 'ururEand the switch was rnade cxactly irr the rniddle of the input string.cr:eptingconfigura. B This shows the productions needecl justify membership. (a) The solution is obtained by letting each a put two markers on the stack. if a.tthc timc of the switt:h is n112. d ( q o o .with 4 € V1l and R eVzt a AB.genera.. z ) : { ( q 1 .1 1 l a ) } arrtl . while eaclt b corsurrresorre.tion ca. two.Trr arx:ept tlf a string we rnust get to tlte cotrfiguratiott (r71.1 2. (f) Here w€)u$e nondeterminism to. ( g r o . However." ) } . which is done nondeterministically and need not happen in the middle of the string.
a ) : c r d (gr. The state transitiorrn r:a.n be ca. d ( r 1 1 .o.rld normally be trackrxl hy different states is now tracked by thtr symhol in the stack.r'r. a"s L This is n pda that makes rro us(] of the stack. Z ) } .2)} .1 ) } . The pda thercfbrc a. t t d (.solution is we d ( s o . Since d carr hirvrl only a finite number of eletrerrts and each can only add a finite arnount of infbrmation to the stack. a . finite acx:t:pter. 14.Solu'r'rons Ar'rD HlNrs rron Snl.. 6 (eo.a) : qr r) (qo. to givc .rriedout for any pda..rr llxnRcrsns 385 The rest of l. 2 ) }. What wor. u ) : { ( S o .) : {(Sn.b) : gt d ( q r .nthen be taken directly from the pda. irr trfl'ec:t.de)} d (qj". this rxrrrstnrction ca.a. For exarnplrr. 1) : {(So.me 4(a). 6) : .)}.. thc initia. 2 ) : { ( q l . We have only two states. i .70 11. The transition fiom qs to 92 can bc rnade with a single a. Here we use internal states to rcrnember symbols to be put on the stac:k.]resolrrtiorris then essentially the $a. T(r ovtrrcome this. 16.ccepts the Ianguage L:{"}DL(tbh*a). tr. c d e ) } is repla. we put a syrnbol irr the stack that rernemberswherc irr the sequerrce are.l (go. Here we are not allowrrd enough states to track thrl switr:h from n. d ( Q oA .b: {(qi.l:c. The altcrnative path requires olle a. .taking one path rrt ir time. followed by orrrl or more b's. For cxilmple. h:) { ( q i .{ ( q o . terrninated by irrr n. a . d) : {(qi.a.'s to b's and back.l state qs and the acceptirrgstate gr. Tlace through thc: pror:ess.a . lr. cd)} . These are the only choices. 2 ) .tr.10. 6 ( t u . so that is.t:edby ) 6 (qo.
a ) : { ( q 1 . L We first obtain a grirnrrrrarirr Greibach normal filrm for tr.1r. B .386 ANSWERS Section 7. usitrg'Iheorem 7. a) } a) stirtc:.f) : t(.b.1 ) : { ( q r ' A ) } . b) : { ( q r . 1 n . (qr .l 4. I'he state tJ1ttirrl bc clittritratecl if we use a special stttck syrrtbol z1 to rnark it. > 0}.1' .5'sr) } . S ) : { ( q o .S B ) } o b.I) } 6 ( q o .A .t.tr. where qs is the initirr. giving 5' aS. : { (q.1*.1) {(q2.l tlr you can trotice that 3.. 'Iheoreur 7. .I)] d (q'. Next. .1 to gct art npda with three statcs. for exantple S + o.A) : Fitrall. when {(Sn. A r:ornplctesolution is (qn. Frorrr that gramlnar we can tlrtlrr construct an of equivalent threesta. l . s) : { (qo. B)} d(qr.'s witltout changing the stir. o.5^5'. With t}re latter observatiorr wc get a solution .. d (r. we apply thc corrstructiou in Theortrrn 7. S) : { (qo. a ) : { ( q z .") : {(qr. d ( q i 1n .2 'I'heorem T.aB. givcn arry trpda.l rrpdrr's are also equivathe original and the tivity of equiva. 5 ' r ) } d (qt . I ) } t. .trlrrlltla.tr. a )= { ( q 2 . After tha. I ) } . 11.l d ( q t ' .A)} simply reads o. Then ftrlkrw the cotrstruction of Thslrr:rrt 7.o.z ) } 1 6 ( q r .\.n.klrrcltl. d(g6.l (. h . Be<:ause the trausifintr. z ) } : 6(qr. I ) } d ( q o .5) : { (q1.l state atrd q1 is the finer. o .o . First txlrrvert the gramtnar into Gritlbach trortnal form. SSS) . i ( . Thcrc rnust be at least one a to get started. Florn 7. . .b. 1 . e)} . a r) : { ( s i .111)} 6 ( t 1 2 . z ) : { ( S r ' a ) } d ( q r . Ient.l.liBlb.B . we call constrtrt:t arr equivalent cotrtextfree grammar. .).2.a.r.A .b.y. d ( q r . B ): { ( q n .t:l<. Ytnr carr follow the constnrt:tiorr of tlre language is {a"+?bzntl :'n. (lo)(h)qI.
with a dfa along the lines of thc rxrnstrrrctionin Theorem 4.11)} b d(q'. The grammar is thereftrrtt rrot in LL(4). 0 ) : t ( s .dpda. . if followed try a single Section 7. ca. If more symbols follow. 1 rA . t r" ) : { ( s o . while in the sccxrrxl + SS is the necessary 5 Iirst stt:p. the initial.r:k hnndled as it is for.1 z ) } a .gc. Brrt if we see only the first four syrnbols.r:k.'s into state q1.tire derivation In rrrrrststelrt with 5 + aSB. 16. we can constrrrc:ta rlpdtr.tht: Ianguageis tlcltr:rrnirristic. Consider the strings aabb and aabbbbua^ the first case.1):{(s1. There are two stiltcs. The solution is straiglrtforwrrrrl.qJ}. Prrt a's and b's on the stack. 0 0 ) } . e t c . 15. a strirrg will be ac:t:cpted and only if it consistsof one or more {r.rlrr be tr.4 2.rclto see that the result is a durlir. from whir. wo r:ilnnot decicle wlric:lr t:ir"st: a. d ( s 1 .llsfor two different types of suffixes. Lhe prltr goe. Thc: c signals the switch from savirrg to rnr.3.te q1.h it ca. . When you write this all out.'s. .kea .L1.h stirtcs to q0. nonaccepting statc 96 ir.1 1 ) } : (Qr. Il should rmt tre too ha.vseerrr lle a noncleterminislic larrguir. The rest = is essetrtially tlrtl sitrnc nu Exa.r)} . Complete solutiorr: d ( q o .l.2) O t ) } . At first glauce. The pda will bc in sta. This is ohvious since every regular languirgc r. 11. Wlrcrr this happens. . Nevertheless.3 4.pplies.r:cepted a dfa by artrl such a dfa is a dpcla with an unused stir. d ( .rtching. the pda will switc. a ) : { ( q 3 . it gclrrs orrt of this state and thcrr ir:rnpts at'bt.n only ma. i ) : { ( q 1 .1) {(q1.ack. Therefore.te qr unless a a is on top of thc sl. The basic idea here is to corrrbirrcrr.nd the final sta.a. 9.This dpda. {(qr.. d ( q Ba .b. Since . this tna. 'I'hus we have d (t111.. a ) : { ( q z .mple 7. Section 7.a . .Soluuow$ er'rnHlFrrsnon SnlncrnD ExliR{trsus 387 the first b is encountered. w i t } r d ( q r . goes as into a final statc wlxrrr the first input sytrbol is arr a.a. everything can be dorrc so dctcrrnirristically. you will see thai the pda is deterrnirristic. with thc sta.\transitiorr to tlxr final state.a ) } whereF:{q2. to since the prcfix q.a ) } .
The only ?.L(1).L(a. choice of u irrrd g that treeds any serious tlxarnirratiott is tr : aft and . wit}r u atrd E located in the prcfix a*.''*hL6nt6m OrnOut6rtt.ge. Givetr rrz. (a) S .(t.arrbe made for arhitrarily long strings. ct aba. tltat have to be consiclered.grarnrrar.nlmtrr is urrirtnbiguous.aSc..tl. is wc carr otrly use 5 .91'+ nS1lba Sz + ahbB B .). u: ak and't7: tl{.5 . ftrr exaurple. is converted into ir. There a. if it is r:. The gra. if it is b. Looking a.r c:anonly be in.. s.but in all casesthe string ca.L.1 The ir.a S c 5 1A l l '5'1+ bS1cl.L grammar anrl the two can be combintxl irr an obvious fa. As lotrg as the currentlv st:arrrredsytnbol is rl. If they are (r.t26m.mplcsc.\. necessrrry. then the 4. 51 or S =* 5'z is 7.bRlA.blestring rmrst be in . Ftlr t:ach case' we can find a. the gra.bn). dpda."'b"'brrlartlattlbttr. (a) Use the pumping lcrntna.n sgrammar. which is not in L.51 . :4. Ltrok rrt the lirst three symbols. tltett 1!)O : Q. When this dpdir.rr L.mrrrar . I Chopter Section 8.388 ANSWTTR$ similar exa.If. For a detcrmirristic CFL there exists a. wc rnust use 5 .I (k) fbr any A.shion.t the first tlrree symbols tells rrs if S + The gramtnar is thereftrrt: LL(3).L (abbb. A sohrtiorr is .\. This is almost a. lf the first three svrnbols are abb' then strirrg atty parsa.dvcrsarylrow has several r:htlit:tls 3. aab. pick u. the grirrntnar is not /.reil rurrrlberof olher possible choitxls. we must apply S .51152 .nhe purnped out of the langua. Take ut: u.
choose u) : a*bz*a*. . . use u . (f) Given m.z e r o . )+ ( . so .respectively. : ( .. L2. The npda chooses the fu nondeterministically. wherep and q are prirnessuch that p> m arrd g > rn. Similar arguments hold a fortiori for I > 1. . the language. Construct an npda that counts to some value h (by putting h tokens on the stack) and remembers the hth symbol. we can easily purnp so that for some prefix z. If w C L there must be sorne h for which this happens. If luul: k. n1 (u) < ny (u) which results in an improper expression..^bn'an'b* and examine variou$ choices of u and . L 0 . ) where (. The language is not linear. (b) The la"nguage is not contextfree. 15. the language cannot be contextfree. i >mz+k' Since ru2 is not in the language. .Solurrons AND HrNTS FoR SELECTEnExnRcrsps 389 t A : b I . 20. Use the pumping lemma ftrr linear languages. But m2 (rn+I)2:mz +Znt. With the pumping lemma.r7. .( o ) . which is easily pumped out of E . ..ruis easily pumped out of the language. Now u and y are entirely made of a's. this language is corrtextfree.r arguments holcl for other deurmpositions. the string is accepted.*I T h e n c h o o s e: 2 . then lni+rl: pS+ik. With a given rn. . so that tr2 ha^s + h a's and rn * 1 b's. If this does not match the remembered syrnbol. Perhaps surprisingly. . .( and ).. If we cho11se : pq. If lzl > 1. then i wi+L: aps(l+li)1 which is not in the larrgrrage.S u p p o s e h a t l : 1 . Use'u): qrtQ.( o ) . Use the pumping lemma with I. It then examines the frth symbol in ru2. w i t h k a n d I n o n .) stand frrr rn left or riglrt parentheses.u: e.. Sirnila. choo$erp *qtnfitn+l"m*Z.
L 2.7.tr) : (gz.mmar G.] < n " ( w ) > n o ( w ) .Pz) be a leftlinear grammar for L2.mmar$Gr and d2. It is then easily shown that in this grammar SzlSrwlutu if arrd only if u E L1 atrd tu € tr2.t scans the entire inprrt is . Combine gra. (w) # nu (*) and ?2. I e . d 5.a) : d (sr. L e t ( J r : i y t . n " ( w ) < n o ( u ) . T}re language . A threestate solution thtr. d (sr. m m a rf o r . 1 5 . Hnnu. Chopter I Section 9. Pl U Pz US i Sr l.L (t) : r. L 1 a n d l e t G 2 : (Vz. . (G) if and only if 5 is nullable. Ttr show that linear langrrages are not closed under concatenation. (ru). TIre languages Lr : {u'nbtrt"rttl are both urlam{arrbtlc. (Gr) u L (Gz). Given a contextfree gra. l?) !.T.. and . with VrftVz : @. ? .^9.2 1.Lz is not linear as can be shown by an application of the pumping lemma. Sz.L2 biguous. t S . Given two linear grammars G 1 : (V1. The full Ianguage is then the union of these four case$arrd by closure under union is c(lrrtextfree. d ( g oa ) : ( u .a.91 P1) and G2 : (Vz. take tlre litrear language 1': {a"b": n > 1}. construct a contextfre* gru*** by replacitrg every prochrt:tiorr A . choosing Sz as a start symhol. u . R ) R) d (qr. P r ) b e a l i n e a r g r s . S r . Each of these is context*free as can be shown by construction of a CFG.7. a n dn o ( ' u r ) n " ( u r ) .Sz).r by A . These in turn can be broken into n . Pz) .390 Arrswurrs Section 8.Ill}and .xR ' We can then show by an induction orr the number of steps in a derivation that if tu is a sentential fbrm for G then uft is a sentential form tbr d. n " ( w ) 7 n " ( w ) . The complement irrvolves two caseu: (*) # n. with F: {Sr}. Then d i. 9.h) : (q1. The complemerrt is contextfiee. But their intersection is not evt)rr cotrtextfree.Sz. form thc combined grammar G : (Vr\)V2. Construct a grammar 82 from Gz by replacing every production of the form V + rtn e T* with V + Sr?:. 2T.T. .
ft) d ( q r . placirrg markers at the riglrt and left boundaries of the searched region arrd rrrovingJ the markers outward. but it is typical of the cumbersone wayir in which Thring rrrachirrers often do simple things.on Snr. .a. (ii) Replace the twosymbol combination ca on the left by ac arrcl the twcrsymbol combinatiorr ar: on the right by ca. f l ) a d (qo. If the final state set .emove one of the cr's arrd move the rest of the string to fill the gap' Obviously this is a lorrg joh.( q s . l ( S t .n. ft) .a ) : d ( q r . R) forallq€Fnnda€1.a. 7. Repeat until the two c's meet in the middle of the string.) 6 ((tz.a) * (gz. (a) d (go. ) : d ( s o . introduce a new firral state g.urrolrs arul Hrmrs l. The general scherntrlooks something like this: (i) Place a marker symbol c at cach end of the strirrg.b. R ) with .F : tqi].tr) : (qz. lt) . We cannot just search in one direction since we dorrtt know when to stop.a) : (8r. a) : (q. (iii) R. ) R 6 (qz. We rrrust proceed in a backandfbrth fashion.F: {qr}. fi) with .a.a) : (qz.r.n .b ) : ( q 1 . 10. .Sot. (b) d ( q o . b .lly simple. d (qo.ncrnDExERCISEs Bg1 It is also possible lo get a twostate solutiorr by just examining the first sytrbol and ignoring the rcst of the input.b ) .D . R. ftlr example. l t ): ( q z . 12.F corrtains more than one elernent. 19. The solution is conr:eptua. a.i and the trarrsitions d (q. but tedious to write out in detail.b): (gs.
d The state qs is any state in which the searchrightinstruction rrray be applied.n.l (qo.R) . we cannot yet claim that Thring machines are more powerful than a pushdown autolnata..5} with the restriction that for all transitions d (qz.b. 8. (c) First. Section9.  Chopter 0 Section 10.rt.u) : (Qj. L or . Section 9. an add. while a pda can be nondeterministic. (b) To simulate 6(si.L) with a'+ h of the standard machine. We have ignored the fact that a Ti.1' Then comparethe two parts' in symbol until a mismatch is fouttd' symbol by corresponding A solution: d (go. a) a. S) and 6 (qiz. .a) : (rljz.a) : (qi . we introduce new tratrsitions d (qa.onerrtachinethat jrrst adds one to the input.1 a. and a multiplier that multiplies two numbers. the condition a: b must hold. and so on. (a) We catr think of the machine as constitrrted of two main parts. split the input into two eqrral parts. Therefbre. as defined so far' is deterministic.3 2.b. : (Qo.. . R) .rring machirre. L) for all c € f . (so. This can be done as suggested Exercise10.l) : (qi. b.L. o. (a) The rrrachine has a transition function d:Qxf8xfx{. : (er.b) : (si.392 ANSWHRfi Section 9.B).2 s..b. Stltetnatically they are cornbined in a simple fashion. c) c. lt) for all c e X * to} .
d): (qi. we let 5M have rrr f 1 trirr:ks.c. We introduce a pserrdotrlrr.b}.l so trarrsitirrnd(S. Wc repla. Then.rl by a. L) wt':ilckl . To simulate the original rnachirrc (OM) by a standard'furing machirxr (SM).Sol. This exercise shows tlnt a qrrr. Whenever we need to preservtr this a. One issue to consider is wha..R) fbralldef{a.9M will simulate each move of OM by sca.b.t happens when two reatlwrite heads are on the same cell. the origintr.ll : (qi.. L This dotls rrot limit the power of the rrrachine.nJ"'. the new rnachine writes B.u'r'ror. Section 10. where rn is the rrurrrbcr of reaclwrite heads.ce 6 (s. say A.c. tape content ofOM position oftape head # 1 position oftape head # 2 .ldefinition must provide ftrr the resolution of possible conflicts..rt) by 6(qe. rrrrcl on. to write n. we introdut:tr a pseudosymbol. Thc frrrmrr.llyon the tape. wt: first write A. 1 1 . xl andd: Q xlr 8 x fz x {L.nningand updating its active area.nk Whenever theroriginal machine wants F. On one track we will keep the tape contcnts of the OM.l (qi.'uema.L) must be retained to handle blarrkstha. fbr each r) (qi.ls exn Hrr{rsFoR SELECTEI Exnncrsns 393 6 ..b. F o r t h e f o r r n a d e f i n i t i o n s ef 7 ' : I ' x f l u x.b.2 L .. fr) : (qi. Of course.t are origirra.chine is equivalent to a standard Ttrring machine and tlrrt therefbre a queue is a rnore powerful sttlrage .!) = (Qi.{o.b}) : (qr. For each symbol {t € l. while the other rrutracks are used to show the position of OM's tape heads. then return to the cell in question to replace .tr).
to simulate d (qr. It helps to use additional markers Y itnd Z to denote boundarics.sY to the back. the second thir. (ii) Circulate contents to get bzYdef gXa. one that mirrors the tape of the OM.discarding Y and Z as they crlnte to the front. (i) Remove c fiorl the front and add . For example. readwrite head .t stores the state of the OM. fbr example. L) we carry out the following stcps. so the querre t:tlrttents have to be circrilated several times to get everything in the right place. keep the right side of the OM in the front of the queue?the left side in the back.c) : (qi. e b c d c configuration ofOM configuration ofSM SM needs only two states: an accepting and a nonaccepting state' . we can. A left movtt.394 ANSw}]RS 'Io simulate a standard TM by a queue machine' device thirn a stack. We need just two tapes. ltowever. (iii) Add Z to the back. z. 8. then circ:rrlate.' b c d f tapeof OM c d c f g x b Simulation by queue A right move is easy arJwc just remove the front symbol in the queue arrd place something in the back. goes against the grain.
(i) Start with a copy of the preceding string.92. (ii) Move the readwrite head to the right. If it is.. s ? .4 3.5 2. First. in outline. divide the input by two arrd rnove result to one part of trr.'s to tlrc right of this to 0's.This frcc sperr:e initially occupied by tlrtr input.17. . If they . Section 10. . s r . accept.f 1.uuolrs aNn HrNr. (iv) With tlrc rxlnter of the input rrrirrkedwith Y. .2 . by putting markers at cir(:hend. Rememlrer the symbol hy putting the rnar:hinein the appropriatc state. 9 2 : enumerated by b {tr. For ir. (iv) Repeat frorn step (i).f2. 6. . . (ii) l'ind the rightmost 0. (iii) Compare the syrrrbol there with the rerrrernberedone.completely deterrnirristic solution.} Then t}reir urrionr:ern e 5 1 U 5 2 : t . Section 10. Nondeterministically choose a.ble.For 51 x . . This space carr thcn he used to slore sur:cxrssive divisors.ts is thereforr: r:clrnta.. L L e t 5 1 : { s 1.reject input.Sor. chirrrgcall I's to 0's ancl add a 1 on the left. we lirst find the center of the input (e.pe.g. Dt:termine if the length of the input is a rnultiplc of rz.usc thc ordering in Figure 10. . (iii) If there are no 0's. match.lternatively rnoving left and right. If they don't rnatc:h. write Y irr thr: cell. . s 2f. value for n. The union of l.]re two s. (i) fJtart at the left of the input. Clrrrrrgcit to a L. then there is $ome n for which tiris works. If an € L. is as follows. If sotnc si.sroR SnlncrnD ExERcrsEs BgE Section 10. An algoritlrrn. and rnoving theur inwards until they nrcct). }a n d .3 S.. we carr now proceed deterministic:allg a. . Then change all the J. stopping (rrorrdeterministically) at the center of the irrput. : tit we list it otrly orrc:t:. ] . Then replace it with X. corrrparing symbols.
Left arrd riglrt moves of the OM are easily simrrlated.2 to find a grammar in twostartdard form.b. Example: Configuration OM of *l"1 H fl Stack Configurations o rl ln l l fI rr Steckl l e I [l l Stack2 Stachl c:ontains the symbol under the readwrite head of the OM and everything on the left.'lcells. inFut dividers trial ralue of lwl g . Chopter  Section 11. We know that the union of two counttr. T. Then use the construction in Theorem 7. (e) Use a threetrack machine as shown below.1 2.): (qi. On the second track' we place dividers every ltr. we keep the current trial value for lrul. Stack? contains all the information to the right of the readwrite head. If the set of . For example.hlesets is countable and that the set of all recursively enumerable larrguages is countable. The pda we get from this consumes one input symbol on every move and never increasesthe stack contents by more than one symbol each time.1. On the third track.396 ANSWITRS +.e. Section 6.. Use Exercise 16.L) can be simulated by popping tlre a off Stackl and puttirrg a b on StachZ.6(ei. We then compare the cell contents between the markers.
L1 and t2 be two recursively enumerable languages and M1 and M2 be the respective T\rring machines that accept these two languages.so by Theorem 11. having lzsl ro*s and l5l columns.'tlrr. For any given 'u ( L+ .2 is not square. .. But this is not the ca.4 its complement is also recursive. 6 . . For eaclt split. consider all splits 'tD: 7ltLlz.. that the complement is not necessarily contextfree.lize. . 1 8 . Let .2 1.tion that 2s is not countable for finite 5 fails becauselhe table in F'igure 11. determine whether or not wi e L. When represented with an inprrt u. we can decide whether or not w € L+ . Look at a typical derivatioru Sl aStbB + aaSlbbB1a"S1b"B t on*r6n7R+ an+rb"+l.Sol..B +.. A contextfree languageis recursive.as we krrow.The result is a T\rrirrg machine that acr:eptsLtQLz. however. the result on the diagona... Thc a"rgurnent itttcrnpting to show by diagonaliza.. 1 . 1 1 . 1 4 . we nondeterministically choose M or M2 to processzrr.lcould be in one of the rows below.'.1 . 3 . } .utlolrs altn HIt'Irs ron Snlncrno ExpRcIsns 397 all languages that are not recursively enumerable were also countable. f r : . i l } 1 . Since for each'ur there are only a finite number of splits. lSl columns When we diagona.ge. Flom this it is not hard to conjecture that the grammar derives L : { a " + r b " t k . Section 11. then the set of all langrrages would be cxruntable. Note.
When that happens. Thtl equivaklrr(jc irrgunro\nt is straightfrlrwarrl. 7. f{rr all .5.mma. Section 11.Erl.sa€ S.rSp + . thc: B travcls to thc right to nxret the D. Irr the rtext sttlll. the first stcp is to crtlatt: thc scrrttlrrtial fclrrn o.\. The rrnrestricted gra.3 irrt: txpivalent to this exterrsiorr becarrse to arry given unrestricted grarrurrar we can always add startirtg rrrlrr.2. we can create one d and a return messengerthat will put the b in the right place a.rrdstop. The first part is achievtxl casily with the produr:tiorrs S + aAcDlaBcD A + a.. ABC r:anhe repla. For example. cEEc bEEb aE .nBt:nD. BD .398 ANSwHRS Fornnlly. with 5 f r for any s € .introduced in Example 11.S.rs in Dtllirritiorr 11.P). bv Bc +cB Bh +hB. s* s (V.. . (c) Working with contextsensitive grammars is not always easy.T.AclaBc. insert dummy variables on the righi whenever l"l > lul.3 1. In this problern. To get tlfs forur for unrestricted grammars.ub.B +CI) D .ced by A. The variables B and D will act as rrrarktlrs irrrcl urcsscrrgcrsto assrrrc that the correct nurnber clf b's and dts are r:reaterl irr the riglrt places. fhe idea of a messenger. is often usefirl.9}. the grammar carr be describedby G: (V u r)+ and L(G): { r e T * : .
wlrether or not a specified syrnbol o is ever written. Accept if so.chine. which ha. M halts implies the M writes f. reject otherwise. L}. This ca. / L(Mr) therr (M.:T. This would solve the halting problem. Given M and tu. rnoclifv M to get fr..nrl cr)rrrpiirc syrnbols rrrrtil yorr grtt ir mat<:h.. We carr do this by changing the halting configurations of M so lhat every one writes f..d plus a. / \ .*t. When it halls. is writterr.. Ihus.If we had an algorithm that checks tbr the equality of L(M). M accepts trrn if and only if w tha. Wc (:arr rlrglle frorn a. ?. we first rewrite it as tuft. we could apply the supposed algorithm to the rnodified rnachine.w) halts if and only if fr accepts some simple language. Given (M.but**" tT. 'l'he Ib ubB. Therefore M accepts {c. if we have an algorithm that tells us f implies that M halts. and qf?r. Suppose that a language is 'I'hen contextsensitive.u. that accepts rz. Chopter 2  Section 12.ru) does not halt arrcl we havc a solutiorr to tli trattirrg problem.w) rnoclify M to fr so that (M. and M writes 'Ihus. Because LR : ( LR ..F aF 4.marker D. there exists an Iba M that acceots it.lts if ancl only if a sllcr:ial syrntxrl. 'I'he rnachine {w : wR €. Clearlv. 6. the larrguage is cxrrrtextsen$itiveemd a.s b. easiest argument is from an lba.r'url. Given ?u. with configurations qgu.)halts. Given (M.fr) # L(M) then (M.n lba. check if the input was a. If the giverr problern was decidable. Just stirrt irt oppositt: enrl$ a.nbe done by M first checking the input irnd rurncmbcring whcthe:r thc input was a. . then stops. If L(fr\: t w o l a n g u a g e sw e c o u l d u s e i t t o s e ei f L ( f r \ : . 10.*.urroNs alo HrNrs ron Snrlcrrr Exnncrsls 399 Alternatively.w) we modify M so that it always halts irr the configuration eyw. say arl irrtrrlrluc:ctl syrntrol ff. then apply M to it. If I (.1 3. Now construct a simple Ttrring ma.Sol. Therefore Lft is contextsensitive. Then M carries out its rrorrrral cornputatiorrs. \ . say M1 . This would give us a solution of the halting problem. say {a}. there is an lha tha.} if and only if M halts.t can recognize arty strirrg of thc ftrrtr u. with a diff'erent messenger thatcr.nlba. we create a. a tF + Fr b. rxrntextsensitive grammar must exiut.t reversesa string and applies M is a. Sirrce there is tln lbtr.. we apply ft tu fr with a: f .
Suppose we had rr.2 3. Take the halting problem (M. the problem becomesthe question of Theorem 12.r t ( f r ) 1 = { o } * .n ir. To do this from first principles is a little harder. M will accept ia]" and it (M.ice's theorem. M accepts fl. If the specific instances of the proble[ fl.pn.3 and is therefore undecidable.400 ANSwER. only to guarantee that it exists. Take a universal T\rring machirre and let it simulate computations on an empty ta. then return true Whatever thc truth vahres of the various instances are.rison will stolr when this length is exceeded.lgoritltrrr to decide whether or not L(ML) C L(Mr). Remember that it is rrot nccessary to know what the I\rring machine actually is. there is always some T[rrirrg machirre that gives the right answer. The set is therefbre recursively enumerirbk). Then L (Mr) Q L (Mz) if and only it L (Mr) : fi. 2I..rap1. wQ corr$trlr(]ta Turing machine that behaves a^rf'ollows: if problem : gr1thr)n rcturrr firlst: if problem .We could therr construct a machine M2 such that . There would then exist rr. 1 6 . The universal Ttrring machine is therefore atr accepter for all Ttrring machines that halt when applicd to a blank tape. FI'om M get the grammar G by the construction leading to T h e o r e m1 .S 1 3 . L ( d ) = r ( d ) . accept the T\rring rnac:lftrc:bcirrg sittrulated.original Ttrring machine is amongst the T[rring rnachines generated hy A.But . But this contradicts Theorerm 12.w) does not halt. E.pz then return true : if problem : p. sint:e v/e can construct M1 frorn anv given grammar G. : {a}*.L (Gz) : E*.. Thus. 6. we havea sohrtion to tlx: blirrrk tape halting problem. See if the. If we take . Section 12.4).pe. Since there are $omc grirmrrrars for vrhich L (G) : L (G).*) and modify it (along the lines of Theorem 12. the compa.*) halts..r h e n . Whernevtlr the sintulated computations halt.i?2. ry that if (M. Suppose now the set were recursive..ntrlgrr rithrn A that lists all Ttrring urachines that halt on a blank tape input in some order of increasing lengths of the program.L (Mr): a and apply the algorithm.3. the undeciclability follows from R. Sincc the length of the original program is fixed. and some for which this is not ${r.
( r s f u ..Solurrons errp Hrrurs noR SnlncreD ExERctsHs 401 fi L (fr): o. g . 3. JEJ JE.f afld u$e the assumed algorithrn. 0 ) E + :v12 . 9 * . w{.wfu. 4( 1 ..y2)+2 : . There is no MPCsolution becarrse one string would have a prefix 001. For a oneletter alphabet. A ( 1 . tte original MPCproblern has a solution if and only if the new MPCproblern has a solution.3. g(r.we get the solution (*..s 1 ) ) :A(1. ("). (a) The problem is undecidable. they can all be checkctl and therefore the problem is decidable.y1)+1 :A(1.subtr(I..3 r.y))) ' greater 7. if this problem were decidable. y ) : A ( 0 . we would have an algorithm for deciding the original MPCprohlem. Since wrwi. the other 01.. llsing the firnctiorr subtr irt Example 13.subtr(r. we form tuf. Chopter 3  Section 13.uk: (w[... Section 12. s' (a) A ( 1 .A) : subtr(1. 2.) n'n.I Since there are only a finite numher of subsets. (t) : @ and . A PCsolution is u3u4w1 : uzutat..1 ) ) .0):1.: {A}. If it were decidable. we could get a solution of the haltirrg probletn.. rherefore.u.n} such that flrrl:fluil. a ): m t t l t ( * . there is a PCsolution if and only if there is some subset J of {1.. then .1 2...f)f . Given u)Lj'trz.. 5.
A solutionis V 1. T h e o n l y v a l u e s f r t h a t g i v ea o positivegrare 0 and 1. t ) . n ' ) : A ( 1 .I ) ) :A(1. Then A ( Z . 1 1 1 1 . Assume that for A : I.. {a'".a) :2U + S.c}.in p is {0.D. the only possibleidentification of Vr is with the entire derivcd string. Cx7: {ru} and A : {c}..V 1 I + V 2 : V t V z V1+ V2 : Vt + Vt * VzL: VsVr. fiom part (a).A ( z . we haveA (2.1. 1}.1) 3.. This results in a doubling of the string and n: 5. t h e uA : 3 2 ' " . Since A(2. giving a minimum of value of g : 1. and so on. so the doma. n . (b) UseCr : {a.If 2"'IA3:0. tFor exanrple 1 * 1 : 1 + 1 1 +1 : 1 1+ 1 1* 1 1:. the r: is removedby VnVz + VVy 3.V2{r V lVrhrVzbl crVpl At the end. Section 13. r V 2 : V r .b.0): A(1.2n+1) :2n* 3.4OZ AuswnRs (b) With the resultsof part (a) we can useinductionto provethe next identity. wtt havqt basisand the equation iu true fbr all y. At every step. The nonterminal is r usedas a boundary betweenthe left and right side of the target string and the two ru's are built sirnultaneously by V1rV2+V1 ar.2..2 > 1}. Tlrt:rcfrrrc pa(2"*sB):1.2 1. ir 15.
bSz Pa . 1?6 times. Sz + aSz P3 ^9r b.ecrnD ExERcrsEs 403 Section 13. Take any string. P1 ^9* ^9r^92 P2 5r +a5r. The solution here is reminiscent of the use of messengerswith contextsensltlve grammars. .92 + . ab+n nb+br rc .3 1.urtotrsann Htrvrs noR Snl.. and so on.r way. This can be derived from a127 by applying & + &&& once and (t + ee.\.9r.\. this is one way to solve Exercise 7. 5.\. say a255.Sz . Thus every string in L (aa*) can be derived. 8. Then a127 can be derived from a63 in a simila. Although this is not so easy to see.91 + .Sol.
.
A. D'iscrete Mathematics' Macmillan. New York: John Wiley. Dennis. Englewood Cliffs. Johnsonbaugh. V.on of Comp'ilers' Chichester. Kovahi. New York: Academic Press. J. Harrison. Hopcrofh and J.cations.ni. R.B. Aho and J. "Computations and Automata. and Computafion.: Prentice Hall. Cambridge: Cambridge University Press. 1973. D. Second Edition. N.nd lts Appli.ta Theory. A.J.nes. 7978.ling. 1972' The Theory of Pars'ing. Formal Languages. Introduction to Automa. 1978. N.te Automata Theory. New York: Z. Fourth Ed. P. TTanslation. Vol. 1996.J. and J. 1979. Mass. J.tching and Fi. Englewood Cliffs. 1985. Reading' Mass.: AddisonWesley.: Prentice Hall' M. ?he Des'ign and Constructi. 405 . Swi. Qualitz. Denning.R for Further eoding A.cso. and Compi.Languages. Introduction to Formal Language Theory. Ullman. Languages and Computafion. 1. Machi. Reading. New York: McGrawHil1. 1978. Hunter. 1981. Salomaa. A. D." it Encycloped'ia of Mathernat'i. J. Salomaa. R. E. E. Ullman.: AddisonWesley.
.
18 star. 344 cornplexity classP. 324 complete systerns.5 childparentrelation in a tree. 344 tirne. 26 axiom$. complement of a set. 3 18 of a language. 99 positive. 100 of regular languages.146 baseof a cycle. 136 of a grammar. 233 computation. 326 computability. 25 deterministic.172 D dead configuration. grammars.330 algorithm.15 25 configurationof an autclmatorr. 348 324 consistent$y$tems. 354 composition.355 cycle in a graph. 299 DeMorgantslaws. ?99 computablefunctiotr.26 Ackerman's function. contextfreegrammars. 353 complexityclassNP. conjunctivenormal form. control unit thesis.289 contextsensitive languages.325 ChurchTtrringthesis. 321 corrcaterration 18 of languages.334 B BackusNaurfcrrnt.141 inherent.ton.163 space.I blank.324. 290 contextsensitive 25 of atr automa. 26 nondeterministic.165 Church'sthesis.I CYK algorithm. 2.323 valid. 8 Chomskyhierarchy.52 decidability.15 ambiguity. complexity.325 closure. 2?3 blanktape halting problem. 4 407 . 18 closureproperties 2L3 of contextfreelanguages.295 Chomskynormal forrn. 305 C Cartesianproduct of sets.2. of strings. 8 simple.354 CookKarp Cook'stheorem.144 automaton. 195 deterministic.126 125 contextfreelangrrages. 149.A accepter.343 of a grammar.246 alphabet.228 models.
195 E empty set. 131 deterministic finite accepter. 4 end markers for an lba. 25. 4 distinguishable sets in a dfa. 268 equivalence. 354 L lambdaproductions. 91 regular. 103 dpda. 179 of a Thring machine. 335 language families. 36 diagonalization. 89 linear. 91 Llgrammars. 36 dfa. 224 hierarchy of language families. 289 leftJinear. 196 accepted by an lba. 7 I incompletenesstheorem. 38 accepted by a dpda. 5 total. 226 internal states of an automaton. 324 indistinguishable states in a dfa. 149. 21 leftmost. 2 functions.279 disjoint sets. 283 graplr. 89 leftmost derivation. 201 Lsystems. 51 accepted by a Ttrring machine. S G grammar. 126 contextsensitive. 130 partial. 17 accepted by a dfa. 25 instantarreous description of a pushdciwn arrtomaton. 5 range. 66 inherent ambiguity. 42 final state. 36 intractable problenrs. 233 domain. 168 H halting problems for Turing machines. 156 language. 140 unrestricted. 35 formal languages. 55 of grammars.250 of dfa's and nfh's.36 finite automata. 24 F family of larrguages. 19 contextfree. 301 halt state of a T\rring machine. 129 linear bounded automata. 15. 340 . 7 Greibach normal form. 5 partial. 42 lba.408 Iwnex dependency graph. 129 rightmost. 63 labeled. 89 simple. language. 103 homomorphistn. 154 derivation.73 generated by a grammar. 5 computable. 270 leftlinear grammar. 229 ociated with regular expressions. 131 yield. 275 homorrrorphir: image of a. 271 enumeration procedure. 89 rightlinear. 36 input file. 144 initial state. 129 derivation tree. 2L generated by a Post system. 7 of automata classes. 270 linear grammar. 271 accepted by an nfa.
276 reduction rrf states in a dfa. 313 pda. 269 relation in a tree. 304 polynomialtinre. 334 powerset. 331 monus. I labeled. 206 for linear larrguages. 355 problem. 355 rrpda. 4 Post system. I P parsing. l3(i top*down.trix gramrnar. L8 of a string.48 noncontracting grarrtmars.nguage. 47 nondeterminism.mmars. 136 path in a graph. 62 reductiorr of undecidable problems. 25 MPCsolution. 104 root of a tree. 85 PCsolution. 311 rightlinear gra. 114 polynomialtime reduction. 8 sintple. 338 membership algorithm. 9 proper order. 175 phrasestructure gramrrrar. 8 pattern matching. 165 NPcomplete problems. 327 move of an a. 195 nondeterminiutic. I contradiction.nt.21O for regular languages. 224 projector function. 89 rightmost derivation.T2 productions of a grarrrrrtar. 67 minimalization opertrtor. 325 recursive la. 129 right qrrotient of languages. 293 minimal dfa. 111 for contextfree languages.277 recursively erturnerable languages. 5 reverse of a language. L5 rewriting systems. 339 mir.19 program of a Ttrring macltirre. 8 o order proper. 43 relation. 4 pumping lemma for contextfree langirages. 7^ regular grammar. 175 deterministic. 290 nondeterministic finite accepter. 222 recur$ive futtction. 326 prirnitive recursive functions. 269 propcr subset.115 pushdown automata. 176 !1 e: R readwrite head of a T\rring machine. 4 primitive recursiorr. 149. 355 regular expressions. 136 exharrstive search. 172 for contextser$itive languages. 337 Rice's theorem. 328 primitive regular expressions.334 normal form of a grarnmar. 312 Post corresporrdence 313 modifierd. 313 murtrcursivefutrctions. 11 induction. . 331 N nfa. 338 pigeonhole principle. 52 nonterminal consta.utorntrtotr.It'tlpx 409 M Markov algorithrtt. 177 null set. 326 proof techniques. 89 regular language.
253 mrrltitape. 8 Y yield rlf a derivationtree. z zero function. I6 urrffix. 112 sta. 16 srrbset. 39 trulu. 26 transitiorr firnction.l. J. 244 U urtdcxlitlir.t J I(r) 5 Io 'l T* w walk in a graph.410 Ilt nr:x 1 t r q h 5 t E tl J satisliability protrlcrn.a E. 300 for cotrtcxtfrtxl lir. 354 transducxlr.. 156 start. 261 with multiple tracks. 175 alphabet. 177 standard representation for regular languages. 3 countable. 8l trap statc.trmilr. 266 urrrtlstrir:tedgrammar. 19 timecomplexity.53 V vilriahle of a grarrtrrrar. 258 trondeterrnirristic. 251 upacecomplexity.teentryproblem.153 r47 T tape alphabet. 36 .17 sentential forrn. 140 simulaLion. 21 set.ngrrages. l5 prefix. 226 with sta. 255 with scmiinfinite tape.tiorr.267 set operations. 233 T\rqing machine. 36 genera.J' V R F .26i3 offline. 267 uncounta. 334 terminal symbol. 109 syntan of a prograurrning langua. 308 trnit llroductions. 253 tractatrkl prohlems.ge. 16 successor func. 37 transition graph.ngrrages. 158 urrivtlrutll set. 253 starrdirrd. 344 tracks rlrr ir ta. 304 storage of a. 283 uscltlJsproductions. 343. 13L r* 3 I.tions. l5 olletrir. 15 Iength. 266 T\rring's thesis. 19 nullable. 326 . 4 substring.yoption. i347 sernanticsof prograrnrning langrrages. exterrdcd.bleprohlem. 326 symmetric difference of two sets.I48 sentence.223 tape of a Ttrring rnachirre. 221 multidimensional.4 proper. 25. 4 rrrtivt)r$rrlTtrring machine. lS ernpty.r:k. II 3 for recursively enurrrerable lir. 344 sttr. 3 sgra.pe. 8 T\rringcomputable function. 222 terminal constant. 25 strirrg.lizecl. 19 useless.ble.l .251 rrniversa.ton.n automa. 177 start symbol. J ilfi .
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.