P. 1
Smale - Some Remarks on the Foundations of Numerical Analysis

Smale - Some Remarks on the Foundations of Numerical Analysis

Views: 5|Likes:
Published by Mauro Sellitto

More info:

Published by: Mauro Sellitto on Nov 29, 2012
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

09/28/2013

pdf

text

original

SIAM BEVIEW ?

1990 Sociel¸ Jov InduslviaI and AppIied MalIenalics
VoI. 32, No. 2, pp. 211-220, June 1990 001
SOME BEMABKS ON THE FOUNBATIONS OF NUMEBICAL
ANALYSIS*
STEVE SMALEl
AIslvacl. TIe pvoIIen oJ incveasing lIe undevslanding oJ aIgovilIns I¸ consideving lIe Joundalions
oJ nunevicaI anaI¸sis and conpulev science is consideved. TIe scIisn Ielveen scienliJic conpuling and
conpulev science is discussed Jvon a lIeovelicaI pevspeclive. TIese lIeovelicaI considevalions Iave an
inleIIecluaI inpovlance vIen vieving lIe conpulev. In pavlicuIav, lIe Iegilinac¸ and inpovlance oJ nodeIs
oJ nacIines lIal accepl veaI nunIevs is consideved.
Ke¸ vovds. nunevicaI anaI¸sis, Joundalions, conpIexil¸, aIgovilIns
AMS|MOS) suIjecl cIassiJicalion. 65
Inlvoduclion. I vouId IiIe lo discuss a pvoIIen aIvead¸ vaised I¸ JoIn von
Neunann in 1951 |¨TIe OenevaI and LogicaI TIeov¸ oJ Aulonala,¨ in [von Neu-
nann, 1963]). Ils veIevance conpeIs ne lo quole once nove |i.e., [SnaIe, 1985]) Jvon
lIis avlicIe.
Von Neunann vvole·
TIe lIeov¸ oJ aulonala, oJ lIe digilaI, aII ov none l¸pe, as discussed up lo nov, is cevlainI¸ a
cIaplev in JovnaI Iogic. Il vouId, lIeveJove, seen lIal il viII Iave lo sIave lIis unallvaclive pvopevl¸
oJ JovnaI Iogic. Il viII Iave lo Ie Jvon lIe nalIenalicaI poinl oJ viev, conIinaloviaI valIev lIan
anaI¸licaI.
. . . a delaiIed, IigII¸ nalIenalicaI and nove speciJicaII¸ anaI¸licaI, lIeov¸ oJ aulonala and oJ
inJovnalion is needed.
Von Neunann Ias anlicipaled vIal I IiIe lo caII lIe conJIicl Ielveen scienliJic
conpulalion and conpulev science. Tvo suIjecls vilI nan¸ connon goaIs Iave
gvovn apavl. On lIe lIeovelicaI side especiaII¸, lIis conJIicl seens Ivoad and deep.
Fov exanpIe, nunevicaI anaI¸sls |I speaI oJ nunevicaI anaI¸sls as lIe lIeovisls oJ
scienliJic conpulalion) Iave a disdain Jov Tuving nacIines. On lIe olIev side,
conpulev scienlisls oJlen nininize lIe inpovlance oJ caIcuIus in lIe coIIege cuvvic-
uIun. TIese ave jusl signs oJ a Iiggev scIisn. TIe JoIIoving IillIe laIIe iIIuslvales lIe
conlvasl I an lv¸ing lo naIe.
ScienliJic Conpulev
Conpulalion Science
MalIenalics Conlinuous Biscvele
FvoIIens CIassicaI Nevev
OoaIs FvaclicaI, Innediale Long Bange
Foundalions None BeveIoped
ConpIexil¸ UndeveIoped BeveIoped
¨MacIine¨ None Tuving
TIis dispIa¸ cevlainI¸ ovevsinpIiJies lIe silualion, so I viII eIaIovale. Conlinuous
nalIenalics inpIies lIe cenlvaIil¸ oJ veaI nunIevs and incIudes caIcuIus and diJJev-
enliaI equalions. TIe discvele nalIenalics oJ conpulev science enpIasizes Iogic,
*
Beceived I¸ lIe edilovs OcloIev 5, 1989: accepled Jov puIIicalion OcloIev 5, 1989. TIis papev vas
pvesenled as lIe 1989 JoIn von Neunann Lecluve al lIe SIAM 1989 NalionaI Meeling in San Biego,
CaIiJovnia, JuI¸ 17-20. TIis vovI vas suppovled in pavl I¸ lIe NalionaI Science Foundalion.
l
MalIenalics
Bepavlnenl, Univevsil¸
oJ CaIiJovnia, BevIeIe¸,
CaIiJovnia 94720.
211
212 STEVE SMALE
conIinalovics, and nunIev lIeov¸. ¨Nevev pvoIIens¨ needs lo Ie augnenled I¸
pvoIIens oJ nunIev lIeov¸.
I Iave JeIl lIe pvessuve oJ innediale goaIs vIen nunevicaI anaI¸sls conpIinenl
a papev oJ nine, Iul uvge ne lo inpIenenl lIe aIgovilIns on useJuI pvoIIens.
TIeovelicaI conpulev science seens nucI JavlIev Jvon sucI concevns. AIgovilIns in
nunevicaI anaI¸sis ave pvinaviI¸ a neans lo soIve pvaclicaI pvoIIens, vIiIe in
conpulev science, aIgovilIns ave sludied s¸slenalicaII¸ in lIeiv ovn vigIl.
Lalev, pvoIIens oJ lIe Joundalions oJ nunevicaI anaI¸sis viII Ie addvessed. Fov
lIe nonenl nole lIal aIgovilIns ave lIe nain oIjecl oJ slud¸ in scienliJic conpula-
lion, ¸el lIeve is nol a JovnaI deJinilion oJ aIgovilIn. I an veninded oJ Iov lIe
deveIopnenl oJ lIe deJinilion oJ diJJevenliaIIe naniJoId vas so inpovlanl in lIe
Iislov¸ oJ diJJevenliaI lopoIog¸.
In conlvasl lo nunevicaI anaI¸sis, lIe use oJ Tuving nacIines gives lIe conpulev
scienlisls a uniJ¸ing concepl oJ aIgovilIn, veII JovnaIized. TIus conpIexil¸ lIeov¸
can speaI oJ Iovev Iounds oJ aII aIgovilIns vilIoul anIiguil¸.
A suilaIIe conpIexil¸ lIeov¸ Jov nunevicaI anaI¸sis sIouId al Ieasl possess
lIeovens Jov lIe conpuling line oJ Iasic aIgovilIns, delevninislic ov pvoIaIiIislic,
in levns oJ nunIevs, incIuding desived pvecision and ¨size¨ oJ pvoIIen associaled lo
lIe inpul. See lIe Appendix on lIis suIjecl.
I Iope lIe aIove connunicales n¸ viev oJ nunevicaI anaI¸sis as an ecIeclic
suIjecl vilI veaI Joundalions. On lIe olIev Iand, ils acIievenenls ovev nan¸
cenluvies, and especiaII¸ since lIe vevoIulion oJ lIe conpulev, Iave an undeniaIIe
gvealness.
I an veninded oJ lIe discipIine, ovdinav¸ diJJevenliaI equalions vIicI, in 1960,
vas sonelines vegavded as a Iag oJ lvicIs. TopoIog¸ and nodevn nalIenalics in
genevaI Iave since IeIped give unil¸ lo Iavge povlions oJ lIal suIjecl avound lIe
deveIopnenl oJ d¸nanicaI s¸slens.
A najov oIslacIe lo veconciIing scienliJic conpulalion and conpulev science is
lIe pvesenl viev oJ lIe ¨nacIine,¨ i.e., lIe digilaI conpulev. As Iong as lIe conpulev
is sinpI¸ seen as a Jinile ov discvele oIjecl, il viII Ie diJJicuIl lo s¸slenalize scienliJic
conpulalion. TIe nacIine Ias a cenlvaI voIe in lIeov¸ due lo lIe inJIuence oJ Tuving.
I veJev lo lIe picluve· |1) nacIines, |2) lIe devived nolion oJ conpulaIIe Junclion
deJined as lIe inpul-oulpul nap oJ a nacIine, and |3) an aIgovilIn as a conpulaIIe
Junclion logelIev vilI a cIass oJ pvoIIens il soIves. TIus il is inpovlanl lo conJvonl
lIe queslion· WIal is lIe naluve oJ lIe nacIine?
OJ couvse, ve aIvead¸ Iave an inpovlanl ideaIizalion oJ lIe digilaI conpulev in
lIe Tuving nacIine. Hovevev, lIe nallev does nol end lIeve. IdeaIizalions ave nol
necessaviI¸ unique. As an exanpIe, Jov lIe naleviaI vovId ve Iave ideaIizalions oJ
Nevlon, oJ Einslein, and oJ quanlun necIanics. EacI oJ lIese ideaIizalions iIIuslvales
lIe lvulI oJ cevlain aspecls oJ lIe pI¸sicaI vovId. And, oJ couvse, cevlain oJ lIese
ideaIizalions Iave Ieen uniJied.
Considev lIen lIe pvoIIen oJ veconciIing lIe digilaI nacIine vilI lIe conlinuous
nalIenalics oJ scienliJic conpulalion. Nevlon Jaced an anaIogous pvoIIen in vviling
¨Fvincipia¨ |see, e.g., [SnaIe, 1988]). Nevlon's inlended nodeI oJ nallev vas Iased
on veaI nunIevs Jov Iis caIcuIus and geonelv¸. Yel Ie IeIieved, IiIe olIev scienlisls
oJ Iis da¸, lIal lIe univevse vas conposed oJ alons, indivisiIIe and discvele.
EvenluaII¸ Ie vesoIved lIis pavadox I¸ laIing a Iinil as lIe nunIev oJ alons Iecane
densev and densev. TIis couId Ie jusliJied I¸ lIe Iavge nunIev oJ alons in a naleviaI
suIslance.
I IeIieve lIis IillIe IislovicaI oIsevvalion IoIds a Iesson Jov us on undevslanding
lIe conpulev. TIe IigI densil¸ oJ valionaI nunIevs avaiIaIIe Jov use indicales lIal
FOUNBATIONS OF NUMEBICAL ANALYSIS 213
using veaI nunIev inpuls is nol a Iad ideaIizalion. TIe Nevlon slov¸ is a vesponse lo
an oIjeclion I neel· ¨Hov can a digilaI conpulev inpul an avIilvav¸ veaI nunIev?¨
WIen asIing nunevicaI anaI¸sls Jov a JovnaI deJinilion oJ aIgovilIn, I usuaII¸
Iave nol veceived a salisJaclov¸ ansvev. TIeve vas an exceplion. AvieI IsevIes vepIied
lo n¸ quev¸ vilI ¨a Fovlvan pvogvan.¨ TIis naIes a Iol oJ sense. BecaII lIal Fovlvan
does laIe veaI nunIev inpuls.
BIun, SIuI, and SnaIe [BSS, 1989] allenpl lo neel lIe pvoIIen oJ giving a veaI
nunIev ideaIizalion oJ lIe nacIine. |See aIso [BIun, 19891.)
TIe nodeI descviIed in
lIal avlicIe na¸ Ie seen as a nalIenalicaI ideaIizalion oJ lIe essenliaIs oJ a Fovlvan
pvogvan. Fovlvan is aIvead¸ JovnaI Iul valIev conpIex. Lel ne descviIe vIal ve Iave
done vilIoul nucI pvecision.
Ouv nodeI is nove oJ a sinpIe aIslvaclion oJ a JIovcIavl oJ a conpulev pvogvan
lIan an innovalive discovev¸. TIe JoIIoving Nevlon-MelIod nacIine is a good
iIIuslvalion.
Inpul x E B
L/
J|x)
I J|7) 12·
,
Yes/ No
W
oulpul.
TIis exanpIe is a vevsion oJ a pvogvan Jov using Nevlon's nelIod naiveI¸ lo
Jind appvoxinaleI¸ a zevo oJ a veaI poI¸noniaI, J|x) =
Ed
aix'. TIeve is a space oJ
inpuls Sl, lIe veaI nunIevs. TIeve is an oulpul space, in lIis case aIso B11. Moveovev,
B1 pIa¸s lIe voIe oJ slale space vIeve lIe conpulalion x' = x -J|x)/J'|x) laIes pIace.
TIeve is a dislinguisIed sel QM C aa oJ inpuls Jov vIicI lIe nacIine IaIls. TIus QM
consisls oJ iniliaI x sucI lIal Nevlon's nelIod is deJined ilevaliveI¸ up lo an x' Jov
vIicI I J|x' ) I
· e. QM is lIe HaIling sel |unJovlunaleI¸ caIIed B.E. sel in [BSS]) oJ M
|aJlev Tuving).
An inpul-oulpul nap
OM· QnM
-I BB is deJined I¸ JoIIoving lIe JIov oJ lIe cIavl.
Il is eas¸ lo inagine exlending lIe exanpIe lo Nevlon's nelIod in nan¸
vaviaIIes. In genevaI, a Jinile-dinensionaI nacIine viII Iave an associaled inpul space
J, oulpul space &, and slale space 59, eacI sone veaI Cavlesian space. TIe nodes oJ
lIe genevaI nacIine do valionaI conpulalions ov ave IvancI nodes |covvesponding
lo ¨iJ, lIen¨ inslvuclion) as in lIe exanpIe |Iesides an inpul node and oulpul
nodes). Fov eacI nacIine M lIeve is a IaIling sel QM C Jv and an inpul-oulpul nap
OM· QM
--
&. TIe JovnaI deJinilions ave cavvied oul in [BSS]. TIe deJinilion oJ a
genevaI, pevIaps inJinile-dinensionaI nacIine, Ias a sIigIlI¸ nove lecInicaI JIavov.
Bul il is appavenl Iov naluvaII¸ lIe nodeI JoIIovs and is cIose lo acluaI pvogvanning.
In Jacl, lIeve is an equivaIenl deJinilion in levns oJ a Iisl oJ inslvuclions.
A conpulaIIe nap |ovev
B1), 0·
Q
--
B', Q C B
I,
is a nap vIicI is lIe inpul-
oulpul nap oJ sone nacIine M. Il can Ie cIecIed lIal lIe conposilion oJ conpulaIIe
naps is a conpulaIIe nap: valionaI naps ave conpulaIIe, as veII as lIe cIavaclevislic
Junclion oJ lIe sel )x E B1 I x _ 0?.
214 STEVE SMALE
Since lIe avilInelic opevalions on a conpulev ave so pvinav¸, lIe conpulalions
on ouv nodeI ave veslvicled lo valionaI Junclions |aIlIougI evenluaII¸ lIe povev couId
Ie expanded lo incIude videv cIasses oJ Junclions). TIus lIe lIeov¸ oJ lIese nacIines
Ias a pvonounced aIgeIvaic JIavov.
TIe aIgeIvaic enpIasis pevnils an exlension oJ lIis lIeov¸ oJ conpulalion lo
lIe silualion vIeve lIe veaIs ave vepIaced I¸ a ving Jvon sone genevaI cIass. IJ lIe
ving B is nol a JieId, lIen valionaI naps ave vepIaced I¸ poI¸noniaI naps. IJ B is nol
ovdeved, lIen · is vepIaced I¸ = in lIe IvancI nodes. TIus vilI nacIines ovev lIe
inlegevs Z, ov nacIines ovev C, lIe conpIex nunIevs ave deJined.
TIe genevaIizalion pevnils sone unil¸ vilI lIeovelicaI conpulev science. Con-
pulaIIe Junclions ovev B = Z coincide vilI lIe lvadilionaI conpulaIIe Junclions |ov
pavliaII¸ vecuvsive Junclions). TIis unil¸ is deepened I¸ passing lo conpIexil¸ lIeov¸
as lIe JoIIoving indicales.
TIe ceIeIvaled and IeauliJuI pvoIIen ¨does F = NF?¨ is cenlvaI lo conpIexil¸
lIeov¸ oJ conpulev science |see [Oave¸ and JoInson, 1979]). TIe inpovlance oJ
¨F = NF?¨ is Iased on Sleve CooI's NF-conpIeleness lIeoven [CooI, 1971] and
suIsequenl vovI oJ BicI Kavp [Kavp, 1972].
TIe slud¸ oJ nacIines ovev C ov ovev B1 ¸ieIds a naluvaI exlension oJ lIe pvoIIen
¨F = NF?¨ lo lIe conpIex nunIevs and lo lIe veaIs.
¨Boes F = NF ovev C? ovev B'?¨ seens lo Ie diJJicuIl, and Iecause oJ ouv nev
NF-conpIeleness lIeoven, conlains sone oJ lIe inpovlance oJ lIe oviginaI pvoIIen.
TIe aIgeIvaic cIavaclev oJ lIe nacIines I Iave descviIed cavvies ovev lo lIe pvoIIen
¨F =
NF?¨, IolI in lIe nev |ovev C) and cIassicaI selling |ovev 7).
TIe NF-conpIeleness lIeoven ovev C oJ [BSS] |lIe pvooJ lIeve Jov lIe veaIs
cavvies ovev sinpI¸ lo covev lIe conpIex case) assevls lIe exislence oJ a univevsaI
pvoIIen anong NF |decision) pvoIIens ovev C. TIis univevsaI pvoIIen is essen-
liaII¸ HiIIevl's NuIIsleIIensalz, veJevved lo as HN. TIe inpul is a sel oJ poI¸noniaIs
FI, * * *, Fn· Cn
__
C. TIe oulpul is one iJ lIe Fi Iave a connon zevo: il is zevo
olIevvise.
TIis pvoIIen is in ¨NF ovev C¨ Iecause lIeve is a s¸slen oJ lesls F
E Cn vIicI
covveIales appvopvialeI¸ vilI lIe )Fi[ Iaving a connon zevo. NaneI¸, given inpul
lIe Fi and v E Cn, lesl in poI¸noniaI line iJ Fi|v)
= 0 aII i. Fov pvecision oJ aII oJ lIese
levns, see [BSS].
Il is easiev lo sIov lIal a pvoIIen is in NF |non-delevninislic poI¸noniaI line)
in conlvasl lo F |poI¸noniaI line) in lIe sane sense lIal il is easiev lo cIecI a pvooJ
oJ a lIeoven lIan lo Jind a pvooJ.
Ouv NF-conpIeleness lIeoven assevls lIal an¸ NF pvoIIen ovev C can Ie
veduced in poI¸noniaI line lo HN. TIus an aIgovilIn Jov HN ¸ieIds an aIgovilIn
Jov an¸ pvoIIen in NF ovev C. As a consequence, iJ HN is in F, lIen F = NF ovev C
and convevseI¸.
TIeve ave aIgovilIns ovev C Jov lIe HiIIevl NuIIsleIIensalz and lIeve is a Iavge
Iilevaluve on lIis suIjecl. [BvovnaveII, 1987] gave a slvong Iound. See lIis papev as
veII as [BuIe, 1989] and [Ievavdi, 1989] Jov lIe nalIenalics and Iislov¸ oJ lIe
pvoIIen.
TIe exislence oJ an aIgovilIn Jov HN inpIies as a covoIIav¸ lo ouv NF-conpIele-
ness lIeoven, lIal an¸ pvoIIen in NF ovev C is decidaIIe. TIis Jacl, in conlvasl lo
lIe lvadilionaI anaIogue, is nol oIvious. TIe conlinuun oJ guesses cannol Ie used lo
descviIe a decision pvoceduve.
TIe veaI vevsion oJ an NF conpIele pvoIIen [BSS] is caIIed 4-Feas. Heve lIe
inpul is a poI¸noniaI J· Bn
___
B oJ degvee 4. TIe oulpul is
1 iJ
J Ias a zevo, 0 iJ
FOUNBATIONS OF NUMEBICAL ANALYSIS 215
nol. TIe Iislov¸ oJ aIgovilIns |ovev B1) Jov lIis pvoIIen pevIaps slavls vilI [TavsIi,
1951]. [Cann¸, 1988] Ias Jound good conpIexil¸ Iounds |sliII exponenliaI) Jov 4-
Feas in a nove genevaI conlexl. [Benegav, 1989] Ias a vev¸ genevaI, s¸slenalic, and
conpIele accounl vilI good Iounds. TIe pvoIIens 2-Feas and 3-Feas ave in F |Iave
poI¸noniaI line aIgovilIns), as sIovn in [TviescI, 1989].
Appendix. Bound-oJJ evvov, appvoxinale soIulions, and conpIexil¸ lIeov¸. Since
nacIines ovev B1 do exacl avilInelic il is naluvaI lo asI, Iov couId lIese nacIines
Ie IeIpJuI in nunevicaI anaI¸sis vIeve vound-oJJ evvov is so inpovlanl? Even using
exacl avilInelic, aIgovilIns can usuaII¸ onI¸ soIve nunevicaI pvoIIens appvoxi-
naleI¸, lo vilIin ¨accuvac¸ e.¨ ConpIexil¸ lIeov¸ oJ conpulev science nusl Ie
nodiJied lo deaI vilI lIese pvoIIens iJ il is lo Ie useJuI in nunevicaI anaI¸sis.
LavgeI¸ JoIIoving [BSS] ve suggesl a va¸ oJ Iov nacIines ovev B1 na¸ Ie used
lo give a Iasis Jov a conpIexil¸ lIeov¸ Jov nunevicaI anaI¸sis.
Conpulev scienlisls sa¸ lIal an aIgovilIn deJined I¸ a nacIine M is lvaclaIIe
|ov poI¸noniaI line, ov in F) iJ lIe line T|¸)
=
Tine|¸) oJ conpulalion associaled
lo inpul ¸ salisJies lIe Iound
|*
T|¸)-
c
|size ¸)
q
vIeve lIe conslanls c and q depend onI¸ on M. Heve line is lIe nunIev oJ Tuving
nacIine opevalions and size is lIe nunIev oJ Iils. A pvoIIen is lvaclaIIe iJ lIeve is a
lvaclaIIe aIgovilIn soIving il.
A va¸ oJ lvansIaling |*) inlo a Ianguage oJ nacIines ovev B1 is lo Jivsl Iel line
T|¸) Ie inlevpveled as lIe nunIev oJ nodes lvavevsed in lIe conpulalion oJ
qn|¸).
TIe inpul space oJ an |inJinile-dinensionaI) nacIine is lIe Iineav space BJ = W
oJ aII sequences oJ veaI nunIevs oJ lIe Jovn
Y=|Y1,Y2,..*,YnOA0,,**)
|lIe oulpul space a is lIe sane). TIen s|¸) =
size|¸)
is laIen as n. Fov lIe NF-
conpIeleness lIeoven lIis inlevpvelalion oJ |*) neans lIal M is in F ovev B1.
Fov puvposes oJ nunevicaI anaI¸sis il is inpovlanl lo JovnaIize lIe nolion oJ
¨pvoIIen¨ and lo expand on lIe aIove deJinilion oJ lIe size oJ an inpul.
Tovavd lIis end, Iel lIe space oJ pvoIIen inslances |ov adnissiIIe inpuls) Ie
denoled I¸ Y C -F. A pvoIIen is a suIsel 7 c Y x a |& lIe space B' oJ oulpuls). TIe
idea is lIal Jov eacI ¸
c
Y, lIeve is a sel oJ soIulions 0q|¸)
E a and 7 is lIe gvapI oJ
0,
)|¸, 0q|¸))
E Y x &. Bul 0 is nol a nap oJ Y inlo a unIess lIe pvoIIen Ias unique
soIulions and even lIen ve cannol expecl 0 lo Ie diJJevenliaIIe. On lIe olIev Iand,
lIe condilion dinension 7= dinension Y is veasonaIIe Jvon IocaI uniqueness
and JvequenlI¸ 7 is a snoolI naniJoId. Fov inlevesling pvoIIens, lIe naluvaI nap
7v· 7-- Y deJined I¸ 7v|¸,z) = ¸ is Iellev IeIaved lIan 0. IJ 7 is snoolI, -v is
diJJevenliaIIe even al iII-condilioned pvoIIens |vIeve lIe devivalive oJ 0 does nol
exisl). TIe inlvoduclion oJ 7v· 7
--
Y Ias nan¸ olIev advanlages.
B¸ a sIigIl aIuse oJ nolalion lIe seavcI Jov an aIgovilIn Iecones an allenpl lo
invevl 7- ov Jind

Y
--
7 sucI lIal
0
is a one-sided invevse oJ 1v. TIe oviginaI pvoIIen
is sIigIlI¸ lvansJovned lo lIe pvoIIen· given ¸ Jind x sucI lIal 7v|x)
=
¸.
ExanpIe 1. TIe pvoIIen oJ Jinding a zevo oJ a poI¸noniaI oJ one conpIex
vaviaIIe. Heve ve na¸ laIe Y lo Ie a space oJJ ov ils coeJJicienls |ao, ..., ad), ad # 0,
veaI and inaginav¸ pavls, vIeve J|z)
=
ZiL=0 a,z'. |Heve d is nol Jixed.) TIe space
7C Yx , =C, is lIe sel )|J ) EYxC IJ|v)
=
0O.
216 STEVE SMALE
ExanpIe 2. TIe pvoIIen is lo Jind aII vools oJ
J
Heve Y is lIe sane as in
ExanpIe 1, Iul a is Cd and 7= )|d)
E Y7 C
J J|z)
=
adI
|SlviclI¸ speaIing, |J A,, * * *, ld, 0, 0, .)
IJ il is veslvicled lo lIe case ad = 1, lIen 7 is canonicaII¸ isonovpIic lo Cd, lIe
space oJ
|B,,
* *, Ad) and 7v· 7-- Y is lIe nap deJined I¸ s¸nnelvic Junclions
|sonelines caIIed lIe Viela nap).
ExanpIe 3. TIe n-vaviaIIe vevsions oJ ExanpIes 1 and 2. Heve Y is sone sel oJ
poI¸noniaI naps C' -* C'. TIe conslvuclions oJ lIose exanpIes can Ie exlended lo
lIis case.
We sa¸ lIal an aIgovilIn soIving a pvoIIen 7 c Y x a is a nacIine ovev BB |ov
ils inpul-oulpul nap |M) IaIling al eacI ¸
E Y sucI lIal |¸, IM|¸)) E 7 Jov aII ¸E Y.
Ov ve nigIl considev inslead

Y-` 7, lIe gvapI nap oJ
OM
vIicI nusl lIen salisJ¸
7vx is lIe idenlil¸.
Hovevev, nosl pvoIIens in nunevicaI anaI¸sis cannol Ie soIved exaclI¸ even
using exacl avilInelic. TIus ve sa¸ lIal an aIgovilIn soIving 7 c Y x a appvoxi-
naleI¸ is a nacIine M IaIling al eacI inpul e ` 0, ¸ E Y and salisJ¸ing·
11 q|C, Y)- 7-|Y)jI
·c
vIeve lIe novn neasuves lIe dislance oJ ||e, ¸), IM|e, ¸))
lo lIe sel K`|¸).
Lel Fd|1) denole lIe suIsel oJ poI¸noniaIs in Y oJ ExanpIe 1 aIove vIicI
salisJ¸ lIe exlva conslvainls ad = 1 and I ai I _ 1,
i · d. TIe vovI in [SnaIe, 1981],
[SIuI and SnaIe, 1985, 1986], [SnaIe, 1985], [Kin, 1988a], ov [Benegav, 1987a]
appIies lo ¸ieId a |gIoIaI) Nevlon-MelIod aIgovilIn soIving lIe pvoIIen
7o
=
)|J ?)
E
Fd|I)
7 C I J|v)
=
01 appvoxinaleI¸
in lIe
pveceding
sense. A siniIav
slalenenl IoIds Jov ExanpIe 2.
Tovavd a conpIexil¸ poinl oJ viev, I nole lIal in lIe aIove vovI, lIe eslinale
T|c, J)' c|d Iog d÷
IIog
cj)q, J EFd|I)
is pvoved. Heve T|e, J) couId Ie laIen as lIe nunIev oJ nodes lvavevsed in lIe
conpulalion oJ an appvoxinale soIulion oJJ|v) = 0. EssenliaII¸ T|C, J) is lIe nunIev
oJ nodiJied Nevlon ilevales.
TIal eslinale suggesls lIal |*) couId Ie used as a cvilevion oJ lvaclaIiIil¸ in
nunevicaI anaI¸sis pvovided size|¸) is vepIaced I¸ size|¸) ÷ I Iog c
1. Indeed lIis
suggeslion is in lIe vigIl diveclion Iul nove is needed.
Lel us veluvn lo lIe oviginaI ExanpIe 1 vIeve lIe inpul space Y is lIe sel oJ aII
poI¸noniaIs J vIose Ieading coeJJicienl ad is nol zevo. BeJine a Junclion B = BJ
oJJ I¸
ad 1/|d-i))
B=
nax
-
i=O, * * * ,d- Iad
TIis nunIev B is used in a novnaIizalion lo pul J inlo Fd| 1). TIe pveceding eslinale
Iecones
T|c,J) ' c|dIogd÷ IIog
eI
÷ IogB)q.
TIe Junclion B is lIe pvolol¸pe oJ a veaI Junclion W oJ inpuls, I viII caII lIe
veigIl. TIe veigIl oJ ¸ is lo neasuve lIe ¨diJJicuIl¸ oJ lIe pvoIIen inslance ¸.¨
Sonelines W couId Ie laIen as a novn, sonelines as a condilion nunIev.
In vviling [SnaIe, 1981], I Ieavned oJ an appavenlI¸ genevaI pIenonenon lIal I
an suve is JaniIiav lo nunevicaI anaI¸sls. TIeve is a cevlain sel oJ iII-posed pvoIIens
Jov vIicI aIgovilIns JaiI, and as a pvoIIen inslance lends lo lIese iII-posed pvoIIens,
FOUNBATIONS OF NUMEBICAL ANALYSIS 217
lIe line oJ conpulalion lends lo oo. In lIis IigIl lIe veigIl is lo vesenIIe lIe
vecipvocaI oJ lIe dislance lo lIe sel oJ iII-posed pvoIIens.
A consequence oJ lIese vavious considevalions and olIev exanpIes is lIal lIe
conpulev science lvaclaIiIil¸ eslinale |*) exlends lo nunevicaI anaI¸sis in lIe Jovn
|*NA)
T|e,
¸)-c4e|e j)q
s4,¸)
=
s|¸)
÷ I Ioge I ÷Iog W|¸).
Heve lIe veigIl Junclion W, and even lIe spaces Y, 7, Iave lo Ie cIosen vilI
nucI lIougIl, laIing inlo accounl lIe pavlicuIav nunevicaI anaI¸sis selling, ils
Iinilalions, and jusl vIal a good aIgovilIn can Ie expecled lo acconpIisI.
We nigIl aIso considev ExanpIes 1, 2, and 3 Jvon lIe poinl oJ viev oJ quadvalic
convevgence al a soIulion. Il vouId Ie veasonaIIe in lIis case lo vepIace I Iog e I I¸
Iog I Iog c
I and noveoevev incovpovale inlo lIe veigIl oJ Ja quanlil¸ going lo so as J
appvoacIed a poI¸noniaI vilI nuIlipIe vools. TIe vovI oJ [Benegav, 1987a, I]
on
Nevlon's nelIod couId no douIl Ie inlevpveled Jvon lIis poinl oJ viev.
Il vouId Ie JvuilJuI lo see Iov vavious olIev vesuIls in nunevicaI anaI¸sis couId
Ie JovnaIized via |*NA). Fov exanpIe, using [KoslIan, 1988],
il seens lIal lIe povev
nelIod oJ Jinding lIe Iavgesl eigenvaIue couId Ie deaIl vilI. On lIe olIev Iand,
aIgovilIns Jov appvoxinaling soIulions oJ nonIineav pavliaI diJJevenliaI equalions give
gvealev diJJicuIl¸ since a gIoIaI anaI¸sis is vequived, nol jusl an as¸nplolic anaI¸sis.
ConpIexil¸ lIeov¸ oJ nunevicaI anaI¸sis is nol sinpI¸ an acadenic exevcise.
SpeciJicalions oJ Iinils on inpuls Jov vIicI an aIgovilIn is lvaclaIIe Iecone necessav¸.
III-posed pvoIIens nusl Ie isoIaled. TIe genevaI s¸slenizalion vouId cIaviJ¸ and in
lIe Iong vun IeIp inpvove lIe pvaclice oJ scienliJic conpulalion.
Condilion |*NA) needs lo Ie suppIenenled I¸ lIe condilion oJ nunevicaI
slaIiIil¸ in ovdev lIal an aIgovilIn quaIiJ¸ as lvaclaIIe. TIe deJinilion oJ nunevicaI
slaIiIil¸ in lIe nunevicaI anaI¸sis Iilevaluve needs JovnaIizalion. Fov exanpIe, [Conle
and de Boov, 1980, p. 391] vvile· ¨LooseI¸ speaIing, ve can sa¸ lIal a nelIod is
unslaIIe iJ evvovs inlvoduced inlo lIe caIcuIalions gvov al an exponenliaI vale as lIe
conpulalion pvoceeds.¨ Vev¸ lenlaliveI¸, I viII lv¸ lo capluve lIal neaning vilI
nacIines ovev BO.
In [BSS], lo eacI nacIine M ovev BB, lIeve is an associaled conpuling endo-
novpIisn HM· 7 x Y l vIeve 7 is lIe sel oJ nodes and 5' lIe slale space BI7.
TIen lIe sequence oJ conpulalions slavling vilI inpul ¸ is essenliaII¸·
|ni, xi)
=
HM|ni_1, xi-I), i= 1,2, *
no
=
inpul node, x0=¸
nT= an oulpul node, 7T=qM|¸),
T= T|¸).
Nov sa¸ lIal 6 `0 is an adnissiIIe voundoJJ evvov Jov |e, ¸) iJ Jov an¸ se-
quence |ni, xi) salisJ¸ing I|ni, xi)
-
HM|ni1, xi-,)I
· 6 and no = inpul, x0 = ¸, lIen
I7T
-
M|Y)I · e and nT is an oulpul node |ve couId vepIace 4M|Y) I¸ 9M|e, ¸)).
Lel IM|e, ¸) Ie lIe supvenun oJ lIe adnissiIIe voundoJJ evvovs.
Sa¸ lIal an aIgovilIn deJined I¸ a nacIine M ovev B4 vIicI soIves a pvoIIen
7 C Y x a appvoxinaleI¸ is nunevicaII¸ slaIIe iJ·
|**) 6M|e, Y)-CS|e j¸)q
Jov sone conslanls c, q depending onI¸ on M.
Nole lIe vesenIIance oJ |* *) lo |*NA).
218 STEVE SMALE
As an exanpIe, lIe vesuIls in [Kin, 1988I] inpI¸ lIe nunevicaI slaIiIil¸ oJ a
gIoIaI vevsion oJ Nevlon's nelIod Jov ExanpIe 1 aIove. SIe gives a good expIicil
eslinale Jov IM|c, Y).
TIe veigIl oJ a pvoIIen inslance seens cIose lo lIe condilion nunIev oJ lIe
pvoIIen. Yel I Iave Iesilaled lo use lIe sane vovd Jov IolI oIjecls.
TIe condilion nunIev Ias a veasonaII¸ cIeav-cul deJinilion in ils ovn vigIl and,
undev lIe vigIl I¸polIeses, lIeve nigIl Ie a vesuIl assevling lIal lIe condilion nunIev
couId Ie laIen as lIe veigIl. Bul il is necessav¸ lo Ie caveJuI as lIe JoIIoving exanpIe
sIovs.
Lel 7 c Y x a Ie a pvoIIen vIeve 7 is snoolI, and 7v|x) = ¸ vIeve 7v· 7- Y is
lIe associaled nap oJ lIe pvoIIen. TIe condilion nunIev oJ ¸ na¸ Ie deJined as
11 B-x|x)- 11 vIeve B-7v is lIe Jivsl devivalive |ov nalvix oJ pavliaI devivalives). In case
oJ nonuniqueness, ve couId laIe lIe naxinun ovev aII x sucI lIal 7v|x) = ¸. See
[WiIIinson, 1973] and [BenneI, 1987a, I] Jov a good IacIgvound.
Considev nov ExanpIe 1 Jvon lIis poinl oJ viev. FoI¸noniaIs vilI nuIlipIe
vools ave iII-condilioned |oo condilion nunIev). Yel lIe aIgovilIns nenlioned aIove
vovI veasonaII¸ Jasl on lIese iII-condilioned pvoIIens and so lIe covvesponding
veigIl does nol need lo veJIecl lIe condilion nunIev.
In an¸ case lIe condilion nunIev does pIa¸ an inpovlanl voIe in nan¸ aspecls
oJ lIe lopics discussed Ieve. Fov one lIing, lIe condilion nunIev oJ a pvoIIen puls
Iinils on lIe nunevicaI slaIiIil¸ oJ an¸ aIgovilIn soIving lIal pvoIIen |vougII¸
speaIing I¸ lIe cIain vuIe).
FvoIaIiIil¸ considevalions enlev inlo ouv viev oJ conpIexil¸ lIeov¸ in lvo va¸s.
In lIe Jivsl va¸, il is supposed lIal lIe space oJ pvoIIen inslances Ias a pvoIaIiIil¸
neasuve. A conpIexil¸ eslinale lIen nigIl Iave lIe Jovn
T|e,¸)?c|s|¸)÷ IIogeI
÷ jIog|1
u)_)¸
Jov a sel oJ inslances oJ neasuve ,u. TIis is lIe poinl oJ viev laIen in [SnaIe, 1981]
and [Benegav, 1987I].
TIis eslinale is oIlained I¸ conIining |*NA) vilI an eslinale oJ lIe l¸pe·
neasuve
I ¸ I W|I¸) ' A -
-
c -A)
TIe Iasl inequaIil¸ Ieads quicII¸ lo eIininalion lIeov¸ oJ aIgeIvaic geonelv¸ and
voIune anaI¸ses oJ diJJevenliaI geonelv¸. [Benegav, 1987I], [BenneI, 1987a], and
[Ocneanu, 1989] Iave nice conlviIulions on lIis. BenneI uses lIese ideas lo IeIp
undevsland vI¸ poovI¸ condilioned pvoIIens ave unIiIeI¸.
A pavlicuIav case oJ lIe Iallev is lIe queslion, ¨can Iavge Iineav s¸slens Ie soIved
in lIe pvesence oJ nacIine voundoJJ evvov?¨ On one Iand, lIe pvaclice oJ vecenl
decades Ias given one ansvev. On lIe olIev Iand, il vouId Ie salisJ¸ing lo Iave a
lIeovelicaI anaI¸sis. In Jacl von Neunann sludied lIis queslion.
TIe Iog oJ lIe condilion nunIev oJ lIe nalvix can Ie inlevpveled as a Ioss oJ
pvecision deJined I¸ lIe covvesponding Iineav s¸slen as discussed Jov exanpIe in
[SnaIe, 1985] and [BIun and SIuI 1986]. In n¸ papev I posed lIe queslion oJ
eslinaling lIe expecled Ioss oJ pvecision. Becenl papevs lIal deaI vilI lIis suIjecl
incIude [BenneI, 1987a], [KoslIan, 1988], [EdeInan, 1988], and [SzaveI, 1989].
TIe second va¸ lIal pvoIaIiIil¸ pIa¸s a voIe is vIen lIe aIgovilIn ilseIJ uses a
guess. Fov exanpIe, a sinpIe vevsion oJ Nevlon's nelIod cIooses an iniliaI poinl al
vandon. TIis appvoacI na¸ Ie Jound in [SIuI and SnaIe, 1985-1986], [SnaIe,
FOUNBATIONS OF NUMEBICAL ANALYSIS 219
1985], and [SnaIe, 1987]. Heve JovnaIizalion in levns oJ nacIines ovev B1 couId use
a nev Iind oJ node vIicI cIooses a nunIev al vandon.
BeJove cIosing Iel ne sIave sone oJ n¸ vesevvalions aIoul Jocusing on lIe nolion
oJ ¨lIe oplinaI aIgovilIn.¨ OJ couvse, in genevaI il is inpossiIIe lo Iave a nininun
oJ lvo Junclions sinuIlaneousI¸. Nov considev lIe eslinale |*NA) aIove vIicI couId
Ie given in nove delaiIed Jovn as
T|e, ¸) _CIS|Y)
q
÷ C2
IIogeI
q2 ÷ c3|10g W|¸))q3.
TIe conslanls c,, q, i = 1, 2, 3 ave Junclions oJ lIe nacIine ov aIgovilIn, and
conpIexil¸ eslinales vouId ain lo Iave lIese nunIevs as snaII as possiIIe. Bul I¸
lIe aIove venavI ve cannol expecl even lo nininize lvo oJ lIese nunIevs I¸ lIe
sane aIgovilIn. TIe silualion is aIvead¸ nel lo a Iessev exlenl in lvadilionaI conpulev
science vIeve pvoIIens posed I¸ Iavge cI ave ignoved in deJining oplinaI aIgovilIns.
On lIe olIev Iand, I IeIieve lIal conpIexil¸ Iovev Iounds oJ vavious l¸pes Iave
a Iig Juluve in nunevicaI anaI¸sis.
In lIe Iod¸ and in lIis Appendix, I Iave nol lvied lo Ia¸ dovn vIal lIe
Joundalions oJ nunevicaI anaI¸sis sIouId Ie. BalIev I Iave pvesenled sone ideas
vIicI nigIl inpvove lIe pvesenl silualion in lIis vegavd.
BEFEBENCES
TIe veJevences IeIov ave lIe ones lIal I Iave veJevved lo in lIe lexl. In no va¸ ave lIe¸ neanl lo Ie
conpIele. TIe¸ veJIecl slvongI¸ n¸ ovn Iiniled pevsonaI InovIedge and pevspeclives.
L. BLUM, Lecluves on a lIeov¸ oJ conpulalion and conpIexil¸ ovev lIe veaIs |ov an avIilvav¸ ving), in
pvepavalion, 1989.
L. BLUM ANB M. SHUB, EvaIualing valionaI Junclions· InJinile pvecision is Jinile cosl and lvaclaIIe on
avevage, SIAM J. Conpul., 15 |1986), pp. 384-398.
L. BLUM, M. SHUB, ANB S. SMALE [BSS], On a lIeov¸ oJ conpulalion and conpIexil¸ ovev lIe veaI nunIevs·
NF-conpIeleness, vecuvsiveJunclions, and univevsaI nacIines, BuII. Anev. MalI. Soc., 21 |1989),
pp. 1-46.
W. B. BBOWNAWELL, Bounds Jov lIe degvees in lIe NuIIsleIIensalz, Ann. oJ MalI., 126 |1987), pp. 577-
591.
J. CANNY, Sone aIgeIvaic and geonelvic conpulalions, in F-Space, Fvoceedings oJ lIe 20lI AnnuaI ACM
S¸nposiun on lIe TIeov¸ oJ Conpuling, 1988, pp. 460-467.
S. CONTE ANB C. BE BOOB, EIenenlav¸ NunevicaI AnaI¸sis, TIivd edilion, McOvav-HiII, Nev YovI,
1980.
S. COOK, TIe conpIexil¸ oJ lIeoven pvoving pvoceduves, in Fvoceedings oJ lIe TIivd AnnuaI ACM
S¸nposiun on lIe TIeov¸ oJ Conpuling, 1971, pp. 151-158.
J. BEMMEL, TIe geonelv¸ oJ iII-condilioning, J. ConpIexil¸, 3 |1987a), pp. 201-229.
On condilion nunIevs and lIe dislance lo lIe neavesl iII-posed pvoIIen, Nunev. MalI., 51 |1987I),
pp. 251-289.
T. BUBE, Quanlilalive anaI¸sis oJ pvoIIens in conpulev aIgeIva· OvoInev Iases and lIe NuIIsleIIensalz,
FI.B. lIesis, Couvanl Inslilule, Nev YovI Univevsil¸, Nev YovI, 1989.
A. EBELMAN, EigenvaIues and condilion nunIevs oJ vandon nalvices, SIAM J. Malvix AnaI. AppI., 9
|1988), pp. 543-560.
M. OABEY ANB B. JOHNSON, Conpulevs and InlvaclaIiIil¸, Fveenan, Nev YovI, 1979.
B. IEBABBI, QuanliJiev eIininalion in lIe Jivsl-ovdev lIeov¸ oJ aIgeIvaicaII¸ cIosed JieIds, Fvoceedings oJ lIe
21sl AnnuaI ACM S¸nposiun on lIe TIeov¸ oJ Conpuling, 1989, and FI.B. lIesis, CovneII
Univevsil¸, IlIaca, NY.
B. KABF, BeduciIiIil¸ anong conIinaloviaI pvoIIens, in ConpIexil¸ oJ Conpulev Conpulalions, B.
MiIIev and J. TIalcIev, eds., FIenun Fvess, Nev YovI, 1972, pp. 85-104.
M. KIM, TopoIogicaI conpIexil¸ oJ a vool Jinding aIgovilIn, BeIIcove, Movvislovn, NJ |1 988a), pvepvinl.
Evvov anaI¸sis and Iil conpIexil¸· FoI¸noniaI vool Jinding pvoIIen, Favl I, BeIIcove, Movvislovn,
NJ |1988I), pvepvinl.
220 STEVE SMALE
E. KOSTLAN, ConpIexil¸ lIeov¸ oJ nunevicaI Iineav aIgeIva, J. Conpul. AppI. MalI., 22 |1988), pp. 219-
230.
A. OCNEANU, On lIe voIune oJ luIuIav neigIIovIoods oJ aIgeIvaic vavielies, MalIenalics Bepl., Fenns¸I-
vania Slale Univevsil¸, MavcI 1989, pvepvinl.
J. BENEOAB, On lIe vovsl-case avilInelic conpIexil¸ oJ appvoxinaling zevos oJpoI¸noniaIs, J. ConpIexil¸,
3 |1987a), pp. 90-113.
On lIe eJJicienc¸ oJ Nevlon's nelIod in appvoxinaling aII zevos oJ a s¸slen oJ conpIex poI¸noniaIs,
MalI. Opev. Bes., 12 |1987I), pp. 121-148.
On lIe conpulalionaI conpIexil¸ and geonelv¸ oJ lIeJivsl-ovdev lIeov¸ oJ lIe veaIs, Favls I, II, and
III, CovneII Univevsil¸, IlIaca, NY, 1989, pvepvinl.
M. SHUB ANB S. SMALE, ConpulalionaI conpIexil¸· On lIe geonelv¸ oJ poI¸noniaIs and a lIeov¸ oJ cosl,
I, Ann. Sci. EcoIe Novn. Sup |4), 18 |1985), pp. 107-142.
ConpulalionaI conpIexil¸· On lIe geonelv¸ oJ poI¸noniaIs and a lIeov¸ oJ cosl, II, SIAM J.
Conpul., 15 |1986), pp. 145-161.
S. SMALE, TIe JundanenlaI lIeoven oJ aIgeIva and conpIexil¸ lIeov¸, BuII. Anev. MalI. Soc. |N.S.), 4
|1981), pp. 1-36.
On lIe eJJicienc¸ oJ aIgovilIns oJ anaI¸sis, BuII. Anev. MalI. Soc., 13 |1985), pp. 87-12 1.
AIgovilIns Jov soIving equalions, Fvoceedings oJ InlevnalionaI Congvess oJ MalIenalicians,
BevIeIe¸, CA, 1986, Anev. MalI. Soc., 1987.
TIe Nevlonian conlviIulion lo ouv undevslanding oJ lIe conpulev, in Nevlon's Bvean, Mavcia
Sla¸ev, ed., McOiII-Queens Univevsil¸ Fvess, Kingslon, Canada 1988.
S. SZABEK, Condilion nunIevs oJ vandon nalvices, IHES, Buves suv Yvelle, Fvance, Ma¸ 1989, pvepvinl.
A. TABSKI, A Becision MelIodJov EIenenlav¸ AIgeIva and Oeonelv¸, Univevsil¸ oJ CaIiJovnia Fvess, 1951.
E. TBIESCH, A nole on a lIeoven oJ BIun, SIuI, and SnaIe, BWTH, AacIen F.B.O., 1989.
J. VON NEUMANN, CoIIecled WovIs, VoI. V, A. TauI, ed., MacMiIIan, Nev YovI, 1963.
J. WILKINSON, Bounding Evvovs in AIgeIvaic Fvocesses, Fvenlice-HaII, EngIevood CIiJJs, NJ, 1973.

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->