MAn. AB is a regisleredl rademarkof 宵 leMal hWorks, Inc.

Thi s book is p巾 ledonncid伽 paper . @
Copyri ghl @2∞2 by J ohnWi l ey & Sons, NewYork. AII righls 陀 賠 刊 ed.
Publ i shedsi mul taneousl yinCnnadn.
No pnrt of Ihis publiclllionmay be reprodu回 d,storedina問 triev叫 systemor 1悶 nsmi ttedinnny formor
byanym回 ns,el ecl roni c, mechani caJ , phol ocopyi ng,問 cordi ng・scanni ngor otherwi se, except as
permi ttcdunder Secti ons 107or 108ofthe 1976Uni ted, Stalcs Copyri ghl Acr, wi rhoul either theprior
wri ttcnp苦 rmi ssi onof IhePubl i sher, or outhori 7.3lionthroughpaymenl of theappropri ul eper-copyfcc 10
IheCopyri ghl Cl earanceCenl er, 222Ro揖 W∞dDri ve, Dnnver百 , MA01923, (978) 750・8400,fnx(978)
750・4744. Requ田 IS10IhePublisher for permi ssi onshoul dbe nddressed10thePeml i ssi ons Departmenl ,
J ohnWi l cy & Sons, Inc. , 605百 irdAvenue, NewYork, NY 10158-∞12, (212) 850- ω11,加 (212)
850- 6008, E-Mai l : PERMREQ@WI LEY. COM.
η 首spubl i cal i onis desi gned10provi deaccu問 lenndauthori l al i vei nfomtal i onin冊 E副甘 10山 esubj ecl
mattercove問 d. It is 80l dwi ththeundersl undi ngIhal thepubl i sher is nol engagedinrend自吋 ng
professi onal services. If professi onnl ndvi ceor othcr expert assi sl anceis 民 qui red,出 eservi ces of a
compel ent professi onal pe目 。 nashoul dbesoughl.
Ubrary ofCongress Catal og; lI g; n P"bl;catiotl Data:
Venkol urom且 n,P.

Appl i edopl i mi Zl l lionwi th Programmi ngI P. Venkal ar百 mlln.
p. com.
“A Wi l ey-I ntersci encc publ i cnl i on."
I SBN0-471剖 958・.5(c1oth: alk. pnper)
1. Mnl hemal i caJ opti mi zati on--Dol l l proc巴 出 ing・2. MATLAB. J. Title.
QA402. 5. V42 2001
519. 3- dc21
Pri nl edinIheUni l edSlales of Ameri cn.
拍 泊 当 , “ 、 “ 由 幽 以 罰 ,   一 一   一 一 ←
T 0 coexi stence of peace and har mony bet ween the di fferent
mel odi es of peopl e ever ywher e
一 一一一一 6 巾 一 ← 一 十 十

1 I ntroductl on
1.1 Opti mi zati onFundamental s I 2
1.1.1 El ements of Probl emFormul ati on I 4
1.1. 2 Mathemati cal Model i ng I 10
1.1. 3 Nature of Sol uti on I 16
1.1.4 Characteri sti cs of theSearchProcedure I 20
1.2 I ntroducti ontoMA11..As I 25
1. 2.1 Why MATLAB? I 25
1. 2.2 MATLAB Instal l ati onI ssues I 26
1. 2.3 Usi ng MATLAB theFirst Ti me I 27
1.2.4 Usi ng theEdi tor / 33
1.2.5 Creati nga Code Sni ppet / 37
1. 2.6 Creati nga ProgramI 40
Probl ems I 44
2 Graphl cal Opt l mi zat l on 45
2.1 針 。 bl emDefi ni ti on I 45
2. 1.1 Exampl e 2.1 I 46
2. 1. 2 Format for l heGraphi cal Di spl ay I 47
2.2 Graphi cal Sol uti on I 48
2.2.1 MATLAB Hi gh- Level Graphi cs Functi ons I 48
2.2.2 Exampl e 2. I - Graphi cal Sol uti on I 50
2.2.3 Di spl ayi ngtheGraphi cs I 53
2.2.4 Customi zi ng theFi gure I 54
2.3 Addi ti onal Exampl es I 56
2.3.1 Exampl e 2.2 I 56
2.3.2 Exampl e 2.3 I 64
2.3.3 Exampl e 2.4 I 73
2.4 Addi ti onal MA11..AB Graphi cs I 79
2.4.1 Handl e Graphi cs I 80
2.4.2 Graphi cal User I nterface I 81
2.4.3 GUI Code I 84
References I 91
Probl ems I 92
3 Ll near Pr ogr amml ng
3.1 Probl emDefi ni ti on I 94
3.1.1 StandardFormat / 94
3.1.2 Model i ng I ssues / 98
3.2 Graphi cal Sol uti on / 107
3.2.1 Exampl e 3.1 / 110
3.2.2 Characteri sti cs of theSol uti on / I I 1
3.2.3 Di fferent Sol uti onTypes I 114
3.3 Numeri cal Sol uti on- the Si mpl ex Met hod I 115
3.3.1 Features of theSi mpl ex Met hod / 115
3.3.2 Appl i cati onof Si mpl ex Met hod / 117
3.3.3 Sol uti onUsi ng MA11..AB / 120
3.3.4 Sol uti onUsi ng MATLAB' S Opti mi zati onTool box I 123
3.4 Addi ti onal Exampl es / 124
3.4.1 Exampl e 3. 2-Transportati on Probl em/ 124
3.4.2 Exampl e 3. 3- EqualityConstrai nts and Unres凶 cted
Vari abl es / 130
3.4.3 Exampl e 3.4- A Four-Vari abl e Probl em/ 134
3.5 Addi ti onal Topi cs inLi near Programmi ng / 138
3.5.1 Pri mal and Dual Probl em/ 138
3.5.2 Sensi ti vi tyAnal ysi s / 148
Refer芭 nces / 151
Probl ems / 152
4 Nonl i near Pr ogr ammi ng
4.1 Probl emDefi ni ti on I 155
4.1.1 Probl emFormul at i on- Exampl e 4.1 / 155
4.1.2 Di scussi onof Cons釘 aints / 157
4.2 Mathemati cal Concepts / 159
4.2.1 Symbol i c Computati on Usi ng MATLAB / 159
4.2.2   a ~ i c Mathemati cal Concepts I 162
4.2.3 Tayl or' s Theorem/Seri es / 169
4.3 Graphi ca1Sol uti ons / 171
4.3.1 Unconstrai nedProbl em/ 171
4.3.2 Equal i ty Constrai nedProbl em/ 172
4.3.3 I nequal i tyConstrai nedProbl em/ 173
4.3.4 Equal i ty and I nequal i tyConstrai nts / 174
4.4 Anal yti ca1Condi ti ons / 175
4.4.1 Unconstrai ned Probl em/ 176
4.4.2 EqualityCons回 i nedProbl emI 179
4.4.3 I nequal i tyConstrai nedOpti mi zati on I 186
4.4.4 General Opti mi zati onProbl em/ 191
4.5 Ex創 npl es / 194
4.5.1 Exampl e4. 2/ 194
4.5.2 Exampl e 4.3 / 196
References / 200
Probl ems / 201
5 Numeri cal Technl ques-The One- Dl mensl onal Pr obl em
5.1 Probl emDefi ni ti on I 204
5. 1.1 Constrai nedOne- Di mensi onal Probl em/ 204
5.2 Sol uti ontotheProbl em/ 205
5.2.1 Graphi cal Sol uti on / 205
5.2.2 Newton- Raphson Techni que / 206
5.2.3 Bi secti onTechni que I 209
5.2.4 Pol ynomi al Approxi mati on / 211
5.2.5 Gol denSecti onMet hod I 214
5.3 I mportance of theOne- Di mensi onal Probl em/ 217
5.4 Addi ti onal Exampl es I 219
5.4.1 臥 ampl e5. 2-Il1us回 tionof Geneml Gol denSecti onMethod / 219
5.4.2 Exampl e 5.3ー T wo- Poi nt Boundary Val ue Probl emI 220
5.4.3 Exampl e 5. 4- Root Fi ndi ngwi thGol den Secti on I 223
References I 225
Probl ems I 225
6 Numerl cal Technl ques f or Unconst r al ned Optl ml zatl on
6.1 Probl emDefi ni ti on I 227
6. 1.1 Exampl e 6.1 I 228
6.1.2 Necessary and Suffi ci ent Condi ti ons I 228
6.1.3 El ements of a Numeri ca1Techni que I 229
6.2 Numeri cal Techni ques- Nongradi ent Methods I 230
6.2.1 RandomWa1k I 230
6.2.2 PattemSearch / 234
6.2.3 PoweJ1' s Met hod I 238
6.3 Numeri cal T,配 hni ques- Gradi ent- BasedMethods I 241
'6.3.1 Steepest Descent Met hod I 241
6.3.2 Conj ugate Gradi ent (F1etcher-Reeves) Met hod I 244
6.3.3 Davi don-FJ etcher-PowelI Met hod I 246
6.3.4 Broydon- Fl etcher-Gol dfartト ShannoMet hod I 249
6.4 Numeri cal Techni ques- Second Order I 251
6.5 Addi ti onal Exampl es I 253
6.5.1 Exampl e 6. 2- Rosenbrock Probl em/ 253
6.5.2 Exampl e 6. 3-Three-Di mensi onal Aow n飼 r aRo凶 ngDi sk I 255
6.5.3 ExampJ e 6.4- Fi tti ng Bezi er Parametri c Curvcs I 258
References I 262
Probl ems I 263
7 Numerl cal Techni ques f or Const ral ned Optl ml zatl on 265
7.1 Probl emDefi ni ti on I 266
7. 1.1 Probl emFormul ati on- Exampl e 7.1 I 266
7. 1. 2 Necessary Condi ti ons / 267
7.1.3 El ements of a Numeri cal Techni que I 269
7.2 Indi rect Methods for Constrai nedOpti mi zati on I 270
7.2.1 Exteri or Penal ty Functi on( EPF) Me出 od I 271
7.2.2 Augment ed Lagrange Mul ti pl i er ( ALM) Met hod I 276
7.3 Di rect Methods for Cons住 ai nedOpti mi zati on I 281
7.3.1 Sequenti a1Li near Programmi ng (SLP) I 284
7.3.2 Sequenti al Quadrati c Programmi ng (SQP) I 289
7.3.3 Genera1i zedReduced Gradi ent ( GRG) Met hod I 297
7.3.4 S句 uenti al Gradi ent Restorati onAJ gori thm( SGRA) I 302
7.4 Addi ti ona1ExampJ es I 307
7.4.1 ExampJ e 7. 2- FJ agpoJ e Probl emI 307
7.4.2 ExampJ e 7. 3- BeamDesi gn I 310
7.4.3 ExampJ e 7.4ー Opt i ma1Control I 313
References I 316
Probl ems I 316
8 Dl screte Optl ml zati on
8.1 Concepts inDi screteProgrammi ng / 320
8. 1.1 Probl emRel axati on I 321
8. 1. 2 Di screteOpti mal Sol uti on I 322
8.2 Di screteOpti mi zati onTechni ques I 324
8.2.1 Exhausti ve Enumerati on I 326
8.2.2 Branch and Bound I 329
8.2.3 Dynami c Programmi ng / 336
8.3 Addi ti onal Exampl es I 341
8.3.1 Exnmpl e 8.4--1BeumDesi gn I 341
8.3.2 Zero- One I nteger Programmi ng I 343
References I 348
Probl ems I 348
9 Gl obal Optl ml zati on
9.1 Probl emDefi ni ti on I 351
9. 1.1 Gl obal Mi ni mum/ 351
9.1.2 Nature of theSol uti on I 354
9. 1. 3 El ements of a Numeri ca1Techni que I 356
9.2 Numeri cal T,白 hni qu巴sand Addi ti onal Exampl es I 357
9.2.1 Si mul atedAnneal i ng (SA) I 358
9.2.2 Geneti c Al gori thm( GA) I 366
References I 377
Probl ems I 378
10 Optl mi zati on Tool box f r omMATl AB
10.1 The Opti mi zati onTool box I 380
10. 1. 1 Programs I 380
10. 1. 2 Usi ng Programs I 382
10.1.3 Setti ngOpti mi zati onParameters I 384
10.2 Exampl es I 385
10.2.1 Li near Programmi ng I 385
10.2.2 Quadrati c Programmi ng I 386
10.2.3 Unconstrai ned Opti mi zati on I 388
10.2.4 Constrai nedOpti mi zati on I 389
Reference I 391
I ndex
百 les u ~ j e   t of opti mi zati onis recei vi ngseri ous attenti on仕 o mengi neers, sci enti sts,
managers, and most everybody else. Thi s is dri venby compe“ti on, qual i tyassurance,
cost of producti on, and fi nal l y, the success of the busi ness ente中 市 e. l gnori ng 出 e
practi ceof opti mi zati onis not an opti onduri ngcurrent ti mes.
Opti mi zati on is practi ced through softwar芭 programs and requi res si gni fi cant
computer resources. The techni ques of opti mi zati onhave not changed si gni fi cantl yin
陀 cent years, but tlle areas of appl i cati ons have mushr oomed at a si gni fi cant rate.
Successful l y embeddi ng theuse of opti mi zati on in professi onal practi cerequi res at
least threeP陀 巴 qui si tes. τbeyi ncl udemathemati cal model i ng of thedesi gnprobl em,
knowl edge of computer programmi ng, and knowl edge of opti mi zati on techni ques.
Many speci al -purpose opti mi zati on software packages 白 紙 rel ax 由巳 requi red
knowl edge of programmi ng are avai l abl e today. To use t hemeffi ci entl y, the
remai ni ng t wo areas still have tobe addressed.
百 l ere are several excel l ent books on the subj ect of opti mi zati on, a f ewof them
rel easedrecentl y. Most of t hemcov巴 rthesubj ect indepth. whi ch is necessary because
th巴 mathemati cal model s arenonl i near and requi respeci al techni qu巴sthat areusual l y
notpaは of any corecurri cul um. Al l of 山 ebooks assume白 紙 出 ereader isfami l i ar wi l h
a programmi ng l anguage,町 adi ti onal l yFORTRAN, and recentl y, C. Another assumpl i on
h芭 quentl y made is 白 紙 the opti mi zati on 包 chni ques wi1l be i mpl emented in a
mai nf rame computi ng envi ronmenl. Such a combi nati on is ex官 emel ydi ffi cu1t for the
sel f-I earner even wi thenormous moti vati on.
An expl osi onof i nexpensi ve desktopcomputi ng resources ai ds engi neeri ngdesi gn
practi 印 today. Paral l el i ngthis devel opment is 出 eavai l abi l i ty of extensi ve陀 sou陀 凶
on theI nternet , bothfor l earni ngand depl oyment. This isj oi nedby theavai l abi l i tyof
sof抑 制 官 systems that provi de叩 opportuni tyfor handl i ngmathemati cs, graphi cs. and
programmi ng ina consi stent manner. These softwaresystems aresi gni fi cant1y easy to
master compar吋 to the hi gher-l evel programmi ng l anguages of thep陀 vi ous ye町 S.
百 lis book pri mari l y seeks to harness this 釘 i angul ati on of s巴 rvi ces to provi de a
practi cal approach to the study of desi gn opti mi zati on. 百 le book uses MATI . AB@to
i11ustrateand i mpl ement thevari ous techni ques of opti mi zati on. MATLAB is a product
f romMat hWorks. Inc.
Thi s book attempts 10accompl i sh t wo i mportant obj 田 tives. The fir
of symbol i c. numeri cal . and graphi cal f l回 tures of MATLAB・lt i ntegrates 山 is powerful
combi nati on duri ng 恥 凶 nsl ati on of many al gori thms i nto app1i ed numeri cal
techni ques for desi gn opti mi zati on. There is a constant enhancement of the
programmi ng skills throughout th巴 book. The second 0吋ecti veis th巴 pri mary taskof
侃 book- t ocommuni cate and demonstrate vari ous lI umeri cal techni ques that are
currenl か usedintltearea ol opti mal des刷 .
AH of thc numeri cal t echni quesMEsupport ed by MMLAB code avai l abl e ぉ
computer files. These files are avai l abl e on a compani on web site
www. wi l ev. coml venkat. l ti s necessary to visit theweb sitetodownl oad thc files to
fol l o w a l l of t h e E X tl I n p i es -Th e b oo k w i l l b e e 蹴 n削tia叫l l yi n 脚 c ∞ompl e巴t旬.ewi ωtho 削 削 0肌凶u 凶山』此 tthcsc files.
At the web site you will al so fi nd uscful l i nks 10other resources, a web course on
MATLAB programmi ng,and updated i nformati on about Appl i ed Opti nti muon wi th
MA Tl.AB @ Programmi ng.
The deci si onto l ocate thefiles on a web sitepmvi dEs an avenue toshadowthe
dynami c changes inMATLAs sof1wareitsclf, whi ch is bci ng transformedin a maj or
way al most every ycar.l t provi des a mechani sm10correct bugs i日 出 ecode ina ti mel y
manner. It estab¥i shes an opportuni ty toi mprove自己 codethroughreader suggcsti ons
and provi des a way for 出 eauthor 10keep thc rcader engaged wi thnewdcvcl opments.
It doωcut down thesi zc of thebook enormousl y by l ocati ngall of thepag回 of code
outsi dethebook.
An unusual featureof thebook is 由 ci ncl usi onof di screteopti mi zati onand gl obal
opti mi zati on. Tradi l i onal conti nuous desi gn opti mi zati on techni ques do take up a
si gni fi cant porti onof thebook. Conti nuous probl ems general l y al l owtheal gori thms
tobe mathemati cal l y devel oped ina convi nci ngmanncr, and由 民 ei deas coul d be向
core of ori gi nal a1gori thms dcvc10pcd by the readers to address their own speci al
necds. Every chaptcr i ncl udes addi ti onal nontri vi al cxamp1es 由 at oftcn prescnt a
novel use or extcnsi onof thc opti mi zati ontechni ques inthechapter. The mathcmati cal
al gori thms and cxampl es ar芭 accompani ed by MA'礼 AB code avai l abl e at the
compani on web sue- Thenumeri cal techni ques usual l y i ncl ude strong graphi cal
support toil1ustrate出 copcrati onof thc techni ques.
Di screteandgl obal opti mi zati ontechni ques arebasedon verydi fferental gori thms,
and 回 .ch of t hcmcan easi l y j usti fy ful l -fl edged i ndependent courscs de叩 tcd
excl usi vel y 10thcir S1udy. In a comprchensi vc book such
cl assroomsetti ngwi thou1computati onal cxperi ence is al so feasib1e. In由 is casethe
a1gori thms can be presentedand thenurneri cal resul ts illustrated. The seni or/gradua1e
s1udents in vari ous di sci p¥ i nes. especi al l y engi nceri ng, are the targc1 audi ence.
Opti mi zati on techni qucs are an i mportanl 1001 to sol ve dcsi gn probl ems in all
professi onal areas of study. Many i l l ustrati ons are 仕 o mthe area of mechani cal
engi neeri ngrefl ecti ng出巴 experi enceof 白 eau白 or.
Independent l earnel' s, parti cuI arl yprofessi onal s who need tounderstandthesubj ect
of opti mi zati on, shoul dal so fi ndthebook very useful . A reader who has access to
MATLAB software c剖 us巴 the book to its best advantage. The book is l argel y
sel f-contai ned and devel ops all ncccssary mathemati cal concepts when ncedcd.
Abstract mathemati cal i dcas of opti mi 国 ti on創 芭 i ntroduccdgraphi cal l y and through
iIl ustrati veexampl es inthc book. Inmany i nstances, theoperati onof thc numcri cal
techni que is 祖 国 atcdto showhow thedesi gnis changi ng wi thiterations. Thi s has
been possi bl ebecause MATLAB graphi cs areri chinfcatures and si mpl e toi nco叩 oratc.
maki ng Ul cmcffccti ve for convcyi ng 出 e i dcas. It is for this reason 出 e reader is
cxposed to graphi cal programmi ng earl y on, incI udi ng a chapter on graphi cal
opti mi zati on. To bal ance thc devel opment of programmi ng skills and th巴 presentati on
of opti mi zati on concepts, programmi ng i ssues are i ncremental l y expl ored and
i mpl emented. I nvestment of ti me in d巴 vel opi ng programmi ng skills is 出 e most
effecti ve way to i mbi be them. Thc author is nOl aware of shortcuts. Studcnts arc
expected to programi ndependentl y. make mi stakes, debug errors. and i ncorporate
i mprovements as p紅 t of their l earni ngexperi ence del i veredthrough出 is book.
The book proceeds at a bri sk pace toprovi de experi encc inMATLA8 programmi ng
and communi cate i deas inopti mi zati onina reasonabl e number of pages. This coul d
not be possi bl ewi thout taki ngadvantage of Iheaccompanyi ng web site10l ocatemuch
of the code for al gori thms and exampl es. Merel y runni ng the code will n01
si gni fi cantl y e油 田 cethereader' s programmi ng skills. It is i mportant for thestudent
tounderstandthecode also. To箇 sist in白 is process, thecode carri es Ii beral comment s
oftencorrespondi ngtotheal gori thmthat is bei ngi mpl emcnted. The au出 or expects
出 ereader tohave read出 ecomments. The accompanyi ng code al l ows thei nstructor
toassi gnboth programmi ng assi gnments as wel l as so
approaches, the i nstructor can choose to i ncl ude onl y a subset of conti nuous
opti mi zati on al gori thms to al l owthe i ncl usi on of 出 ese popul ar techni ques. I f 出 e
student has access totheOpti mi zati onTool box f romMATl AB,白 enit canbe i ntegrated
i nto 出 e course for handl i ng the final desi gn proj ect . The last chapter provi des an
overvi ewon theuse of thetool box.
Thi s book startedout usi ngMATI . AB Versi on5. About hal f way throughthebook
血 e transi ti on to MATl AB Versi on 5.3 was compl ete. At the compl eti on of the
m叩 uscri pt MATLAB Versi on6 wωshi ppi ng for some ti me. As of this tirne, Versi on
6.1is avai l abl e. Most i nsti tuti onal MATl AB l i censes areusual l ybasedon subscri pti on
so白 紙 they will have出巴 latest versi onof 白 esoftware. Ina sense, and出 is is trueof
all books that depend on parti cul ar softwaresystems, thebook rnayappear out of date
wi threspect to白 esoftware. Thi s is not real l yso. Whi l e theusage of MATLAB functi ons
rnay change between versi ons, thereshoul dbe sui tabl ewami ngs and i nstructi onabout
出 e di fferent usage・An i mportant i nsurance is built i nto the book, parti cul arl y to
handl e such changes if they arise. Spri nkl edthroughout thebook is thehabi t of 由 e
usi ng出 eonl i nehel p. Inmost i nstances 出 is was pri mari l ydone toexpose thereader
toal ternati veuseof thesame command. Another si gni fi cant change themanuscri pt
had toendure was 白 eloss of access toa UNI X versi onbecause of th巴 uni versi ty' s
change to di stri butedcomputi ng on PC clusters. The author' s experi ence wi 白 血 e
earl y chapters sugg巴s18 that this shoul d not mat1er. Except for external files and
resources,也巴 useof MATl AB as illustratedinthebook is i ndi fferent tothe vari ous
operati ngsy自 由 ( Wi ndows,Mac, UNI X, or Li nux).
The topi cs chosen for i ncl usi onand thesi mpl i ci tyof presentati onof 出 etopi cs are
di rectl y rel ated to the experi ence of teachi ng the course on opti mi zati on at the
seni or/graduatel evel in出 e D巴partment of Mechani cal Engi neeri ngfor over tenyears.
Experi ence proved白 at comprehensi on i rnprovedwi thsi mpl e iII ustrati ons, however
compl i catedthemathemati cal i deas. On theother hand, strai ghぜorwardmathernati cal
expressi ons elicitedno i nterest or understandi ng. The emphasi s on appl i cati on was
i mportant tokeep thestudents' attenti on. Inthis connecti ontheauthor woul d liketo
出創1kall hi s studen18for pushi ng hi rnfor si mpl e expl anati ons, for provi di ngposi ti ve
i nteracti on, and for thei r contri buti onof ori gi nal and useful i deas inthedi scuss


The book was made possi bl ethroughsupport f romJ ohn Wi l ey叩 dSons,Inc. , 叩 d
Mat hWor ks, Inc. Si ncere thanks are owed to Bob Argenti eri , seni or edi tor at J obn
Wi l ey for accepti ng theproposal ,and who all al ongdi spl ayeda lot of pati encei n
getti ngthe book movi ng f orward. Same is due toBob Hi l bed-associ ate managi ng
edi tor at J ohn Wi l ey, for his i mp陀 ssi ve work at c1eani ngup the manuscri pt . Bri an
Snapp, New Medi a edi tor at J ohn Wi l ey, created 出巴 compani on web site
(www. wi l ey. coml venkat) and w辿 al so be rnai ntai ni ng it. Naomi Femandes 企o m
Mat hWor ks, Inc. , sawtoit 白 at 1 had thelatest versi onof MATLAB as soon as it was
avai l abl e. My regardfor Dr. Angel o Mi el e, Professor Emeri tus, at Ri ceUni versi ty is
mor e thancan be expressedintheseIi nes. It was he who i ntroducedme to1h巴 subj ecl
of opti mi zati on and demonstrated the effecti veness of si mpl e presenuti on-I will
al ways regardhi mas a great teacher. Of course, my fami l y deserves speci al rnenti on
for putti ng up wi th all the “Not now,"“ Later,"“ How about tornorrow?" d町 i n2
debuggi ng白 ecode. Sp配 ial 出 回 ks are toA帥 ana阻 d Vi nayak, rny offspri ng, t i町
their pati ence, understandi ng, and encouragement. The au白 or apol ogi zes for any
shortcorni ngs on the presentati on and wel comes comment s, cri ti ci sms, and
suggesti ons for i mprovement at all ti mes.
Rochesteれ NewYork
1・・-・ a回 . . 圃・・・・・・・

Opti mi zati on has become a necessary part of desi gn acti vi ty in all maj or
di sci pl i nes. These di sci pl i nes arenot restri ctedtoengi neeri ng. The moti vati onto
produce economi cal l y rel evant products or servi ces wi thembedded qual i ty is the
pri nci pal reason for 由 is i ncl usi on. I mproved producti onand desi gntool s, wi 出 a
synergi sti c thrust through i nexpensi ve computati onal resources, have ai ded the
consi derati on of opti mi zati on methods in new devel opments, parti cul arl y
engi neeri ng products. Even in the absencc of a tangi bl e product, opti mi zati on
i deas provi de the ability to defi ne and expl ore probl ems whi l e focusi ng on
sol uti ons 出 at subscri be tosome measure of useful ness. General l y, theuse of the
wor d opti mi zati on i mpl i 巴s thc best resul t under th巴 ci rcumstances. Thi s i ncl udes
theparti cul ar set of constrai nts on thedevel opment rcsources, current knowl edge,
market condi ti ons, and so on. Every one of us has probabl y used thc termat some
ti me to descri be 出 e pri mary qual i ty of our wor k or endeavor. It is probabl y 山 e
mosl used or abused l ermin adverti si ng and presentati ons. Neverthcl ess,白 c
abi l i ty tomake thebest choi ce is a perpel ual desi reamong us al l.
Opti mi zati on is frequentl y associ ated wi th desi gn, be it a product , servi ce, or
strategy. Aerospace desi gnwas among出巴 earliest di sci pl i nes toembrace opti mi zati on
ina si gni fi cant way dri venby a natura1need tol ower thetremendous cost associ ated
wi thcarryi ngunnecess町 ywei ght inaerospace vehi cl es. Mi ni mummass structures arl巴
山 norm. Opti mi zati onf orms part of thepsyche of every aerospacedesi gner. Savi ng
on fuel throughtr司j ectorydesi gnwas another probl emthat suggesteditself. Very soon
曲巴 enti 問 engi neeri ngcommuni t y coul d recogni ze由 cneed todefi nesol uti ons based
on meri t. Recogni zi ng thedesi refor opti mi zati onand actua1lyi mpl ementi ng we陀 t wo
di fferent issues.
Unti l recentl y. for much of the ti me. opti mi zati onwas usual l y attemptedonl y in
those si tuati ons where there were si gni fi cant penal ti es for generi c desi gns. The
appl i cati onof opti mi 却 ti ondemanded l argecomputati onal resources. Inthe nascent
years of di gi tal computati on these were avai l abl eonl y to l arge nati onal l aboratori es
and research programs. These resources were necessary to handl e th巴 nonl i near
probl ems 出 at 創 芭 associ ated wi th engi neeri ng opti mi 回 .tion. As a resul t of these
constrai nts most of the everyday products were desi gned wi thout regard to
opti mi zati on. This i ncl udes everythi ngyou s回 ぽ oundyou or use inyour dai l ylife. It
is i nconcei vabl e 出 at the newgenerati on of repl acement products. likc the car. the
house. the desk. or 出 epenci l . redesi gnedand manuf actured today arc not desi gned
opti mal l y inone sense or anothcr.
Today. you woul d def mi t巴Iy expl ore procedures toopti mi ze your i nves回 l ents by
tai l ori ng your portfol i o. You woul d opti mi ze your busi ness travel ti me by
appropri a旬 Iychoosi ng your dcsti nati ons. You can opti mi ze your commut i ng ti m巴 by
choosi ng your ti me and route. You can opti mi ze your necessary expendi ture for Ii vi ng
by choosi ng your day and storefor shoppi ng. You can opti mi zc theuseful ti me you
connectto由巳 I ntemet by detcrmi ni ng your ti me of connecti on. You can buy software
白 紙 wi l l opti mi ze your connccti on tothc l ntemct. You can buy books or readarticles
白 紙 tcll you how to perf ormthese vari ous opti mi zati ons. The above acti vi ti cs
pri mari l y rel atc to :;crvIces or strategy. It is now apparcnt 白 at every acti vi ty, except
a巴 S出 etic,provi des the scope for opti mi zati on. 百 lis j usti fi es l ooki ng at 白 estudy of
ooti mi z副 onas a tool 白 紙 canbe appl i ed10a vari ety of di sci pl i nes. If so, themyri a 叫
of o 叩矧刑 p 戸凶ti m 凶 iz 捌a悦叫t “io 叩 n吋 叩 p仰r叩ob 愉le 巴m 附ns 凶s 合伽伽伽 o ω叩 11 削I
Thi siおs 曲 eempha おsiぬsof t由hebook.
The partnershi p between d巴 si gn and opti mi zati on acti vi ty is often f ound in
engi neeri ng. Thi s book recogni zes that connecti on and many of thc probl ems used for
i l l ustrati ons and practi ce are f romengi n田 ri ng,pri mariI y mechani cal , civil , and
aerospace desi gn. Ncverthel ess the study of opti mi zati on. parti cul arl y appl i ed
opti mi zati on. i s not m excl usi ve pmpert y of my sptci nc di sci purl e.I t i nvol ves the
dECOVEr yanddesi gnof 叫 uti ons through appropri atetechni ques 鰯 oci atedwi th the
formul ati on of the probl emin a sp田 ific manner. Thi s can be done for exampl e in
economi cs, chemi stry, and busi ness man
Opti mi zati on can be appl i ed to all di sci pl i ncs. Qual i tati vel y, this asserti on i mpl i es
mul ti pl e deci si on choi ces; i mpl i ci tl y recogni zi ng the n巴cessi ty of choosi ng among
a1temati ves. Thisbooks deal s wi th opti rni zati onina quanti tati veway. Thi s means that
an out come of appl yi ng opti mi zati on to 曲 e probl em, desi gn. or servi ce must yi el d
number富 山 at wi l l defi nethesol uti on. or inother words, numbers or val ues 白紙 wi l l
characteri ze theparti cul ar desi gnor servi ce. Quanti tati vedcscri pti onof 出 巴 回 l uti on
requi res a quanti tati ve descri pti on of theprobl emitself. This descri pti onis cal l eda
; :   ~
mathemati cal model . τbedωi 伊 . i ωcharactcri zati on. 四 d its ci rcumstances must be
expressedmathemati cal l y. Consi der thedesi gnacti vi ty inthefol l owi ngcases:
• New consumcr researchi ndi catωthat peopl e like10dri nk about 0.5litcr of soda
pop al a ti me duri ngthesummer months. The fabri cati onCOSI of theredesi gned
soda can is proporti onal to出巴 surfacearea. and can be esti matcdat $1. 00 pcr
square meter of themateri aI used. A ci rcul ar cross secti onis themost pl ausi bl e
gi ven current tool i ng avai l abl e for manufacture. For aestheti c reasons,白 e
hei ght must be at l east twi ce thedi ametcr. Studi es i ndi cate出 at hol di ngcomf ort
requi res a di ameter between 6 and 9 cm.
• A canti l ever beamnceds to be dcsi gn巴d tocarry a poi nt l oadF at thecnd of a
beamof l engthL. The cross secti onof thebeamwiII be intheshupe of th巴 l巴 tter
I (路 島 町 edωas 却 I-b回 m). The beamshoul d meet prescri bedfai l urecriteria.
百 ereis al soa l i mi t on i ωdefI田 tion. A beamof mi ni mummass is requi r吋 to
be desi gned.
• MyPC Company has deci dcd to i nvcst $12 mi l l i on in acqui ri ng several n巴W
Componenl Pl acem巴 nl Machi nes to manuf acture di ffercnt ki nds of
mothcrboards for a ncwgcncrati on of personal compul ers. Three model s of
thcsc machi nes areunder consi derati on. Total numbcr of operators avai l abl eis
100 becausc of the l ocal l abor mar ket . A fI oor spacc constrai nt needs 10 bc
sati sfi ed because of the di ffcrent di mcnsi ons of thcse machi nes. Addi ti onal
i nformati onrcl ati ngtoeach of themachi nes is gi ven inTabl e 1.1. The company
wi shes to determi ne how many of each ki nd is appropri ate 10 maxi mi ze the
number of boards manuf actured per day.
The above list rcpr凶 ents threeprobl ems 山 at wiII be used 10defi nc formal el ements
of an opti mi zati onprobl em. Each probl em陀 qui res i nformati onf romthespeci fi c arca
or di sci pl i ne to whi ch it refers. To rccogni ze or desi gntheseprobl ems assumes 出 at
出巴 desi gncr is conversant wi 白 山 eparti cul ar subj 田 t matter. 百 leprobl ems arekept
si mpl e tofocus on opti mi zati oni ssucs. Probl ems of such vari el yarei mportant today.
Recent adverti sements in general consumer magazi nes i1I ustrate 山 at Al coa (an
al umi numm卸 ufacl urer) is qui t巴 happy to have rcducedLhc wei ght of the standard
soda pop can by over 30%inrecenl years. A si mi l ar c1ai mby thepl asti cs i ndustry wi th
respect tothestandardmi l k j ug (gaIl on) is al 50evi dent inthesemagazi nes, al though
in出 is case thenumber is 409る. Roof coUapses in白 cNortheast due 10excessi ve snow
Tabl e 1.1 Component Pl acement Machi r悶
Machi ne Operabl c
Model BoardTypes Boardsl Hour Opemt ors/Shift HoursIDay CostI Machi ne
A 10 55 18 400,000
B 20 50 2 18 600,000
C 18 50 2 21 700,000
duri ng the 1998- 1999 wi nter will have structura1 desi gners and homebui l ders
expl ori ng 白巴 second probl em. Th巴 vi brant stock market in these ti mes has made
ba1anci ng the i nvestment portfol i o mor e chal l engi ng. Thc thirdcase may suggest a
mathemati ca1model appropri atefor such deci si onmaki ng.
1. 1. 1 El ement s of Pr obl emFor mul at l on
Inthis secti on, we will in凶 duce出 ef orma1el ements of thc opti mi zati onprobl em. In
this book, the tcrmproduct a1so 問 fers to a servi c巴 or a strategy. It shoul d be
understood出 at op自 凶 zati onpresupposes theknowl cdge of 由 edesi gnrul es for the
sp配 ific probl em, pri mari l y theability todescri bethedesi gninmathemati ca1terms.
These terms 泊cJ ude desi gn vari abl es, desi gn parameters, and desi gnfuncti ons.
Tradi ti ona1desi gnpracti 田 , 出 at is, desi gnwi thout regardtoopti mi zati on, incJ udes a11
of these el ements 剖 伽 ugh 血 ey were not formal l y recogni zed as such. Thi s al so
j usti fi es 出 eprerequi si 飽 that you must be capabl e of dcsi gni ngthe0吋∞t if you are
pl anni ngtoappl y 出 etechni qucs of opti mi 四 tion. It is al 80a good i dea 10recogni zc
that opti mi zati onis a procedurc for scarchi ng1hebest desi gnamong candi dat巴s,cach
of whi ch can produce an acceptabl eproduct . The oced for thc obj ec1or product is not
questi onedhere. but this may be due toa deci si onbased 00 opti mi zati onappl i edin
another di sci pl i nc.
Desl gn Varl abl es: Desi gn vari abl es areentities that i denti fya parti cul ar desi gn. In
出 esearchfor 白巴 opti mal desi gn,出 eseentities will change over a prcscri bedrange.
The val ues of a compl ete set of these vari abl es charactcri ze a speci fi c desi gn. Th巴
number and typeof cntities bcl ongi ng tothis set arevcry i mportant ini denti fyi ngand
setti ngup出 equ姐 titativedesi gnprobl em. It is essenti al 出 鉱 山 is choi ce captu関 白 e
essence of 山 obj ect bei ng desi 伊 ed 叩 d at 出 e samc ti me provi de a quanti tati ve
characteri zati onof thedesi gnprobl em. Inappl i cdmathemati cal termi nol ogy. desi gn
vari abl es serveas theunknowns of theprobl embei ngsol vcd. Borrowi ng an ana10gy
f romtheareaof systemdynami 回 andcontrol theory,出 ey areequi va1ent todefi ni ng
thestateof thesystcm, in由 is case, thc statcof desi gn. Typi cal l y, desi gnvari abl es Can
be associ atcdwi thdescri bi ng出巴 obj ect' ssi zesuchas its l engthand hei ght. Inother
cぉ es,they may represen1thenumber of i tems. The choi ce of desi gnvari abl es is the
responsi bi l i ty of 由 edesi gner gui dcd by i ntui ti on, expe凶 se,and knowl edge. 百 le民 is
a f undamenta1requi rement tobe mct by山 is set of desi gnvari abl es, namel y,山 eymust
be I i nearl y i ndependent . This means 出 a1you cannot cstabl i shth巴 val ueof onc of 出 e
desi gnvari abl es f rom山 eval ues of 出 eremai ni ng vari abl es throughbasi c ari thmeti c
(scal i ngor addi ti on) operati ons. For exampl e, ina desi gnhavi ng a rcctangul ar cross
secti on, you cannot have由 民 evari abl es reprωenti ng白 el cng出, wi dth, and area. If the
first t wo ar巴 prescri bed,出 e thi rdis automati ca1l y establ i shed. Incompl ex desi gns,
theserel ati onshi ps may not be very apparent . Neverthel ess, thechoi 田 of theset of
desi gn vari abl es must meet 由巳 cri teri on of l i near i ndependence for appl yi ng the
techni ques of opti mi zati on. Fr oma practi cnl pcrspcc1i vc, thc property of I i near
i ndependence i denti fi es a mi ni mumset of vari abl es that cun compl ctel y descri bethe
砕 l
'" ち
desi gn. This is si gni fi cant because the effort in obtni ni ng the sol uti on vari es as an
i nteger power of thenumber of vari abl es, and this power is 1ypi ca11y greater 山 an2.
M回 tingtherequi rement ensures reduced di ffi cul ty inmathemati cal l y expl ori ngthc
sol uti on.
百 leset of desi gn vari abl es is i denti fi edas thedesi gn vector. 百1I s v田 tor will be
consi der吋 acol umn vector inthis book. Infac1, a11 vectors arecol umn vectors 切 出 E
text. The l engthof 血 is vector, whi ch is n, is thenumber of desi gnvari 油 les io 曲 E
probl em. The desi gn vari abl es can cxpress di fferent di mensi onal qu叩 tities in 出巴
probl em, but inthemal hemati cal model , they aredi sti ngui shedby山巴 charac1er x. AlI
of thetechni ques of opti mi zati oninthis book nrebascd on出 eabstract mathemati cal
model. 百 lesubscri pt on x, for cxampl c, X3・rcpresents thethirddesi gnvari abl e, whi ch
may be thehei ght of an obj ec1in 白 e ch釘 ucteri zati onof theproduct. Thi s abstract
model is ne回 ssary for mathemati cal conveni ence. Thi s book will refer to曲 edesi gn
vari abl es inone of 出巴 fol l owi ngways:
(1) [羽 - r ef c町 i ngtothevcc10r 01 ' d巴 si gnvari abl cs
(2) X or x- ref erri ng to thc vcctor agai n, omi l l i ng 1hc square brackets for
convcni cncc if appropri ate
(3) [xj , x2' ・... ,XIl ] T_ーi ndi cati ng thc vccl or l hrough its el ements. Nol e 山 e
superscri pt transposi ti onsymbol toi denti fy it as a col unmvector.
(4) X;. i = 1, 2. . . . , n- ref erri ng toall of theel ements of thedesi gnv巴ctor.
Thc above notati ona1conveni ence is extcndcd toall vcctors inthebook.
Desl gn Paramet ers: In出 is book,山 esei denti fyconstan岱 伽 1wi l l not change凶
di 釘 erent desi gns are compared. Many 1exts use 出巴 termdesi gn parameters 10
問 prescnt 白巴 dcsi gnvari abl es we defi nedearl i er and do not formal l y recogni ze desi gn
parameters as defi ncdhc問 . The pri nci pnl reasonis that parameters have no rol e10pl ay
泊 deten凶 ni ng由巳 opti ma1desi gn. They aresi gni fi cant inthedi scussi onof model i ng
issues. Exampl es of parameters i ncl udematcri a1propc民 y,appl i edl oads. and choi cc
of shape. The paramctcrs in the abstract mathemati cal model are rep隠 sentcd in a
si mi l ar f ormas thedesi gnvec1or, excep1伽 t we use thecharactcr p. Therefore. [P] ,
P, [P"P2' ・.. .Pq1rep隠 sent 曲 eparameters of theprobl em. Note白 紙 白 el eng白 ofthe
par副 ne飽 r vector is q. Except in山 c di scussi onof model i ng,白 epar沼 neters will not
be expl i ci tl y児 島 rredto, as they arepri mari l y predet巴rmi ncdconstants in白巴 desi gn.
Des句 'nFunct l ons: Desi gn functi ons defi ne meani ngf ul i nformati on about the
desi gn. They are eval uated usi ng the desi gn vari abl es and desi gn parameters
di scussedearlier. They establ i shthemathemati cal model of thedesi gnprobl em. These
functi ons can represent desi gnobj ecti ve(s) andl or constraillts. As i ωname i mpl i es,
desi gn 0肖ecti ve dri ves the search for the opti mal desi gn. The sati sfacti on of the
印 刷 traillts establ i shes theval i di ty of thedesi gn. If not expl i ci t1y stated,出 edesi gner
is responsi bl c for id巴 nti fyi ngthc obj ecti vc and constrai nts. Mi ni mi ze themass of the
structurewi l l transl ateto叩 obj ecti veρU1ction. The 由 民 sinthemateri al must be less
thantheyi eI d strengthwi1l transl atetoa constrai ntfuncti on. Inmany probl ems, it is
possi bl efor thc same functi ontoswi tchrol es toprovi de di ffcrcnt desi gnscenari os.
ObJ ect l ve Funct i on向井 The tradi ti onal desi gn opti mi zati on probl emis dcfi n巴d
usi nga si ngl eobj ecti vefuncti on. 百 leformat of this statemcnt is usual l ytomi ni mi zc
or maxi mi ze some quanti ty 白 紙 is cal cul ated usi ng somc dcsi gn functi on. Thi s
functi on must depend, expl i ci tl y or i mpl i ci tl y, on the desi gn vari abl es. In the
literaturc,出 is probl cmis cxp問 ssed exc1usi vel y, wi thout loss of generaI ity, as a
mi ni mumprobl em. A maxi mumprobl emcan b巴 rccast as a mi ni mi zati on probl cm
usi ngthencgati vc or 出巴 reci procal of thc functi onused for theobj ecti vc functi on. ln
thefirst exampl e in加 ducedearl i er,出 c o句ecti veis tomi ni mi ze cost. 百 l erefore, 恥
desi 伊 functi onrepres印 刷 g cost wilI be theobj ecti vefuncti on. In由 esecond伺 se,
白 e obj ecti ve is to mi ni mi ze mass. I n the thi rd case, the obj ecti ve is to maxi mi zc
machi ne utilization. 百 lear国 of si ngl eobj ecti vedesi 伊 is consi deredmature today.
No newsol uti ontechni ques for cl assi cal probl cms hav巴 becnadvanced for somc ti me
now. Today. much of the wor k in appl i ed opti mi 回 ti on is di rccted at expandi ng
appl i cati ons topracti cal probl ems. lnmany cases. this has i nvol vedcrcati veuse of the
sol uti on techni ques. In 由巳 abstract mathemati cal model . the obj ecti ve functi on is
reprosentedby thesymbol f To i ndi cateits dependence on thedesi gnvari abl es. it is
f陀 quentl y cxpressed ぉ f ( x" x2.. . . • x
). A more conci sc 陀 prcsentati on is j{X).
Si ngl e obj ecti veprobl cms have onl y one functi ondcnoted by f It is a scaI ar (not a
V民 tor). Not c山 at al thoughtheobj ccti vefuncti ondepends on P (parameter vector). it
is not expl i ci tl y inc1uded inthef ormat .
Mul ti obj ecti ve and mul ti di sci pl i nary d巴si gns arc i mportant devel opments today.
Mul ti obj ∞tivedesi gn, or mul ti pl eobj ecti vedesi gn. refersωusi ngsevcraI di 仔erent desi gn
functi ons to 命 的 巴 曲 e s田 町 h for op曲 nal dωi gn. GeneraIl y,山 ey are expec舵 d ωb e
confl i cti ng0同cctives. They coul dal sobe∞opcrati ngobj ecti ves. 百 lecurrent approach
to出 esol uti onof theseprobl ems i nvol ves standardopti mi zati onproced山 首 appl i cdtoa
si ngl e ( 田 onstructed obj ecti ve opti mi zati on probl cmbased on the di fferent mul ti pl e
obj ecti ves. A pop叫 ar approachis tousea sui tabl y wei gh包 dl i n伺 r combi nati onof 出巴
mul ti pl c obj ecti ves. A practi cal l i mi tati onwi 白 血 isapproachis也 .echoi 回 of wei ghts us吋
inthemodel . Thi s approachhas not been cmbraced wi del y. An al temati veapproachof
recogni zi nga prcn抑 制 ecti ve,and sol vi nga si ngl c obj ecti vc probl emwi thaddi ti onal
constrai nts basedon血 児 問 ni ngo同 制 ve釦 ncti onscanusual l ygene峨 岨 accep帥 le
sol uti on. Inmul ti obj ecti veprobl en

Const rai nt Funct l ons: As desi gn functi ons, these wi l l be i nfl uenced by the
desi gnvari abl es. The format of thesefuncti ons requi res t hemtobe compared tosome
numeri cal l y l i mi ti ngval ue山 at is establ i shedby desi gnrequi rement, or thedesi gner.
This val ueremai ns constant duri ngthc opti mi zati onof theprobl cm. A wcl1-descri bcd
dcsi gn probl emis expected to inc1ude scveral such functi ons. whi ch can be
represented as a vcctor. The compari son is usual1y set up usi ng the threc standard
叫 甜 onal operators: = , S, andミ. Consi der our first exampl e. Letf i mJ (X) represent
出 efuncti on 白 紙 cal cul ates the vol ume of the newsoda can we aredesi gni ng. 百 le
constr凶 nt on thed巴 si gncan bc expresscdas
ルI I J (X) = 500 c m
IntlI e second exampl e. l etβ m2( X) b巴 thefuncti onthat cal cul ates thedefl ecti onof the
beamunder theappl i edl oad. The constrai nt can be statcdas
ft11l 2(X) :S; I mm
Thccons回 int functi ons can be c1assi fi edas eqllalityconstrai nts [l i kefunl (X) aboveJ
or i nequal i tycons町出 nts [l i kefun2(X)].
Probl ems wi thout constrai nts aretcrmcd unconstrai nedprobl ems. If constrai nts are
present, then meeti ng themis mor e p創 司 mount 山 an opti mi zal i on. Constrai nt
sati sfacti on is necessary beforc the dcsi gn establ i shed by 白巴 currcnt val ue of the
desi 伊 vari abl es is consi dered val i dand acceptabl e. If constrai nts arenot s姐 sfied ,
thenthereis no sol uti on. A feasi bl edesi gnis one inwhi ch all of theconstrai nts are
satisfied. An opti mal sol uti on is one 出 at has met the desi gn obj ecti ve. An opti mal
dcsi gn must be fcasiblc. The desi gn spacc cncl oscd by thc constrai nts is cal l ed the
feasi bl edomai n. Dcsi gn space is descri beda f ewparagraphs bcl ow.
Equal l ty Constral nts: Equal i ty constrai nts are mathemati cal l y neat and easy to
handl c. Numeri cal l y,出 eyJ1叫 ui remore effort tosatisfy. They areal somore restrictive
on tlI e desi gnωt hey l i mi t theregi onf romwhi ch thesol uti oncan be obtai ned. The
symbol representi ngequal i ty constrai nts intheabstract model is h. Theremay be
mor e thanone equal i ty constrai nt inthedesi gn probl cm. A vector representati on
for 叫 ual i ty constrai nts is i ntroducedthrough thefol l owi ngrepresentati on. [8] , [h

, . . . . h,]. and h
: k = 1,2, . . . .1areways of i denti fyi ng出 eequal i tyconstrai nts. 百 le
dependence on thedesi gnvari abl es X is omi ttedfor convcni ence. Notc出 at thel ength
of thevector is 1. An i mportant 陀 箇 onfor di sti ngui shi ngtheequal i ty and i nequal i ty
constrai nts is 山 at they arc mnni pul ated di fferentl y in the senrch for the opti mal
sol uti on. The number n of desi gnvari abl es in出 c probl cmmust be greatcr 出 叩 the
number of equal i ty constrai nts 1 for opti mi zati ontotakepl ace. lf n is cqual to1 . then
恥 probl emwi l l be sol vedwi thout referencetotheobj ecti ve. lnmathemati cal terms
thenumber of equati ons matches 白 enumber of unknowns. If n is less than1, thenyou
have an overdetermi ned scl of rel ati ons whi ch coul dresult inan i nconsi stent probl cm
defi ni ti on. The sct of cqual i tyconstrai nts must be l i nearl yi ndcpendent . Broadl y, this
i mpl i es 白 紙 you cannot obtai n one of the cons町aints f romel ementary ari thmeti c
operati ons on thercmai ni ngconstraints. This serves 10ensurethat themathemati cal
searchfor sol uti onwill not fai l. These tcchni ques arebasedon methods f romIi ncar
al gebra. Inthestandardformat for opti mi zati onprobl ems, the叫 ua1itycons町aintsare
wri ttenwi tha 0 on theri ght-handside. Thi s means 白 紙 山 eequal i tyconstrai nt inthc
first exampl e will be exp陀 ssedas
hl (X): f unl (X) - 500 = 0
Inpractica1probl ems, equal i tyconstrai nts arerar官 Iyencountcred.
I nequal l ty Constrai nts: In叫 ua1itycons釘 aints app回 r more natural l y inprobl cm
formul ati on. Inequal i tyconstrai nts alsoprovi demore flexibility indcsi gnselection.
The symbol represcnti ngi nequal i tycons釘'aints intheabstract model is 8. There may
be more 出 叩 onc in珂 ua1ity constrai nt in the desi gn probl em. The vector
repr巴sentati onfor i nequal i tyconstrai nts is si mi l ar towhat we hav巴 seenbefore. Thus,
[G) , [81,82, . . . ・ 811/], andめ: j = 1, 2, . . . , m arc ways of i denti fyi ngIhei nequal i ty
cons甘 aints. m reprωents 出 enumber of i nequa1ity constraints. AIl desi gn functi ons
explicitlyor implicitlydepend on thedesi gn(or i ndependent) vari abl eX. 8 is usedto
descri ho bo出 loss thun or cqual 10恒 ) 朗 dgreal cr 白 山 01" equlll 10(と) constralnts.
Strictly greater 出 叩 ( >) and strictly Icss th聞( <) arenot usedmuch inopti mi zati on
because出巴 sol uti ons are 凶 ua11y expectedto lie at 出 econstrai nt boundary. In the
standard format, all probl ems arc expressed wi 自 由 eS relationship. Moreover, the
ri ght-handsi deof 出 eSsi gni sO. 百 lei nequal i tyconstrai nt f romthesecondexampl e
f t m2( X) is set up as
81(X): ル n2( X) - 1三 O
In 由巳 case of i nequal i ty constrai nts a di sti ncti onis madc as to whether the dcsi gn
vari abl es lieon theconstrai nt boundary or in出巴 interior of theregi onbounded by白 E
constrai nt. If the set of d巴 si gn vari abl es lie on the boundary of 白 e constrai nt ,
mathemati cal l y, this expresses 出 efact that constrai nt is satisfiedwi thstrict equal i ty,
白 紙 is,g = O. The constrai nt acts like an equal i ty constrai nt. In opti mi zati on
termi nol ogy,血 is parti cul ar cons住 aint is referredtoas anacti veconstrai nt. If theset
of desi gnvari abl es do not lieon theboundary, that is, theyli 巴 i nsi de出 eregi onof thc
constrai nts, they arc consi de陀 d i nacti veconstraints. Mathemati ca1ly,出 econstrai nt
satisfies 出 erclationg < O. An i nequal i tyconsU' ai nt canthercforebe either activeor
lllactJ ve.
Sl de Constral nts: Si decons釘 aints area necessarypart of thesol uti ontechni ques,
es戸 ci a1ly numeri cal oncs. They express the range for the desi gn variablcs. Each
desi gnvari abl emust be bound by numcri cal va1ues for its l ower and upper Ii mi t. The
desi gner makes this choi cebasedon his anti ci pati onof anacceptabl edesi gn.
Desi gn Space: ηl e desi gn space,出 e space 白 at wi¥l be searched for opti mal
desi gn, is the Eucl i dean or Cartesi an n-di mensi onal space generated by the n
i ndependent desi gn vari abl es X. Thi s is a genera1ization of the three-di mensi onal
physi cal space wi th whi ch we are familiar. For ten d巴si gn vari abl es, it is a
ten-di mensi onal space. Thi s is not easy toi magi ne. Il is a1sonot casy toexpress this
i nformati on 白 rough a fi gure or graph because of the Ii mi tati on of the
three-di mensi onal worl d. However, if 也 ed巴:signvari abl es arei ndependent,出 enthe
l1-di mensi onal consi derati ons aremere extrapol ati ons of the由 民 e-di mensi ona1reality.
Of course, we cannot geometri ca1lydefi n巴 themthoughwe will be worki ng wi ththc
numbers. The si deconstrai nts Ii mi t thesearch陀 gi on,i mpl yi ngthat onl y sol uti ons山 at
lie wi thi n a c巴 rtain regi on will be acceptabl e. They defi ne an n-di mensi onal
rectangul ar regi on(hypercube) f romwhi ch白 efeasi bl eand opti mal sol uti ons must be
chosen. Later, we wi¥l s白 血 at themathemati ca1model s inopti mi zati onareusual l y
descri bed by nonl i near rel ati onshi ps.τ11e sol uti ons to such probl ems cannot be
ana1ytically predi cted 鎚 they are typi ca1ly govemed by 由 e underl yi ng numeri cal
tec加 i queusedtosol vethem. It is neccssary to( 田 町 ict thesol uti ons toanacceptabl e
regi on. The si deconstrai nts provi des ready mechani smfor i mpl ementi ngthis limi t.
Care must be taken that thcse limits arenot i mposed overzeal ousl y. Thcre must be
reasonabl espacefor 出巴 numeri cal techni ques 10operate.
The St andar d Format : The above defi ni ti ons a1l owus to assembl e Ihe general
abs回 .ct mathernati cal model as
Mi ni mi ze f(x. , x2, . . . ,x
Subj ect to: hl (x" X2・・・・ ,XI I )=O
h2(x. , x2, . . . ,x
) = 0 (1. 2)
、‘., 1


h/(XI' X2・・.. ,XI I ) =0
8. (X" X2・・.. ,Xn) 壬 O
82(XI' X2,・ ・ ・ ,Xn) S 0 (1.3)
gm(XI ・X2・・.. ,Xn) SO
xl SXj Sxi , i = 1,2,..., n (1.4)
The same probl emcanbe expressedconci sel yusi ngthefol l owi ngnotation:
Mi ni mi ze f(x. , X2, . . . ,X
) (1. 5)
Subj ect to: hk(xt , X2・・・・ ,X
) = 0, k = 1, 2, . . . , I (1.6)
め(X" X2, • . . ,X
) S 0, j = 1,2,. . . . m (1. 7)
画 幽 画 面 孟 副
xl sxj Sxれ i =1,2,... ,n
Expl oi ti ngvector notati onthemathemati cal model is
Mi ni mi ze j(X) , [X].
Subj ect to: [h( X)]1= 0
匂( X)]tnS 0
X10wSX S Xup
(1. 8)
(1. 9)
( 1. 11)
The above mathemati cal model expresses 恥 fol l owi ng standard format of the
opti mi z岨 onprobl emexpressedinnatural l anguage:
Mi ni mi ze theobj ecti vefuncti onj , subj ect to1 equa¥ityconstrai nts, m i ncqual i ty
constrai nts, wi th 山 e n desi gn vari abl es I yi ng between pr, ωcri bed l ower and
upper limits.
百 letechni qucs in出 is book will appl y tothc probl emdescri bcdin出 c abovc f ormat.
To sol veany speci fi c desi gnprobl emit is requi rcd10refunnul atc theprobl cminthe
above manner so 白 紙 自 己 methods can be appl i ed di rectl y. Al so in this book, the
techni ques are devel oped progressi vel y by consi deri ng the standard model wi 出
向 ducedel ements. For exampl e, th巴 unconstrai nedprobl emwi¥l be expl o問 dfirst. The
equal i ty cons回 ints a陀 consi derednext, fol l owedby thei nequal i ty constrai nts. 叩 d
fi nal l y thecompl ete model . Thi s represents a natural progressi onas pri or knowl edge
is used to devel op addi ti onal condi ti ons 白 紙 need to be sati sfi edby the sol uti on in
白 隠 ei nstances.
1. 1. 2 Mat hemat l cal Model l ng
Inthis secti on,出 ethreedesi gnprobl ems i附 ' Oducedearl i er wi l l be佐 ansl atedto出 e
standardf ormat. The mathemati cal mode1wi l l first be i denti fi ed. The second probl em
requi res i nformati onf roma course inmechani cs and materi al s. Thi s shoul dbe wi thi n
血 .epurvi ewof most engi neeri ngstudents.
Exampl e 1.1 New consumer research i ndi cates 山 at peopl e like to dri nk about 0.5
liter of soda pop at a ti me duri ng the summer months. The fabri cati on cost of the
redesi gnedsoda can is proporti onal 10thesurfaceareaand can be esti mal edal $1. 00
per square meter of l hemat巴 rial used. A ci rcul ar cross secti on is the most pl ausi bl e
gi vencurrent l ool i ngavai l abl efor manufacl ure. For aestheti c reasons, tl1e hei ght must
be at least twi ce tl1e di ameter. Studi es i ndi cate 血 at hol di ng comf ort requi res a
di ameter between 6 and 9 cm.
Fi gurc 1.1 shows a sketch of tl1e can. In most product desi gns, parti cul arl y in
engi n巴eri ng,the model is easi er to devel op usi ng a figure. The di ameter d and the
hei ght h are suffi ci ent to dcscri be tl1e soda can. What about the tl1i ckness t of 出 e
Fl gure1.1 Exampl e 1.1- Desi gn01a newbeveragecan.
matcri al of the can? What arc l he assumpti ons for thc dcsi gn probl cm? One of the
assumpl i ons coul d be lhat t is smal l cnough to be i gnored in thc cal cul ati on of thc
vol ume of soda in出 ecan. Another assumpti onis that 由 emateri al 陀 qui redfor l hecanis
onl y thecyl i ndri cal surfac巴 宙 開 . τ 'hetopand botl omof thccan will be fittedwi thcnd caps
白 at w泊 provi de出 emechani smby whi ch thesoda canbc pow叫 . 官 邸 is not part of 白 is
dcsi gnprobl em. Inthe白百 tattempt at dcvel opi ngthemathemati cal model we coul dstart
out by consi deri ng由 equ叩 tities i denti fi ed曲 oveas des取 1variables:
Desi gn vari abl es: d, h, t
Revi ewi ng thestal ement of thedesi gnprobl em, one of theparameters is thecost of
materi al per uni t area白 紙 is gi venas l (t per sqU8I 官 centi meter. τbi s is i denl i fi edぉ C.
Duri ng thesearchfor theopti mal sol uti on出 is quanti ty will be hcl dconstant at 出 e
gi ven val ue. Note that if this val uc changes, l hen 出 e cost of thc can will
correspondi ngl y change. This is what we mean by a desi gn p釘 ameter. Typi cal l y,
change inparameters wi¥l cause thesol uti onto民 間 computed.
Desi gn parameter: C
The desi gnfuncti ons wi l l i ncl ude白 ecomputati on of thevol ume encl osedby thecan
and thesurfaceareaof thecyl i ndri cal secti on. The vol ume inthecan isπ cf2h14. ηl e
surfaceareais 1tdh. The aesthcti c constrai nt requi res 出 at h 2d. The si decons甘 aints
on山 di ameter areprescri b吋 in出 eprobl em. For compl el 巴ness thesi deconstrai nts
on恥 other vari abl es have tobe prescri bedby thedesi gner. We can formal l y set up
theopti mi zati onprobl em硲
Mi ni mi ze f ( d, h , t): C7rd1z (1.13)
Subj ecl l O: h M,h,t): 7rd
/1/4- 500 = 0 (1.14)
掴 一一ーーーー


Intui ti vel y, therc is some concemwi th the probl emas expressed by Equati ons
(1.13ト( 1. 15) even though山巴 descri pti onis valid. How can theval ue of thedesi gn
vari abl et be estab1i shed? The vari ati onint does not a仔 'ect thedesi gn. Changi ng the
val ueof t does not changef , h
, or gl ・Hence,it cannot be a desi gnvari abl e. (Note: the
vari ati onint may affect theval ue of c.) lf 山 is werc a practi cal desi gnprobl cm, then
出巴 cans have to be desi gned for i mpact and stacki ng strength. In 白 紙 C凶 e,t wi l l
probabl y be a critical desi gnvari abl e. Thi s wi l l requi resever心 addi ti onal s凶 cl ural
constrai nts intheprobl em. It coul d serveas an i nteresti ngextensi ontothis probl em
for homewor k or proj ect 明 leopti mi zati onprobl emafter droppi ng t 佃 d expressi ng
[d. h] ぉ [XI ・X2] becomes:
ィ ,



gM. h. t): 2d- h S O
6Sds 9; 5 s /t s20; O. OOI St sO. Ol (1.15)
Fl gure 1.2 Exampl e 1.2- Canlllever beam.
Mi ni mi ze f(XI ,X2): c : 町 内 ( 1.16)
gl (ズl・XU: 2x1- X2S0 (1.18)
l ntui ti vel y,出 el arger theval ue of F. thegreater is山巴 cαro 田 s鴎8-S総ecl 叫lio 叩 na 陥a叫l are 可官潤=泊 anecωs紅 v
t ω.0ha 肌 nd 仙leit. 11c 棚 a 叩n b 恥e蹴 捌i 均l 砂yconcl uded 山伽a飢t i げf F w悶 ad 伽巴 ωs叩i
b巴 sel at its l ower limit, a8l hemass of thebeamwill be di recl l y proporti onal 10由 e
areaof croS88cc(I un. No techni ques arer,叫 ui redf. or lhisconcl usi . on. By . our defi ni ti on,
F is a go. od candi date as a par創 neter rather 白 肌 as a desi gnvari abl e. Si mi l arl y, the
l arger theval ue of L , 白 eg問 ater is themoment 白 紙 F generates about thecanti l evered
end. 百 lis wi l l requi re an i ncreasein 山 earea. of cross-secti on. Therefore, L is not a
good choi ce f. or 山 edesi gn vari abl eei ther as thebest choi ce f. or it will bc i臼 l. ower
I i mi t. To represcnt materi al as a desi gnvari abl e. we necd touseitsSl ruCl ural propcrti es
in出 eformul al i onof the probl em. Typi cal l y. a mal eri al is characl eri zedby its spec凶 c
Subj ecl l o: hl (xl ・Xv: ru弘 / 4- 500=0
6 SX) S 9; 5 SX2 S 20
The probl em問 presenl edby Equati . ons (1.16)一(1. 18) is 恥 mathemati cal mode¥f. or
th巴 desi gnprobl emexpressedinthesl andardf ormat. For this probl emno extranc. ous
i nformal i . on was n田 氏 dtoset up出 eprobl emexcept for some geometri cal rel ati ons.
Exampl e 1. 2 A canti l ever beamneeds 10be desi gnedtoc紅 ry a poi nt l oadF al the
end . of thebeam. of l engl hL. The cross secti onof thebeamwiII be intheshape of the
letter 1 (ぉ ferredtoas 阻 1・b回 m). 百 lebeamshoul dmeet prescri bedfai l urecriteria.
Thcre is al so a I i mi t on its defl ecti . on. A beamof mi ni mummass is 閃 qui red 10be
dωi gncd.
Fi gure 1.2shows a si devi ew. of thebeamcarryi ngthel oadand Fi gure 1.3a typi cal
cr. oss s目 白 nof t h巴 be釘 n. The I -shapedcross secti onis symmetri c. The symbol s d, tw.
, and t r c. orrespondtothedep出. of thebcam, thi ckness of theweb, wi dth of th巴 fl ange,
and thi ckness . of thefl ange, respecti vel y. Thes巴 quanti ti es aresuffi ci ent todefi ne出 E
cr. oss secti on. A handbo. ok or textb. ook on strcngth of materi al s can ai d 白 e
d巴vel opment . of th巴 desi gnfuncti ons. An i mportant assumpti on for this probl emis白 紙
we wi l l be w. orki ng wi thi n the elastic I i mi t of thc materi al , where there is a I i near
relati . onshi pbetween血 estress and thestrain. AlI . of thevari abl es i denti fi edinFi gure
1.3wi l l s回 ngl y affect thesoluti . on. They aredesi gnvari abl es. The appl i edf. orceF
wi l l al sodi rectl y affect theprobl em. So will its locati . on L. H. ow about 由巳 materi al ?
Stecl is dcfi ni tel y supcri . or 1.0 c. opper for the beam. Shoul d F, L, and the materi al
pr. operti es be i ncl udedas desi gnvari abl es?
' l
4・t W -

Fl guro1.3 Exampl e 1.2- Cross-sectlonal shape.
wei ght , y, its val ue of the modul us of elasticity. E. its modul us of rigidity. G. its yi el d
l i mi t intensi onand compr凶 si on.σyi e' d. i ωyi el d insh閲 兵 τyield. its ul ti matestreng白
intensi on.σul t. and its ul ti matestrengthinshear.τul l ・Opti mi 却 l i ontechni ques i denti fy
di ffe陀 nt va1ues of the desi gn vari abl es. If E is the vari abl e used to 閃 present 出 E
materi al desi gn vari abl e in a probl em. then it is possi bl e the sol uti oncan 問 qui rea
materi a1白 at does not exi st,白 at is. a va1ue of E still undi scover芭 d. For a11 i ntents and
p凶 poses. 也 is is beyond the scope of thedesi gner. Agai n. materi a1 as a parameter
makes a lot of. sense, and血 eopti mi zati onprobl emshoul dbe rei nvesti gatedif F,ム
or themateri al changes. Concl udi ng f rom出 is di scussi onon model i ng. thefol l owi ng
I i st can be establ i shedfor this exampl e:
Desi gn p釘 祖 1隠et胞er肱sぶ F. L ム. {y. 芯 ι G. σ 叫Y'悶eld'τ1: yi陪 州 eld' σ 円町u凶11' 1: τ 九E
Desi gn vari abl es: d, t
' b 糾r ' I 今
In the dev巴 l opment of the mathemati cal model , standard techni cal defi ni ti ons 紅 巳
used. Much of the i nformati on can be obtai ned f roma mechani cal engi neeri ng
handbook. Th巴 first desi gnfuncti onis 出巴 wei ght of thebeam. Thi s is theproduct of
y, L, and Ac, where Ac is 出 eareaof cross secti on. The maxi mumslress due tobendi ng
σ""ndcan be cal cul atedas theproduct of FLdI21c・wher eι i s thc momenl of inertia
about thc ccntroi d of the cross secti on al ong the axi s paral l el to the f1ange. The
m凱 i mumshear stress in出巴 cross secti onis expressedas FQcl l c1w, wherc Qc is first
moment of areaabout thecen位。 idparal l cl tothc fl ange. The maxi mumdefl ecti on
of thebeamwi11be at theend of thebeamcal cul atedby theexpressi onF L
/ 3El
' Ac
of thecross secti onof thebeamis 2brtr + Iw(d- 2tf). The first momcnt of areaQc can
bc cal cul atedas 0. 5b
tf (d- tr) + 0.51
(0. 5d- tf)2. Fi nal l y. 出 c momcnt of i nerti aιi s
obtai nedf rom(b凶 112) ー ((b
- tw)(d- 2tf)3/12). In山 fol l owi ng,for convcni ence,
we wi l l conti nue to use Ae, Qc' andι i nst cad of detai1i ng their dependence on 曲 e
desi gnva1ues. Associ ati ngx, wi thd, X2wi tht
' X3wi thb
, and X4wi thlr, so白 at X=
[x,・X2・X3' x41,白 eprobl eminstandardformat is
Mi ni mi ze .ftX): YLAC ( 1.19)
Subj ect to: g,(X): FL x , / 21c- σyie'd 0
g2(αx 均): FQJ 1.x: ぬx巧2- τ1: yie 眼 刷 e

,   由

0. 01 x, 0.25; 0. 001 X2 0.05;
0. 01 X3 0.25; 0. 001 X4 0. 05
(1. 20)
To sol ve 出 is probl em, F must bc prescri bed (10,000 N), L must be gi vcn (3 m).
Matcri a1must bc sel ected(stee1: γ=7860 kgl m
; O"yield= 250 E+06 N/ m
; τyicld= 145
E+06 N/ m
). A1so themaxi mumdct1ecti onis prcscri bcd = 0. 005m).



d 町山

Thi s is an exampl e wi th four desi gnvari abl es, threei nequal i ty constrai nts. and
no equal i ty constrai nts. Othcr vcrsi ons of this probl emcan easi l y be formul ated.
It can be reduced toa two-vari abl c probl cm. Standard fai l urecri teri awi th respect
to combi ned stresscs or pri nci pa1 strcsses can bc formul ated. If the canti l evered
end is bol ted, then addi ti ona1 desi gn functi ons regardi ng bol t failure need to be
exar凶 ned.
Exampl e1. 3 MyPC Company has deci dedtoi nvest $12 mi l l i oninacqui ri ngsevera1
ncwComponent P1acement Machi nes tomanuf acture di 仔 erent ki nds of mo出 eぬ oards
for a newgenerati onof personal computers. Three model s of thesemachi nes areunder
consi derati on. Tota1number of operators avai l abl e is 100 becausc of 出 el ocal l abor
market. A fl oor space constrai nt needs to be sati sfi ed because of 血 e di fferent
di mensi ons of 恥 se machi nes. Addi ti ona1 i nformati on rel ati ng to each of 白 e
machi nes is gi veninTabl e 1.1. 百 ecompany wi shes todetermi ne how m叩 y of each
ki ndis appropri atetomaxi mi zc thenumber of boards manuf actu悶 dperday.
lt is difficu1t touse a fi gureinthis probl emtosct up our mathemati cal model . The
number of machi nes of cach model necds to be determi ned. T his wi l l serveas our
desi gnvari abl es. Let XI represent th巴 number of Component Pl acement Machi nes of
Model A. Si mi l arl y, X2wi1l be associ atcdwi thModel B, and X3wi thModel C.

D凶 igll vari ubl es: x,・X2,X3
T he i nfonnati on in Tabl e 1.1 is usedto set up the desi gn functi ons interms of 出 e
desi gnvari abl es. An assumpl i on is made that all machi nes arerunfor threesh出 s. T he
cost of acqui si ti onof themachi nes is thesumof 出 ecost per machi ne mu1ti pl i edby
出巴 number of machi nes (g1)' Th巴 machi nes must satisfy山 efl oor spacc constrai nt (82)
whi ch is defi ned wi thout details in Eq. (1. 23). The constrai nt on the number of
opemtors is three ti mcs 白巴 sumof 曲 eproduct of the number of machi nes of 伺 ch
model and theoperators per shift (g3)' T he uti l i zati onof each machi ne is thenumber
of boards per hour ti mes the number of hours the machi ne operates per day. T hc
opti mi zati onprobl emcan be assembl ed in山巴 fol l owi ngform:

Ma瓦 i mi ze f (X): 18 (1. 21)
T Mi ni mi ze f ( X) : - 18x 55 x x, - 18x 50 X x
- 21X 50 x X
Suゆ ct to: g,(X): 400,OOOx, + 600. 000X2 + 700,000X3 12,0肌 000 (1. 22)
g2(X): 3x, -X2+ X3 30 (1. 23)
83(X): 3xI + 6X2+ 6x
100 (1. 24)
XI 0; X2 0; X) 0

一 阻 止 一-
  甲 山 岨 臨 調 甘 通
Equati ons (1. 21)ー(1. 24) express 恥 mathemati cal model of theprobl em. Note伽 tin
this probl emthereis no product bci ngdcsi gned. Here a strategyfor pl aci ngorder for
恥 number of machi nes isbei ngdctermi ned. Equati on(1.21) illus国 防 thetranslation
of the m蹴 i mi zati on obj ecti ve into an 叫 ui val ent mi ni mi zi ng one. The i nequal i ty
constrai nts inEquati ons (1.22)ー(1. 24) aredifferent f romtheprevi ous two ex剖 nples.
He隠 theri ght-handsideof theS operator is nonzero. Si mi l arl y, thesi deconstrai nts
are bound on the l ower sideonl y. Thi s is done deliberately. There is a significanl
di fferencebetween出 isprobl emand theprevi ous two. Al l of thedesi gnfuncti ons here
arelinear. Thi s probl emis c1assifiedas a linear prograrnmi ngprobl em. The sol uti ons
10 these typcs of probl ems are very different than thc sol uti on 10 thc first two
cxampl 巴 s,whi ch arerecogni zedas nonl i n巴ar programmi ng probl 巴 ms.
1.1.3 Nature of Sol uti on
百 le出向 ecxarnpI 巴sin出cprevi ous sectionarc usedtodi scuss 出enatureof sol uti onsω
the opti mi zati onprobl em. Exarnpl es 1.1 and 1.2dcscri be血 enonl i n田 r programmi ng
probl cm, whi chforms themaj or part of 山 is book. Exarnple1.3is a lincar programmi ng
probl em, whi chis very significant indcci si onsci cnccs, but ra路 inproduct design. Its
inc1usi on here, whi l e necessary for compl ctcness, is also i mportant for understandi ng
contemporary opti mi zati ontechni ques for nonl i near programmi ng probl cms. Exarnpl es
1.1and1. 2a回 dealt wi thftrSt. Between thel wo, we noticethat Exarnpl e1.1isquitesi mpl e
relativetoExarnpl e1.2. Second , thefour variables inExampl e 1.2make it difficult touse
i1l ustrationtoestablishsome of thcdiscussion.
叩 伺 representation01Exampl e1.1.
bo山 of thesc functions. 百 le叫 uality constraint gl is lincar. lnEquation(1.18) 出 is is
evi dentb∞ausethedesi 伊 variablesappcar by出 emsel veswi thout bei ngraised10a powcr
o山 町 出 回 1. As we devel op白 etechni ques for appl yi ngopti mi zati on,出 is distinctionis
i mportant to keep in mi nd. It is also significant 白 紙 graphi cal representati on is
typicallyrestrictedtotwo variables. For 出 reevari abl es, we needa fourthdi mensi on
10 resol ve the i nformalI on and three-di mensi onal conl our plots are not easy to
iII ustrate. The power of i magi nati onisnecessarytoovercome自己 sehurdles.
百 1巴 probl emreprcsentedinFi gure1.5provi dcs叩 opportuni tytoidentifythegraphical
solution. First, thefeasibleregi onisidentified. InFi gure1.5山 is is 出 E叫 ual i tycons回 int
above theincquality cons回 int . Bothfeωibleandopl i mal must occur fromthis 陀 :gion.
Any solutionfrom出 is regi onis anaccepl abl e(仰 sible) solution. There areusual l ya
largenumber of suchsol uti onsー infinilesolutions. If opti mi zati onis i nvol ved, thcn
thesc choi ces must be rcduced tothebest one wi th 陀 spcct to some criteriaー the
objeclive. 1nthis probl eml fi gure, thesmal l cst val ueoffi s desired. The l owest val uc
of f is just less than the contour val ueof 308. lt is at 山 eintersection of the two
constraints. Whi l e the val ueof f needs to be cal cul ated, theopti mal val ues of the
desi gnvari abl es, readf romthefigure, areabout6. 75and 13. 5 , respectively. Another
significant i temof i nformati on obtai ned f romFi gure 1.5 is 白 紙 gl is an active
constrai nt . Whi l e there are infinite feasible sol uti ons 10 thc probl em, the opti mal
sol uti onis Ul tique.
Fl gure1.4


SOLUT10N T O EXAMPLE 1.1 Thc si mpl cst determi nati on of nonl i neari ty is
througha graphi cal 陀 P陀 sentati onof 出 c dcsi gnfuncti ons i nvol vedinthc probl em.
百 lis canbe donc 回 sily for ODe or two variables. If thefUDCti ODdoes not plot as a
straight lineor a pl ane,出 enit is nonl i ncar. Fi gu隠 1.4shows a three-di mcnsi onal plot
of Exampl c 1.1. The figureisobtai nedusi ngMATLAB. Chapter 2 will provi d巴 dctailcd
instructions for drawi ngsuchplots for graphi cal opti mi zati on. The threc-di mcnsi onal
問 pr凶 entati onof Exarnpl e1.1inFi gure1.4does not real1y enhance our understandi ng
of theprobl emor thesolution. Fi gure1.5isanal tematerep陀 scntati onof theprobl em.
Thi s is th巴 oneencouragedinthis book. Thc hori zontal 蹴 is rep陀 scnts thedi ameter
(XI), and 出巴 vertical axis 由 e hei ght (xu. 1n Fi gure 1. 5 the equal i 砂 constrai nt is
markcd appropri atel yas h 1. Any p創 rof val ucs on出 is linewill gi vea vol ume of 500
• The i nequa/i tyconstrai nt is shown as g 1. The pair of val ues on this lineexact 1y
satisfies thc aes山 etic requi 陀 ment. Hash lines on thesideof thei nequal i tyconstraint
cstablish thedi sal l owed陀 gi on for thedesi gn variables. 百 leconstrai nts aredrawn
thi cker for emph凶 is. The scal ed 0匂'ecliveルnCl i on is repr, ωen句 d through sevcral
l abel cd contours. Each contour is associ ated wi th a 白xed val ue of the obj ccti ve
functi on and these val ues are shown on the figure. 百 le range of the two axes
establ i shes thesi deCOl l Straints. The obj ecti vefuncti onf andtheequal i l yconstrai nt h 1
創 'cnonl i near si ncc theydo not plot IISstraight lines. Rcferri ngtoEquati ons (1.16) and
(1.17) , this is substanti atedby thepl ' OduCISof thetwo unknowns (desi gnvarillbles) in
An i nteresti ngobservati on in the above set of equati ons is 山 at there areonl y
t wo desi gn vari abl es but three constrai nts. Thi s is a val i d probl em. If the
constrai nts were equal i ty constrai nts, 出 en this woul d not be an acceptabl e
probl emdefi ni ti on as it woul d vi ol ate 白 e rel ati onshi p between the number of
vari abl es and the number of equal i ty constrai nts. Theref ore, the number of
i nequal i ηconstrai nts arenot rel atedto thenumb巴 r of desi gnvari abl es used. Thi s
fact al so appl i es to nonl i near constrni nts. Fi gure 1. 6 is a scal ed pl ot of ゆc
functi ons (1.25) ー(1.28). Choosi ng thefirst quadrant sati sfi es thesi deconstrai nts.
τ'he l i neari ty of all of the functi ons is i ndi catedby the strai ght-l i ne pl ots on the
fi gure. Agai n, several contours of f are shown toai dtherecogni ti onof the sol uti on.
Al l of thei nequal i ti es aredrawn wi thhash marks i ndi cati ngthenonfeasi b1e si de
of the constrai nt . The feasi bl e regi on is encl osed by the t wo axes al ong wi th
constrai nts g2 and g3' The obj ecti veis toi ncreas巴 theval ue of f (whi ch is i ndi cated
by the contour l evel s) wi thout l eavi ng the feasi bl e regi on. The sol uti on can be
(1. 28)
83(X): 3xI + 6x2 70
X 2 ~   X
O ;

25 20
Conl our plol 01modl l l edExampl e 1.3.
number 01Model A
Fl gure1.6
SOLUTI ON T O EXAMPL E 1.3 Incontrast toExampl e 1.1, all of therel ati onshi ps
in 恥 mathemati cal model , expressed by Equati ons (1.21)ー(1. 24),are linear. The
word equati onis general l y used todescri betheequi val ence of t wo quanti ti es on ei ther
si de of the = sign. In opti rni zati on, we come across mostl y inequalities. 百】 e word
equati ol l will be出 edtoi ncl udeboth of thesesi tuati ons. InExampl e 1. 1,出 ena随 時
of sol uti on was expl ai ned usi ng Fi gu問1. 5. In Exampl e 1. 3, the p閃 sence of thr田
desi gnvari abl es deni es 山 is approach. I nsteadof desi gni nga newprobl em, suppose
白 紙 thecompany p毘 si dent has d配 i dedto buy fi vemachi nes of thethi rdtype. The
mathemati cal model , wi tht wo desi gnvari abl es Xl and X2' is reconstructedusi ngthe
i nformati onthat X3 is now a parameter wi ththeval ueof 5. Substi tuti ngfor x3 inthe
model for Exampl e 1. 3, and cl eani ngup thefirst cons回 int :
M砿 i mi ze
Subj ect to:
(1. 25)
(1. 26)
( 1. 27)
j(X): 990 XXI + 9∞x X2 +5250

Conl our plol 01Exampl e 1.1.
gl ぽ) : O.4xl + 0. 6x2 8.5
dlameler (cm)
g2( X) : 3xI - X2 25
Agure 1.5
spottcdat thei ntersecti onof constrai nts g2叩 dg3. The val ues of thedesi gnvari 油 Ics
at thesol uti oncan be readoff f romthc pl ot. It is not over yet. Bef ore proceedi ng
further il is neccssary toacknowl edgc that a sol uti onof 6.5machi nes for Model B
is unacceptabl e. The sol uti on must be adj usted to the ncarest i nteger val ues. In
aCl ual practi ce. this adj ustmcnt has to bc made wi thout vi ol ati ng any of the
constrai nts. The need for i nteger vari abl es is an i mportant consi derati onindcsi gn.
V回 abl es 白 at ar巴 飽 qui redtohave onl y i nteger val ues bel ong to白 etypeof vari abl es
cal l eddi scretevari abl es. τ'his book most1y consi ders conti nuous vari abl es. Al most a11
of the mathemati cs an engi n田 r encounters, especi al l y academi cal l y, bel ong to 曲 E
domai n of conti nuous vari abl es. Thc assumpti on of conti nui typrovi dcs very fast and
cffi ci cnt techni ques for the search of constrai ned opt i mumsol uti ons, as we will
devel op in this text . Many real- I i fc desi gns, especi al l y those that use off-the-shel f
materi al s and components, determi ne di screte model s. Di screte programmi ng and
i nteger programmi ng are di sci pl i nes that address these model s. The nature of 出巴
searchmethods,出 at is, methods used for searchi ng出 eopti mal sol uti on, indi scrctc
prograrnmi ng is very di fferent f rom山 at inconti nuous programmi ng. Many of th凶 巴
methods are based on the scanni ng and repl acement of the sol uti ons through
exhausl i ve search of the desi gn space in the desi gn regi on. In this book, di sc陀 tc
probl ems arc typi cal l y handl edas if they were conti nuous probl ems. The conversi on
to由 edi scrctesol uti onis thefinal part of thedesi gneffort and is usual l y perf ormed
by the desi gner outsi d巴 of any opti mi zati on techni que we devel op in this book.
Chapter 8 di scusses di screteopti mi zati onwi th exampl es.
Getti ngback to出 el i near programmi ng probl emj ust di scussed, it shoul d al sobe
notedthat onl y theboundary of the feasi bl eregi onwi l l affect thesol uti on. He陀 is a
way that will make白 is apparcnt . Not e that theobj 巴cti vefuncti oncontours ar官 strai ght
lines. A di fferent obj ecti vefuncti onwill be di spl ayedby paral l el l i nes wi tha di fferent
sl ope. I magi ne theseIi nes on thefi gurei nsteadof theactllal obj ecti ve. Not e出 at the
sol uti onal ways appears tolieat th巴 i ntcrsecti onof theconstrai nts, that is, th巴 corners
of thed巴 si gnspacc. Sol uti oncan nevcr com巴 f romi nsi deof thefcasi bl e陀 gi on,unl i ke
in the nonl i near programmi ng probl em, whi ch accommodat es sol uti on on the
boundary and f romwi thi nthefeasi bl eregi on. Recogni zi ng thc natu問 of sol uti ons is
i mportant in devel opi ng a search procedure. For exampl e, in l i near prograrnmi ng
probl ems, a techni que empl oycd tosearchfor theopti mal d巴si gnneed onl y searchthc
boundari es, parti cul arl y the i ntcrsecti on of the boundari es, whereas 泊 nonl i near
probl ems, thesearchprocedure cannot i gnorei nteri or val ues.
1.1.4 Characterl stl cs of t he Sear ch Pr ocedur e
百 lesearchfor theopti mal sol uti onwill depend on出 enatureof theprobl embei ng
sol ved. For nonl i near probl ems, it must consi der thefact that sol uti oncoul d liei nsi de
the feasi bl e regi on. Exampl e 1.1 throl l gh Fi gurc 1.5 wi l l provi de 出 e context for
di scussi ngthescarchprocedure for nonl i ncar probl ems. Si mi 1arI y, thecharacteri sti cs
of the search process for l i near probl cms wi l l be expl oredusi ngEx四 lple 1. 3 and
Fi gure 1. 6.






Nonl l near Prob/ ems: Except for a f ewc1asses of probl cms, sol uti ons tonon1i near
probl ems are obtai ned through numeri cal anal ysi s. Through computβr code,
numeri ca1 anal ysi s becomes numcri cal techni ques. The methods or techni ques for
fi ndi ngthesol uti onto opti mi zati onprob1ems arecal l edsearchmethods. Inappl i ed
mathemati cs and numeri cal t田 hni ques these創買う referredto as i terati ve techni ques.
I mplici t1y this means seveml tries will be necessary beforc the sol uti on can be
obtai ned. A1so i mpl i edis that cach try or searchis executedina consi stent manner.
I nformati onf rom山 previ ous iterationis uti l i zedinthecomputati on of val ues inthe
present sequence. Thi s consi stcnt manner or pr∞ess by whi ch thesearchiscarri edout
and 恥 sol uti on determi ned is ca¥1ed 叩 al gori thm. Thi s terma1so refers to the
tr佃 sl ati onof theparti cul ar searchprocedure i ntoan ordereds叫 uenceof step-by・step
acti ons. The advantage is deri ved mostl y in theconversi on of these steps, through
some computer 1anguage, i ntocode出 at wi1l execute on山 ecomputcr. Devel opi ng山 E
a1gori thmand trans1ati ngit i ntocode that wil¥execute inMATLAB wil¥be a standard
approach in白 is book.
The searchprocess is startedtypi ca¥1y by tryi ngtoguess th巴 initial desi gnsol uti on.
The desi gner can base this sel ecti on on his experi ence. On sevcm1 occasi ons the
success of opti mi zati onmay hi nge on his abilitytochoose a good initial sol uti on. The
search met hod, even when used consi stent1y, responds di fferent1y to the parti cul ar
probl em出 at is bei ng sol ved. The degree and type of nonl i neari ty may frequent1y
cause 白 e met hod to fail , assurni ng there are no i nconsi stenci es in probl em
formu1ati on. Often,出巴 number of desi gnvari abl es,出 enumber of cons回 ints,as we¥1
as a bad initial guess p1ay a part in unexpected behavi or under standard
i mDl ementati on of the method. Thi s has l ed to the creati on of seveml di ffercnt
techni ques for opti mi zati on. Practi ca1experi ence bas a1so suggested白 at a cl ass of
probl ems respondwel l tocertai na1gori thms.
The mcthods for 出 e search, or it巴 mti ve methods. are current1y the onl y way to
i mpl ement numeri cal sol uti ons tononl i near probl ems, i rrespecti veof thedi sci pl i nein
whi ch these probl ems app回 r. 百 1凶 e probl ems are p1enti fu1 indesi gnopti mi zati on.
opti mal contro1. structural and t1ui d mechani cs. quant umphysi cs. and astronomy.
Whi1eopti mi zati ontechni ques areappl i ed回 all of thenoteddi sci pl i nes. in回 di ti onal
desi gn

at a distan四 sS.. Thi s poi nt satisfies theconstrai nt h.. A 出 irds岡 崎 Ywoul d beto
choosc X. ,・This lies on theconstrai nt boundary of g.. lt is tobe notedthat all 出 向 E
choi ces arenot feasible. Whi ch poi nt is chosenis dcci dedby therules of a particular
al gori thm. Assume poi nt X) is thenext poi nt in出 is di scussi on.
Point X) nowbecomes 恥 newpoint xo・Anew剛 氏 hdircction is de匂 nni n回 .
Poi nt X
is10伺 旬 d. Anothぽ S明 uenceof calculations 郎
• 百 next iterationyields
白 esolution. The essenceof updati ng曲 edesi 伊 duri ngsuc偲 ssiveiterationlies intwo
maj or computati ons, thc sea陀 h directioll 8/ andstepsi zeα. 百 lCchange in由 edesi gn
vector, defin氏 I as .6X, whi chrepresents 出巴 veclor diffcrence, for ex創 npl e,X
- X. , is
obtai nedasαS/. For eachiterationthefol l owi ngs珂 uenceof activitycan be identified:
The above is thebasi c structureof a typical opti mi zati onal gori thm. A compl ete
one will indicate出巴 m加 ner inwhi ch S andαar e computed. Il wil1incJ ude tests,
whi ch will determi nc if 山巴 sol uti on has bcen reached (convergence) , or if the
procedureneeds tobesuspendedor restarted. The diffcrent techni ques tob巴 expl orcd
inthis book mostl y differ in出 eway S is established. Take a l ook at Fi gure1. 7 again.
At X
there are infinite choi ces for the search direction. To reachthe sol uti on in
民 邸 onabl eti meanacceptabl eal gori thmwill trynot tosearchal ongdi recti ons 白 紙 do
not i mprove thecurrent desi gn. 百 lerearemany ways toaccompl i sh出 is strategy.
Determi neα
XM W= Xo+. 6X
Set Xo ←Xnen
GoToSt epO
g 8.5 日
A typical ssarchmethod.
dlamoler (cm)

Li near Pr ogr ammi ng Pr obl em: It was previ ousl ymenti oned白 紙 thesol uti onto
白巴 se probl ems woul d be on the boundary of 白巴 feasible 陀 gi oo (also called the
j 面倒 ibledomai n). InFi gure 1.6,出 e蹴 es andsome of thecons回 ints detenni nethis
regi on. The lalter is di sti ngui shedalsoby出 epoi nts of interscctionof theconstrai nLs
among themsel ves,出 eorigin, and白 eintersectionof thebi ndi ng(active) coostrai nts
wi ththeaxes. These areverti ces or corners of thequadrilateral. Essenti al l ythedesi gn
i mproves by movi ng f romone of thesevertices to thenext one throughi mprovi ng
feasibledesi gns. To do that you have tostart at a feasibl 巴 comer. Thi s will beexpl ored
more fullyinthechapter on linear programmi ng. It is tobe emphasi zed出 atonl ywi th
two vari abl es canwe actual l yseethegeometri c iII ustrationof thetechni que,白紙 is,
theselectionof vertices as thesol uti onconverges.
Thi s part of Chapter 1 has dealt wi ththei ntroducti onof theop白 泌 却 tionprobl em.
Whi leit was approachedf romanengi neeri ngdesi gnperspccti ve, themathemati cal
model , inabstract terms, is not sensitiveto any p町 ticular discipline. A f ewdesi gn
rules wi th resp回 t to specific exampl es were exami ned. TIlestandardmathemati cal
model for opti mi zati on probl ems was established. Two broad cJ asses of probl ems,
initial starting 伊 ess is 白 e poi nt marked xo・ Remember 白 紙 when sol vi ng 山
opti mi zati onprobl em. sucha figuredoes not 目 ist. If it di d,血 enwe caneasilyobtai n
thesol uti onby i nspecti on. 百 lis statement is espcci al l ytrueif 出 erearemore thantwo
variables. In the event th巴 陀 are two vari abl es,自己防 is no reason not to obtai n a
graphi cal sol uti on (Chapter 2). Thc mcthods in opti mi zati on rely on thc dircct
ex凶 pol ati onof theconcepts andi deas devel o戸 dfor two vari abl es toany number of
desi gnvariables. That is thereasonwe will usetwo vari abl es todi scuss anddesi gn
most of theal gori thms andmethods. Moreover, usi ngtwo vari abl es,自己 conceptscan
alsobereinfor四 dusi nggeometry. The desi gnat thepoi nt X
is not feasible. lt is not
useful to di scuss whether it is an opti mal solution. At least one more iteration is
necessary toob凶 nthesolution. Consi der 出 enext sol uti onat X
. Most methods in
opti mi zati onwill attempt tomove toX I by first i denti fyi nga searchdi recti on, S 1・Thi s
is a vector (same di mensi onas desi gnvari abl evcctor X) poi 凶 略 的 mXol oX
I・百 e
actual di stancef romxotoX

call it XoX.. is some mul ti pl eof SI' s町 αSI ・X
agai n is not feasible al tbough it is an i mprovement on xosi nce it satis日 目 白 E
i ncqual i l yconstrai nl gl (l yi nginlheinlerior of 出 econstraint). Al tematel y, usi nglhe
same scal'chdi rccti onS 1, another techni queJ me出 odmay choose for itsnext poi nt , X 1' ,
一 一
n ' …可 ' 一 ー
linear andnonl i near. were i ntroduc凶 . Thegeome汀 ical characteristics of thesol uti on
for thesc probl ems were shown血 rough白巴 graphi cal descri pti onof 出 eprobl ems. An
overvi ewof 出 esearchtechni ques for obtai ni ngthesol uti onto theseprobl ems wω
also iII ustrated. It was a1so menti oned 出 at 白 e sol uti ons were to be obtai ned
numeri cal , か. pri mari l ythroughdigital computati on.
The numeri cal methods usedin山 isareaareiterativemethods. Numeri cal sol uti ons
i nvol ve two components: 白附,叩 al gori thmthat will 巴stablish 出 巴 iterative set of
calculations. andsecond. thetranslationof theal gori thmintocomputer codes. usi ng
aprogrammi ng l ang凶 'ge. Thi s fi ni shedcode is referredto箇 software. Until recentl y
inengi neeri ngeducati on as wel l as practice. thecompu旬 r l anguageof choi cewas
predomi nantl y FORTRANwi tha I i ttlePascal and C. The computati onal envi ronment in
whi ch it was del i veredandpracti cedwお 山 emai ψ' amecomputi ng systems. Today,
由 esceneis verydi fferent. Indi vi dual personal computers 伊 Cs),runni ngWi ndows,
Maci ntosh, LI NUX, or other flavors of UNI X, operati ngsystems arenowthemeans
for techni cal computi ng. Theypossess more power thantheol dmai n合 amecomputers
andtheyarei nexpensi ve. The power of thegraphi cal user interface(GUI ) ,出 eway
出 euser interacts wi 白 thecomputer and山 esoftware, hぉ revol uti oni zedthecreati on
of software today. New programmi ng paradi gms, 抑 制 a 11y 0匂'ecl ori ented
programmi ng, have led to newand i mproved ways to devel op software. Today,
cngi nccri ngsoftwarc is crcatcdby Buchgcncl' al programmi ng l anguagcs Il S C, C++,
J ava, Pascal , andVi sual Basi c. Many softwarevendors devel optheir own extensi ol1s
tothestandardl anguage, provi di nga compl e1e巴 nvi ronment for softwaredev巴l opment
by 出 e user. Thi s makes it very conveni ent for users who are not necessari l y
programmers toget 白 eir j obdone ina reasonab1y short time.
One suchvendor is Ma出 Wo此 s,wi 山 their fl agshi pproduct MA TLABa10ng wi tha
compl ementary col l ecti on of tool boxes 出 at del i ver specific di sci pl i ne-ori ented
i nformati on. In血 i sb∞k,MATLAB is usedexcl usi vel yfor numeri cal support. No pri or
fami l i ari tywi thMATLAB is expected, thoughfamiliaritywi ththePC is requi red. Li ke
all hi gher-1cvcl l anguages, effecti veness wi ththis newprogrammi ng resourcecomes
wi theffort, frequency of use. original work, anda consi stent manner of application.
By thecndof this book, it is hoped the問 ader will bevery fami l i ar wi ththc u




MATLAB is i ntroduccdby MathWorks Inc. , as thel anguagefor techni cal computi ng.
Borrowi ng fromthe descri pti on in 叩 y one of its manual s, MATLAB integrates
∞mputati on, visualizatio民 間 dprogrammi ng inaneasy-to・useenvi ronment where
probl ems andsol ul i ons areexpressedinfarniliar mathemati cal notation. The typical
uses for MATI.. ADi ncl ud巴
• Mat h andcomputati on
・A1gori tl l mdevel opment
• Model i ng, si mul ati on, and prototypi ng
• Data anal ysi s, expl orati on. andvi sual i zati on
・Scientificand engi neeri nggraphi cs
• Appl i cati ondevel opment, i ncl udi ngGUI bui l di ng
The useof MATI..AB has expl odedrecentl yandsohave i ωfeatures. Tbisconti nuous
enhancement makes MATI..AB anideal vehi cl efor expl ori ngprobl ems inall disciplines
that mani pu1ate mathemati cal content . Thi s abi1i ty is mul ti pl i ed by appl i cati on-
specific sol uti ons through tool boxes. To learn more about the latest MATI.. AB. its
features, the tool boxes, col l ecti on of user-supported archi ves, i nformati on about
Usenet activities, other forums, i nformati onabout books, and80on, visit MATLAIl a1
hup: /l www. ma1hworks. com.
1.2.1 Why MA11.AB?
MATLAB is a standard tool for in佐 ' oductory and advanced courses inmathemati cs,
engi neeri ng, andsci enceinmany universities around曲巴 worl d. Ini ndustry, it isa too1
of choi cefor res悶 悶 h,devel opment, andanalysis. In山 is book, MATLAB' S basi c array
el ement is expl oi tedtomani pul atevectors andmatri ces that arenatural tonumeri cal
techni ques. In the next chapter, its powerful vi sual i zati on features are used for
graphi cal opti mi zati on. In出 erest of this book, theinteractivefeaturl 凶 佃 dbuilt-in
support of MATLAB are utilized to translate a1gori thms into functi oni ngcode, in a
fractionof theti meneeded inother l anguages likeC or FORTRAN. Toward白 eend,we
will in町oducetheuseof its Opti mi 回 tionTool box.
Most books on opti mi zati on provi de excell 巴nt coverage on the ~ 回   n i q u s and
a1gori thms, andsome a1soprovi dea pri ntedversi onof thecode. The code is 民 qui 陀 d
tobe usedinmai nf mmEor l egacyenvi ronments that must support a compi l er inthe
l anguage 出 ecode is wri tten, whi ch is predomi nantl y FORTRAN. Each student 出 en
釘 ansl ates theori ntedcode intoa forrnsuitablefor computati on. 百 lenext stepis to
compi l e the code, debug compi l ati on errors, run 出 巴 program,and troubl ωhoot
executi oncrrors beforeuseful 隠 sults canbeobtai ned. An i mportant consi derati onis
白 紙 thestudent is expectedtobeconversant wi 由 theprogrammi ng l anguage. Whi l e
FORTRANwas formal l y taught toengi n田 rs several y回 rs ago,自 is is no l onger true.
』竺 7十 竺 一 一 ー -一一一一一一』
Many electrical engi neeri ngdepartments have startedtoembrace C and C++. PCs are
great for theuseof ready-made appl i cati ons, especi al l ythosewi thGUI , but its usefor
programmi ng inFORTRANhas yet totakeoff.
MATI..ABsuffers veryf ewof thesedi sadvantages. Its∞de iscompact 釦 di ti si nωi ti ve
to l eam. To make probl ems work requires f ewlines of code compared to traditional
l anguages. Insteadof programmi ng f rom白 egroundup, stan伽 rdMA TI..ABpi eces of code
canbe由 民 adedtoge出 erωi mpl ement thenewal gori thm. It avoi ds thes包 ndard鉛 :pru:ヨ飽
∞mpi l ati onandlinks叫 uenceby usi ng叩 inte.叩 reter. Another i mportant 隠 asoni scode
po巾 bility. 官 le∞dewrittenfor 出 ePC versi onof MATI . AB d田 snot have tobechanged
for theMaci ntoshor theUNI Xsystcms. 百 liswill not be trueif system-dependent resources
mωedi n白 ec吋 e. As anilI ustrationof MATI . AB'Sl eami ngagility, bo出 MATI . AB叩 d
appl i edopti mi zati onare∞veredinthisbook , somethi ngverydiffic凶 t旬 a∞ompl i shwi 出,
say, FORTRANandopti mi zati on, or C++叩 dopti mi zati on. Fl 町 出 ermore,恥IA'百..AB, likeC or
FORTRAN, isa gl obal l yrel evant product .
1.2.2 MATLABI nstal l al i onI ssues
Thi s book does not Care if you are runni ngMATLAB on ani ndi vi dual PC or Mac, a
networked PC or Mac, or i ndi vi dual or networked Uni x workstati oJ l s. MATLAB takcs
careof this issu巴. The book assumes you have a worki ng and appropri atel y Ii censed
copy of MATLAB in the machi ne in 合ont of you. On your personal machi ne, for
installingMATI . AB, fol l owthe i nstructi ons that accompani 巴dthebox. Typi cal l y 白 e
systemadmi ni strator handl es network installations.
Thi s book uses MATt. AB Versi on5.2(until Chapter 4) and Versi on5.3(山 erest).
Versi on 6 has been shi ppi ng si ncethefall of 2000. MA礼 ABversi on5.0and above
shoul dwork. If MATI . ABhas changed any built-incommands innewer versi ons 血 佃
theone凶 edin thebook, it shoul dprovi dei nformati onon theavailabilityof newer
versi on of 山 commands (or l hrowan error, a rare case). The MATLAB code and
exampl es were created on an Inlel Penti umPro 200 runni ng the Wi ndows NT 4
operati ngsysl em. The earlier codes were al sotestedon a DEC Al pha server runni ng
OSFl V4. The MATI..ABversi onon theserver was 5. 1.
In organi zati ons and instilutions MATt. AB software is usual l y l i censed on a
subscri pti on basis. Users therefore hav巴 the 1atest versi on of the software in these
situations. Insucha casethenewversi on(at 出巴 ti methebook is first publ i shed) , will
beVersi on6 (or Rel easeR 12). Thi s book is basedon Versi on5 (Rel easeRI 1). 百 le
code胡 d由巳 programmi ngi nstructi ons inthebook shou1dexecutewi thoUl probl em
in the newversi on. In the newRel ease R12, on the PC p1atform, the defaul t
appearanceof MATI..ABis a composi te of several wi ndows presenti ngdi fferent pi eces
of i nformati on. Thi s is shown inFi gure1.8. Thi s c回 becustomi zedthroughthemenu
b釘 by Vi ew→DesktopLayout →Command Wi ndowOnl y, whi ch al socoπωponds
to 山 e previ ous 問 1ease. It is shown in Fi gure 1. 9. 百 lis vi ewis si mpl e and less
di stracti ngfor getti ngfami l i ar wi 出 MATl ..AB. Note白 紙 白 eMATt. AB command prompt
is >>. Thi s wi ndowdel i vers standard operati ng systemcommands for savi ng,
pri nti ng, openi ng files, and so on, l hrough menus and a command bar. In order to
T信 配
1.2 I NTRODucnONTOMATl . AB 27
F1gure1.8 MATLAB openi ngview.
createfiles (or code) another graphi ca1wi ndowis n巴eded. Thi s is 山 eMATLABM・file
edi tor/debugger wi ndowand is shown in Fi gu偲1. 10. The edi tor uses co10r for
enhanci ngcode readability. Each of t 山hes弱ewi ndows spawn addi ti ona1di a10gboxes (if
n附 e回c側町均) f i伽o ωぽ r日 川 . s 民刷削則 c“削ttωti同n 略1唱gfi蜘e伺側 a瓜tu附 re 附e邸 sd 削u 町叫Iげ 川ri 叫ir
workspace browser.
Whi lethisisnot somuch asaninstallationissue, it wou1d be appropri al elodi scuss山 e
way仙 胸 k pl aI 1s towri 包 andexec雌 MATI . ABC峨 Gen叫 y,出 erearetwo ways to
work明 白 MATI . AB: i nteracti vel yandthroゆ 附 'pts. Ininteractivemode. 自 己 ∞r nn1ands
areente陀 d田 d郎 氏 utedone at a ti meintheMATt. AB cOrnn1and wi ndow. Once出巴 sess10n
is over andMATt. AB exited, thereis no i nformati on1也 tai nedon尚早 set of commands 白 紙
were used and what worked (unless a choi ce was made to save the session). Large
s詑ea叫 ue叩n隠 s of cOrnn1aI 1ds areusual l y di 血 cu 凶1uω.ok伐 P 釘回ac依k of 泊 加 i加n配E低臨 erac凶Uv刊巴 se白ss劉10n.
入ぷ…4ゐo伽 o∞叫 p 飢凶t 首 帥 i
白印畠 fileinMATI . AB・Thi siscalleda script m-ftle. Most MATLAB fil 郎 町 田 lledm-ftles. The
m is a meextBI ESi onusual l ymserved for MA-I -LAB. Any code changEsmemade thmush
山 eeditor. thechanges to出 efilesaved, and山 e fi l cexecu飽 dinMATt. AB again. Ex巴伐cu凶』胤tin 昭
M肱 ATt.A凶叫 Btl 剛 ghs氏c ふ勾叩 d均印p 似川州 t凶巾白 m伽le 加e ω削 帥 s叫山i お切 s叩pr叫e吋do 叩 側 m 削1泊 lin
tωore 芭 vert tωoi nteracti vemode any ti me.
1.2.3 Usi ng MATLABthe Fi rst Ti me
E 喝g
MA T ωwi¥l be used inte削 i ve1y in 出 is section. The au.thor very 叩 n斜y
recornn1ends a hands-on approachtounderstandi ngMATt. AB・This means typmg tt1e
codes yourself. It a1soinc1udes undersl andi ngthesyntax 巴rrors,anddebuggi ng and
百 i s i s a Cl asstoomLi cense fOt i nsttucti onal use onl y.
Reseatch and com且 etci al use i s ptohi b1ted.
To qet statted, sel ect " HATLAB Hel p" f tom.the Hel p menu.
Fl gure1.9 MATLAB Command wl ndow.
Opti m1zati on 官 1ほ 1    
'iI J ohn
も Dt. P. VenkataJ ; amenl
F1gure 1.10 MaUab Edi tor/Debugger wi ndow.
recti ちring白 ecode. The author is not aware of a si ngl eexampl e where programmi ng
was l earnedwi thout thereader typi ng叩 de. l n出 is b∞k, MATLAB code segments are
couri er font wi th 出 e bol dstyl e lI sedfor emphasi zi ng commands or other pi eces of
i nformati on. Anythi ng el se is recommendati ons, suggesti ons, or exerci ses. The
pl acehol der for command is initalic.
さ l
Bef or e We Start: The fo¥l owi ng punctuati on 卸 d speci al characters are worth
noti ng:
者 A¥l text after the % si gn is consi dered a
comment. MATLAB i gnores anythi ng to
the ri ght of the % si gn. Comment s are
used l i beral l y i n this book to i ndi cate
why' s and what' s of command or code
usage. Do not overl ook them. You do not
have tocode t hemin, however, probabl y
t wo weeks later you wi¥l not r emember
why you devel oped that parti cul ar pi ece
of code.
Thi s is thedefaul t MATLAB pr ompt.
A semi col on at theend of a line prevents
MA礼 ABf romechoi ng thei nformati onyou
enter on thescreen.
A comma will cause the i nformati on to




echo. Defaul t is a comma.
A successi onof threeperi ods at theend of
the line i nf orms MATLAB 白 紙 code will
conti nue ωt he next line. You cannot split
a vari abl e name across t wo l i nes. You
cannot conti nue a comment on another
Ii n巴.
You can s 試to 叩'p MATLAB executi on and get
back t 出h巴 c ∞omm!削md pr omp 戸t b 句y typi ng ^c
(C佐凶l- C)一b句y h 加01凶di n 昭gdown
tωog 酔e出 eぽr.
WilI di spl ay i nformati on on the vari ous
ways thecommand canbe used. Thi s is the
qui ckest way touse help.
Opens a hel p text wi ndowthat provi des
mor e i nformati on on all of the MATLAB
resources i nstal l edon your system.
Provi des hel pusi nga browser wi ndow.


>> hel p command_name
>> hel pwi n
>> hel pdesk
For thoseof you programmi ng for thefirst ti me, theequal tosi gn(=) has a very speci al
meani ng inall programmi ng l anguages, incJ udi ng MATLAB. It is cal l edtheassi gnment
operal or. 百 le vari abl eon thc I cft-handsi deof thesi gnis assi gnedtheval ue of 山 e
ri ght-handside. Thc actual eqllal l Ooperati oninMATLAB is usual l y accompl i shed by
a doubl e cqual tosi gn(==).
Some Addi ti onsl Feat ures
• MA礼 AB is case sensi ti ve. An a is di fferent than A. Thi s is di ffi cul t for
persons who arc uscd to FORTRAN. Al l bui l t-i n MATLAB commands arc in
l owercase.
• MATLAB does not necd a typedefi ni ti onor a di mensi onstatement toi ntroduce
vari abl es. It automati cal l y cr四 tes one on first encounter incode. The typeis
assi gnedincontext
• Vari abl 巴 names start wi tha letter and contai nup to31cbaractcrs (onl y Ictt巴rs,
digits, and underscorc).
• MATLAB uscs some built-invari abl enames. Avoi d usi ngbuilt-invari abl enamω.
• Al l numbers arc storcd i nternal l y usi ng the l ong format spcci fi cd by I EEE
f1oati ng-poi nt standard. Thesenumbers have roughl y a preci si onof 16deci mal
digits. They range roughl y between 10E- 308 and 10E+308. Howeve丸 山 巴 yare
di spl ayeddi ffercntl ydependi ng on thecontext.
• MATLAB uses conventi onal deci mal notati on for numbers usi ng deci mal
poi nts and l eadi ng si gns opti onal l y, for exampl e, 1, - 9, +9. 0, 0. 001,
99. 9999.
• Sci enti fi c notati onis cxprcssedwi ththeletter e, for cxampl e, 2. 0e-03, 1. 07e23,
ー1. 732e+03.
• l magi nary numbers useei ther i or j ωa suffi x, for exampl e, li , -3.l 4j , 3e5i .
Operators: The fol l owi ngarc theari thmeti c operators inMATLAB.
+ Addi ti on(wben addi ng matri ccs/arrays sizemust match)
Sub釘 acti on(same as above)
* Mul ti pl i cati on (the subscri pts of arrays must be consi stent when
mul ti pl yi ngthem)
/ Di vi si on
Compl cx co吋 ugat巴 transpose(山 0 町 aytranspose)
Inthecaseof arrays, each of theseoperators can be used wi 白 aperi odprefi xedto白 e
operator, for cxampl e, ( . * ) or ( • ^ ) or ( 1) . 明白 has a speci al meani ng inMATLAB. It
i mp1 i es 巴 l ement-by-el ement operati on. It is uscful for qui ck computati on. It has no
rel evanceinmathemati cs or anywhere else. We wiJ1usc it a lot inthenext chapter for
gClI erati ngdatafor graphi cal op出 凶 zati on.
An I nteractl veSessl on: S也氏 MA礼 AB. On恥 PCprcss theStart button. Press the
Pr ogr ambutton. Prcss thc MATLAB 針。 ' gramGroup. Press MA TLAB. The MATLAB
Command wi ndowopens up and after somc i ntroductorymessages白 E悶 will be a MATLAB
prompti ndi 回 悦 dby doubl ef orward arrow. 百 lis is叩 i ndi cati on白 紙 MATLABis ready to
aαeptyour ∞mmands. On thenetworked s也 tions,typi ngmatJab at thewi ndowprompt
shoul ddo thesame. Note that theMATLAB programmust be inthepa出 as def med in出 e
.Iogin or. ωh代 file. Ask thelabassistant for help. Thωe紅 巳 standardUNI X proccd町 民 ・
凶 t 陀 知 rnat theend of 出 elincor befo問 山 ecomment . The commcnts rclatc towhat is
bei ngtyped加 dwhat isdi spl ayedintheMATLAB command wi ndow. FI田 1frccω 町 your
own variatiosS. Infact , tounde岡 田 dandrei nf o詑 巴 白 巴 commands,it is問 commended白 紙
you make up your own exampl es often.
>>a 1. 0; b 2. 0; c 3. 0, d 4. 0; e 5. 0
> > 苦 why di d onl y c and e echo on t he scr een?
>> who 者 l i st s al l t he var i abl es i n t he wor kspace
>> a 者 gi ves t he val ue st or ed i n a
>>A 1. 5; 告 var i abl e A
>> a, A % case mat t er s
>>one a; t wo b; t hr ee C;
> > 苦 assi gni ng val ues t o new var i abl es
>>f our d; f i ve e; si x pi ; 者 val ue of pi avai l abl e
>>A1 [a b c ;' d e f] 者 A1 i s a 2 by 3 mat r i x
>> % space or comma separ at es col umns
>> % semi - col on separ at es r ows
>>A1( 2,2) 者 accesses t he Mat r i x el ement on t he
>> % second r ow and second col umn
>> si ze (A1l も gi ves you t he si ze of t he mat r i x
> > 者 ( r ow, col umI l sl
>>AA1 si ze( A1l 者 What shoul d happen her e?
> > 者 f r ompr evi ous sヒat ement t he si ze of A1
> > も cont ai ns t wo number s or gani zed as a r ow
> > 告 mat r i x. Thi s i s assi gned t o AA1
>> si ze( AA1l も AA1 i s a one by t wo mat r i x
>> A1' 者 t hi s t r ansposes t he mat r i x A1
>>B1 A1' も t he t r anspose of mat r i x A1
> > 者 i s assi gned t o B1. B1 i s a t hr ee by t wo mat r i x
>> C1 ' " A1 * B1 者 Si nce A1 and B1 ar e mat r i ces t hi s
> > 者 i 8 a mat r i x mul t i pl i cat i on
> > 巷 Shoul d t hi s mul t i pl i cat i on be al l owed?
> > も consi der mat r i x mul t i pl i cat i on i n C or
>> % For t r an
> > 者 t he power of MATLAB der i ves f r omi t s
> > も abi l i t y t o handl e mat r i ces l i ke number s
>>C2 81 * A1 も How about t hi s?
>> C1 * C2 も What about t hi s?
>> % r ead t he er r or message
> > 者 i t i s qui t e i nf or mat i ve
>>D1 [1 2]' 苦 D1 i s a col umn vect or
>>C3 [C1 D1] 者 C1 i s augment ed by an ext r a col umn
>>C3 [C3 C2( 3, : ) ] 告 Not e i s an assi gnment
> > 告 means do t he r i ght hand si de and over wr i t e t he
> > 告 ol d i nf or mat i on i n C3 wi t h t he r esul t
> > 告 of t he r i ght
> > 告 hand si de cal cul at i on
> > 告 On t he r i ght you ar e addi ng a r ow t o cur r ent
> > 者 mat r i x C3. Thi s r ow has t he val ue of t he t hi r d
> > 告 r ow of C2 - Not i ce t he pr ocedur e of
> > 奄 i dent i f yi ng t he t hi r d r ow. The col on
>> % r epr esent s al l t he col umns
>>C4 C2 * C3 も per mi ssi bl e mul t i pl i cat i on
> > 告 Not e t he pr esence of a scal i ng f act or
> > 者 i n t he di spl ayed out put
>>CS C2 * C3 告 seems t o mul t i pl yl
> > 告 I s t her e a di f f er ence bet ween C4 and CS?
>> % The * r epr esent s t he pr oduct of each el ement
> > 者 of C2 mul t i pl i ed wi t h t he cor r espondi ng
> > 者 el ement of C3
>>C6 i nver se( C2) % f i nd t he i nver se of C2
> > 告 appar ent l y i nver se i s not a command i n MATLAB
> > 者 i f command name i s known i t i s easy t o obヒai n
> > 者 hel p
>> l ookf or i nver se 告 t hi s command wi l l f i nd al l f i l es
> > 者 wher e i t comes acr oss t he wor d "i nver se" i n
> > 者 t he i ni t i al comment l i nes
> > 者 The command we need appear s t o be I NV whi ch
> > 告 says I nver se of a Mat r i x
>> % The act ual command i s i n l ower case. To f i nd
> > 告 out how t o use i t - Now
>> hel p i nv % shows how t o use t he command
>> i nv( C2) 者 i nver se of C2
>> f or i ' " 1: 20
f ( i ) i "' 2;
> > 告 Thi s i s an exampl e of a f or l oop
> > 者 t he i ndex r anges f r om1 t o 20 i n st eps of
> > 者 1( def aul t )
> > 者 t he l oop i s t er mi nat ed wi t h "end"
> > 者 t he pr ompt does not appear unt i l "end" i s
> > 者 ent er ed
>>pl ot ( si n( 0. 01*f ) ,cos( 0. 03合 f) )
>>xl abel ( ' si n( O. Ol *f ) ' ) 者 st r i ngs appear i n si ngl e
> > 毛 quot es
>>yl abel ( ' cos( 0. 03*f ) ' )
>> l egend ( ' Exampl e' )
>> t i t l e ( ' A Pl ot Exampl e' )
>> gr i d
> > も The pr evi ous set of commands wi l l cr eat e pl ot
> > 告 l abel axes , wr i t e a l egend, t i t l e and gr i d t he
> > 告 pl ot
>> exi t も f i ni shed wi ヒh MATLAB
Thi s compl etes thefirst sessi onwi thMATLAB. Addi ti ona1commands and features will
be encounteredthroughout thebook. In出 is sessi on, it is evi dent 白 紙 MA礼 . ABal¥ows
easy mani pul ati onof matri ces, defi ni tel yinrelationtoother programmi ng l anguages.
Pl otti ngis not difficult either. Theseadvantages ar芭 qui tesubstanti al inthesubj ect of
opti mi zati on. 1n 出巴 next sessi on, we wil1 use the edi tor 10 I1ccompl i sh 出 e same
throughscripts. Thi s sessi oni ntroduced
• MATLAB Command wi ndowand Workspace
・Vari abl eassi gnment
• Basi c matri x operati ons
• Accessi ngrows and col umns
• Suppressi ngechoes
• who, i nversecommands
• . * mul ti pl i cati on
• Basi c pl otti ngcommands
1.2.4 Usi ng t he Edi tor
Inthis secti on, we will usetheeditor toα 回 te叩 druna MATLAB a script file. Nor ma1ly,
出 eeditor isusedtogeneratetwo ki nds of MA百 . ABfiles. 百 1凶 efilesaretermedscriptftles
andfuncti onftl es. AI 出 oughbo出 ofth巴sefiI es con凶 nMATLAB commands like山eones
we have used, thesecondtype of fiI es needs tobe organi zedina speci fi edformat . Bo曲
file ty戸 s shoul dhave theextensi on .m. Al though thesefiles areASCI I text fi I 凶,山 e
generi c . mshoul dbe凶 edbeca凶 eMATLABs国 民 hes for 也 i sex蜘 si on.τhi sextensi onis
山首 quet oMA百. AB. 百1c script filecontai ns a Ii st of MA TLABcommands 白 紙 areexe印 隠 d
insequence. This is different 告。 mtheinteractivesessi onof theprevi ous sectionwhere
MAτ 1. AB悶 卯 ndedto回 chconunand i mmedi atel y. The script fileis more useful when
therear巴 many conun叩 ds 白 紙 needtobe ex釘 utedtoac∞mpl i shsome obj ecti ve, like
runni ngan opti mi zati ontechni que. It is i mportant toremember that MATLAB aI l ows you
toswi tchback ωi nteracti vemode at any ti me by just typi ngcommands in白 eworkspace
wi ndowlikein曲 ep問 vi ous secti on.
MATLAB provi des an edi tor for creati ngthesefiles on thePC pl atform. Thi s edi tor
uses col or toi denti fyMATLAB statements and el ements. It provi des 白 ecurrent vaI ues
of the vari abl es (after 白 eyareavai l abl eintheworkspace) when themouse is over
the vari abl e name intheedi tor. Thel 官 邸 官 t woways to access 山 eedi tor through the
MATLAB Command wi ndowon the PC. 5tart MATLAB. Thi s wi l l open a MATLAB
Command or Workspace wi ndow. Inthis wi ndowtheedi tor can be startedby usi ng
the menu or 血 e tool bar. On the Fi l e menu, cl i ck on New and choose M- fiI c.
Al tematel y, cl i ckon theI eftmost i conon thetool bar (出 etool ti preads New Fi l e). The
i confor theedi tor can aI sobe pl aced on thedesktop, inwhi ch case theedi tor can be
startedby doubl e-cl i cki ngthe i con. In this event, a MATl AB Command wi ndowwi l l
not be opened. The edi tor provi des its own wi ndowfor enteri ng出巴 scri pt statements.
At this poi nt we areready touse theedi tor. Make sureyou即 ad出巴 comment s and
understand t hem, as theseprocedu問 s wi l l be used often. Usi ng theedi tor i mpl i es we
wi l l be worki ng wi thm-fi l es.
The commands ar巴 出 esame as in1h巴 i nteracti vesessi onexcept thereis no MATLAB
prompt prefi xi ngtheexpressi ons. To execute thesecommands you wiIl have tosave
t hemtoafi l 巴. Let us call thefi l escri ptl .m. The. mextensi onneed not be typed i f you
are usi ng 由巳 MATLAB edi tor. You can save the file usi ng the Save or Save As
command f rommost edi tors. It is i mport加 t toknowthe full pa曲 to白 is file. Let us
assume thepathfor 出 efileis C: ¥OpCbook¥Chl ¥scri ptl .m. Not e出 創 出 epa白 hereis
speci fi edas a PC pathdescri pti on. The reason we need山 is i nformati onis toi nf orm
MATLAB where tofi nd白 efile. We do出 is in白 eMATLAB Command wi ndow.
Thi s i mpl i es we shoul d have白 eMATLAB Command wi ndowopen. On PCs we start
MATLAB through the i cons. I nthis book we wi l l accompl i sh most tasks by typi ng in
theCommand wi ndowor through programmi ng. l nmany i nstances 山 ereareal temate
ways usi ng menu i tems or 白 e tool bar. Thi s is left to the reader to experi ment and
di scover. In the MATLAB Command wi ndowus巴 山 e addpat h command to i nf orm
MATLAB of the l ocati onof thefile:
>> addpat h C: ¥Opt _book¥Ch1¥
The scri pt th81wi l l be createdand saved inSCT, 伊 tl .mcan be runby typi ng(notethat
出 eextensi on is omi tted)
>> scr i pt l
To understand and associ atewi th theprograrnmi ng concepts embedded in. thescri pt,
parti cul arl y for persons wi thIi mi ted prograrnmi ng experi ence, it is r ecommended to
run thescri pt after a bl ock of statements have been wri ttenra山 er thantypi ng山 efile
inits enti rety. Pri or torunni ng thescri pt you wi l l have tosave thescri pt each ti me 50
白 紙 the changes in 由巳f i J e are recorded and the changes are current. Another
recommendati on is todel i beratel ymi sり'pesome statements and attempt todebug the
error generatedby白 eMATLAs debugger duri ng回 目 uti on.
Creati ng the Scr, 伊 t M-・.flle (The fol l owi ngwi l l be typedl sav吋 ina fil 巴.)
も exampl e of usi ng scr i pt
A1 [1 2 31;
A2 [4 5 61;
も t he commands not t er mi nat ed wi t h semi - col on wi l l
も di spl ay i nf or mat i on on t he scr een
A = [ A1; A2]
B [A1' A2' ]
C A* B
も now r e- cr eat e t he mat r i x and per f or mmat r i x
者 mul t i pl i cat i on as i n ot her pr ogr ammi ng l anguages
者 exampl e of f or l oop
f or i 1 3 者 var i abl e i r anges f r om1 t o 3 i n
も st eps of 1 ( def aul t )
a1 (1, i ) i ;
end 告 l oops mus t be cl osed wi t h end
f or i 6: - 1: 4 も not e l oop i s decr ement ed
a2( 1,i - 3) i ; も f i l l i ng vect or f r omr ear
者 cr eat i ng mat r i x A and B ( cal l ed AA and BB her e)
f or i 1: 3
AA( l , i ) a1 (1, i ) ; も assi gn a1 t o AA.
AA( 2, i ) a2 (1, i ) ;
BB (i , 1) a1 (1, i ) ;
BB ( i , 2 ) a2 (1, i ) ;
も t he same can be accompl i shed b y AA (1,: ) a1
者 wi t hout t he f or l oop
AA 巷 pr i nt t he val ue of AA i n t he wi ndow
who も l i st al l t he var i abl es i n t he wor kspace
者 consi der code f or Mat r i x mul t i pl i cat i on
% whi ch Mat l ab does s o easi l y
告 mul t i pl y t wo mat r i ces ( col umn of f i r st maヒr i x must
者 mat ch r ow of second mat r i x)
szAA si ze( AA) も si ze of AA
szBB si ze (BB) ;
i f ( szAA( l ,2) szBB( l ,l ))
告 onl y i n col umn of AA mat ch t he r ows of BB
f or i l : szAA( l ,l )
f or j 1: szBB( l ,2)
CC ( i , j ) 0 . 0 ; 苦 i ni t i al i ze val ue t o zer o
f or k 1: szAA( l ,2)
e nd
CC( i ,j ) c CC( i ,j ) + AA( i ,k) *BB( k,j ) ;
も k - l oop
も j - l oop
end 者
苦 i - l oop
i f - l oop
者 Not e t he power of M   T ~ B der i ves f r omi t s abi l i t y t o
も handl e mat r i ces ver y easi l y
者 t hi s compl et es t he scr i pt sessi on
Save theabove fi) e (scri ptl. m). Add thedi rectory totheMATLAB path凶 i ndi cated
before. Run thescript fileby typi ngscri ptJ at thecommand pr ompt . Thc commands
shoul d all execute and you shoul dfmal l y seetheMATLAB prompt inthc Command
wi ndow.
Note: You can al ways rcvert to the i ntcracti ve mode by di rectl y enteri ng
commandsi n出 eMATLAB wi ndowafter theprompt . IntheCommand wi ndow:
>> who
>> cl ear C % di scar ds t he var i abl e C f r omt he wor kspace
> > 者 use wi t h caut i on. Val ues cannot be r ecover ed
>> hel p cl ear
>> exi t
This sessi onil1ustrated:
• Usc of th巴 edi tor
・Creati nga script
• Runni ng a scIi pt
• Error debuggi ng (recommcndcd activity)
・Programmi ngconcepts
. Loop constructs , if and for l oops
・ L oopvari abl eand i ncrements
・Array access

• Cl ear stal ement
1.2.5 Creatl ng a Code Snl ppet
Inthis secti on, we will exami ne the0山 er typc of m-fi l 巴 whi chis cal1ed thefuncti on
m-fi l e. For those fami l i ar wi l h other programmi ng l anguages Ii ke C, J ava, or
F'ORTRAN,出 esefiles represent functi ons or subrouti nes. They arepri mari l y uscdt'O
handl e s'Ome speci fi c types of cal cul ati ons. Thcy al sopr'Ovi dea way for themodul ar
devel opment of code as wel l ぉ codereuse. These code modul es ar, 巴 usedby bei ng
cal l cdor refcrredinother secti ons of thecodc, saythrougha script filewe l ookedat
earlier. The code that calls thc functi onm-fi l eiscal l edthecal l i ngprogram/code. The
essenti al paramctcrs indcvel 'Opi ngthefuncti onm-fi l c arc (1) what i npu1is necessary
for the caI cul ati ons, (2) what speci fi c cal cul ati ons must take pl ace, and (3) what
i nformati onmust be retumed tothecal l i ngpr'Ogram. MATLAB requi res theSl ructure
of 出 efuncti onm・filetofol l owa prescri bedf ormat.
We will use1heedi 10r 10devel 'OP a func1i onm-fi l etha1wiI1pcrforma pol ynomi al
curvefi t . II requi res a set of 砂 data,represen1i nga curve that needs tobe fi1, 10ge1her
wi 出 the order of thepol ynomi al to be fi t. Thi s exerci sc is cal l cd curve fiuing. In
Chaptcr 6 such a probl emwill be i den1i fi cd as a probl emin unconstrai ned
opti mi zati on. For now, the cal cul a1i ons necessary to accompl i sh the exerci se are
consi deredknown. It i nvol ves sol vi nga linear equati onwi th出 enomtal matrl x and a
ri ght-h佃 dvcctor obtai nedusi ngthedatapoints.τ 'he output f romthem-fi l ewill be
山 ecoe鉦 icients representi ng白 epol ynorni al .
Before we start todevel opthecode, thefirst lineof this filemust be f ormatted as
speci fi edby MATLAB. Inthefirst line, thefirst w'Ordstartirig合 um出 efirst col umn is
山 c wordfuncti on. It is fol l owedby theset of rel umparameters (retumval ). Next,
山 ename (mypol yfi t) 'Of thefuncti onwi ththeparameters passedtothc functi onwi thi n
p抗 仙 eses (XY , N). 明 lefilemust besavedas name. m(mypol yfi t.m). The comme略
be1ween thefirs1lineand thefirs1executabl c statement wiI1appc釘 if you 1ypehel p
lI ame (hel p mypol yfi t) in thc Command wi ndow. The reason for thc narne
mypol yfi t . m is that MATLAB has a bui1t司 infunclI on pol yfi t. Open theedi tor 10create
thefilecontai ni ngthefol l owi ngi nformati on:
f unct i on r et ur nval mypol yf i t ( XY, N)

も These comment s wi l l appear when t he user t ypes
者 hel p mypol yf i t i n t he Command wi ndow
事 Thi s space i s i nt ended t o i nf or mt he user how t o
も i nt er act wi t h t he pr ogr am, what i t does
苦 what ar e t he i nput and out put par amet er s
も Least squar e er r or f i t of pol ynomi al of or der N
者 x y - Dat a f ound i n XY
告 r et ur ns t he vect or of coef f i ci ent s st ar t i ng f r om
% t he const ant t er m
. i : j ' J S     ょ に 吋 よ , I ¥ , , , , , , 刊 夜 長 翻 最 麗 革 担 躍 盟
司 . .
  ・ 日 九 り   , , : 1 ・'.
end 者 end i 100p
も i f t he x- poi nt s ar e di st i nct t hen i nver se i s not a
も pr ob1em

まor i 1: N+1
a( i ) = 0. 0; も i ni t i a1i ze t he coef f i ci ent t o zer o
r et ur nva1 i nv( A) *b' ;
Save thefileas mypol yfu. m. To usethefuncti onwe will need tocreatesome xydata
阻 dthcncall themypol yfi t functi on. S凶 MA1LABin白 c di rectory伽, t mypol yfit. m
resi des in. or add thedi rectory to thepath. IntheM A11. AB Command wi ndow: The
fol1owi ng cod巴 is typedinthecommand wi ndow.


























sz si ze (XY) ;
NDATA sZ( l ,l ) ; 者 number of dat a poi nt s - r ows 0王
者 xy mat r i x
i f NDATA == 0
f pr i nt f ( ' Ther e i s no dat a t o f i t ' ) ;
r eヒur nva1 a; 者 zer o va1ue r et ur ned
r et ur n; % r et ur n back t o ca11i ng pr ogr am
>>coef f mypol yf i t ( XY1,3)
> > 告 a cubi c po1ynomi a1 was de1i ber at e1y cr eat ed t o
>> % check t he r esul t s. You shou1d get back
> > 者 t he coef f i ci ent s you used t o gener at e t he cur ve
> > 者 t hi s i s a good t est of t he pr ogr am
> > 者 Let us cr eat e anot her examp1e
i f NDATA < 2* N
f pri nt f ' ( , Too f ew dat a poi nt s f or f i t' )
r et ur nva1 a;
r eヒur n
>> XY2 ( : ,1) XYl ( : ,1) ; も same f i r st co1umn
>> XY2 ( : ,2) 2,0 + exp ( XY2 ( : ,1) )
> > 者 not e t he power of Mat 1ab - a vect or f i 11
> > 者 a new set of xy- dat a
The pr ocessi ng st ar t s her e.
The coef f i ci ent s ar e obt ai ned as sol ut i on t o t he
Li near A1gebr a pr ob1em [A] [c] [b]
Mat r i x [A] i s t he Nor ma1 Mat r i x whi ch cont ai ns
t he sumof t he power s of t he i ndependent var i ab1e
[A] i s symmet r i c


>>coef f 1 mypol yf i t ( XY2,
>> hel p mypol y王i t
> > 者 you shou1d see t he st at ement s you set up
> > も I n t he next exer ci se we wi 11 cr eat e a scr i pt
> > も f i 1e t hat wi 11 r un a pr ogr amf or po1ynomi a1
> > 者 cur ve- f i t t i ng we wi 11 save XY2 so t hat i t can
> > 者 be used agai n
agai n 者 cubi c po1y. 3)
も 100p over al 1 dat a poi nt s
者 var i ab1e i s "11" (e1) (e工)
XY( 11,2) *XY( 11,1) ^( i - 1) ;
f or i 1: N+1;
b( i ) 0. 0;
f or 11 l : NDATA;
b( i ) +
も 100p
b( i )
save C: ¥Opt _book¥Chl ¥XY2. dat - asci i ・doub1e
者 t hi s wi 11 save t he f i 1e as an asci i t ext f i 1e
も wi t h doub1e pr eci si on va1ues

Thi s conc1udes 白 e exerci se where a code s凶 p阿 W凶 written to 回 lc叫 ate 出 E
C∞fficients of thepol ynooual usedtofit a curv巴 tosome砂 data. 百 etypeof filc is thc
f or j 1: N+1;
i f j >= i 者 ca1cu1at i ng upper di agona1 t er ms
power (i - 1) + (j ・ 1) ;
A( i ,j ) 0. 0 者 i ni t i al i ze
f or k l : NDATA; 告 sumover dat a poi nt s
A( i ,j ) = A( i ,j ) + XY( k,l ) ^power ;
end 苦 k l oop
end 告 c10se i f st at ement
Aけ, i ) = A( i ,j ) 昔 exp10i t i ng Mat r i x symmet r y
end 苦 end j 100p
fi mcti onm-fi le. It n民 地 ωbeusedina certainway. 百 lecode was 臨 tedusi nga cubi c
pol ynomi al . Nonpol ynomi al datawcre a1sotested. The datawere savedfor later use.
1. 2. 6 Creatl ng a pr ogr am
h 出 is secti ona program白 紙 will 陀 ad砂 dat a , curvefit thedatausi nga pol ynomi al , and
∞mpare恥 original and fi 臨 dda泊 伊 pru回 llywill bedevel oped. The也 tawill beread
usi nga User I n低出国( UI ) wi ndowand a di al ogbox. The code will be devel opedぉ a
script filc. This gi ves us anopportuni ty torevisit most of thecode we used回 rlier for
問 i nforcement . Wc will alsousethcfuncti onm- fiI e createdin由巳 previ ous exercise.
百 ler芭 出 窓 several ways for you or theusers tointeract wi ththecode you devel op. 百 1巴
basi c method is toprompt users for i nformati onat 恥 prompt intheCommand wi ndow.
司 自 is 出 equi ckest . Thi s is probabl y what you will usewhen devel opi ngthecode. Once
the∞dehas beentested, depencl i ngon useful ness it mi ght berel evant ωconsi der usi ng
more sophi sti catedcustomeI ements I i ke i nput boxes and filesel ecti onboxes. 百 lis book
will conti nuetousetheseel ements throughout as appropri ate. Whilethei nput el ements
us叫 in山 is code arcncwcommands, therest of theprogramwill mostl y usecommands
伽 t havc bccni ntrodu田 d回 rlier. Ins叫 uential order theevcnl s in出 sprogramare: (1) to
readthe砂 datasavedearlier usi nga filcscl ecti onbox, (2) toread曲 eorder of fit usi ng
an Input di al ogbox, (3) tousethemypol yfi t functi ondcvel opedinthelast sectionto
obta泊 thec田 fficients,(4) toobtai n白 ec∞必 ina凶 of thefittcdc町 ve,(5) tograprucal l y
compare血巴 original 叩 dfitt凶 dat a , and (6)ω 氏 陣 ton白 efitteda∞uracyon曲 efi 思 m
itsel 五 百 巴 newscript filewill be cal l edpr og...J1j 弘 m.
Start thecdi tor tocreatethefilc cal l edp' 可
告 pr ogr amf or f i t t i ng a po1ynomi a1 cur ve t o x y dat a
者 f r om
告 App1i ed Opt i mi zat i on usi ng Mat 1ab
者 Dr . P. Venkat ar aman

者 Chapt er 1, Sect i on 1. 2. 6
者 The pr ogr am100ks f or a f i 1e wi t h t wo co1umn asci i
者 dat a wi t h ext ensi on . dat . The or der of t he cur ve i s
者 obt ai ned f r omuser . The or i gi na1 and f i t t ed dat a
苦 ar e compar ed wi t h r e1evant i nf or mat i on di sp1ayed on
者 t he same f i gur e. The pr ogr amdemonst r at es t he use
も of t he f i 1e se1ect i on box, an i nput di a10g box,
者 cr eat i ng speci a1 t ext st r i ngs and di sp1ayi ng t hem
[ f i 1e,pat h] =ui get f i 1e( ' *. dat ' ,' A11 Fi 1es' , 200,200) ;
者 ui get f i 1e opens a f i 1e se1ect i on box
者 checkout he1p ui get f i 1e
者 t he st r i ng var i abl e f i 1e wi l l hol d t he f i l ename
苦 t he st r i 叩 var i ab1e pat h wi 11 have t he pat h
も i nf or mat i on t he def au1t di r ect or y poi nt ed wi 11
者 di f f er dependi ng on t he p1at f or m
i f i sst r ( f i l e) 者 i f a f i 1e i s se1ect ed
10adpat hf i 1e [ ' l oad' ,pat h f i 1e] ;
者 10adpat hf i 1e i s a st ど i ng var i ab1e concat enat ed
も wi t h t hr ee st r i ngs "10ad " , pat h and f i 1e
者 not e t he space af t er 10ad i s i mpor t ant
eval ( 10adpat hf i 1e) ;
も eva1uat es t he st r i ng enc10sed - whi ch i nc1udes t he
も Mat 1ab command 10ad. Thi s wi 11 i mpor t t he xy- dat a
告 t he dat a wi 11 be avai 1ab1e i n t he wor kspace as a
者 var i ab1e wi t h t he same name as t he f i 1ename
告 wi t hout t he ext ensi on ( t hi s assumes you se1ect ed
も t he xy- dat a usi ng t he f i 1e se1ect i on box)
newname st r r ep( f i 1e,' . dat ' , " ) ;
も newname i s a st r i ng var i ab1e whi ch cont ai ns t he
告 st r i ng f i l e st r i pped of t he . daヒ ext ensi on. Thi 白
苦 i s a st r i ng r ep1acement command
x eval ( newname) ; 昔 assi gns t he i mpor t ed dat a t o x
者 j ust f or conveni ence. The above st ep i s not
者 necessar y
NDATA l engt h( x( : ,1) ) ; 苦 number of dat a poi nt s
cl ear pat h 10adpat hf i 1e newname
者 get r i d of t hese var i ab1es t o r ecover memor y
者 Not e: i f a f i 1e i s not se1ect ed, not hi ng i s bei ng
% done
者 Use of an i nput di a10g box t o get t he or der
者 of po1ynomi a1 t o be f i t t ed
PROMPT {' Ent er t he Or der of t he Cur ve' };
者 PROMPT i s a st r i ng Ar r ay wi t h one e1er時 抗
告 not e t he cur l y br acket s
TI TLE ' Or der of t he Po1ynomi a1 t o be Fi t t ed' ;
者 a st r i ng var i ab1e
LI NENO 1; 苦 a dat a var i ab1e
get va1 i nput dl g( PROMPT, T工TLE, LI NENO) ;
者 t he i nput di al og capt ur es t he user i nput i n get val
告 ge七 val i s a st r i ng Ar r ay
者 check hel p i nput dl g f or mor e i nf or mat i on
no st r 2num( get val {l ,l }) ;
者 t he st r i ng i s conver t ed t o a number - t he or der
cl ear PROMPT T工TLE L工NENO 者 del et i ng var i abl es
% cal l f unct i on mypol yf i t and obt ai n t he coef f i ci ent s
coef f mypol yf i t ( x,no) ;
者 gener at e t he f i t t ed cur ve and obt ai n t he squar ed
告 er r or
er r 2 0. 0 i'
f or i 1 : NDATA 者 f or each dat a poi nt
f or j l : no + 1
a( l ,j ) = x( i ,l ) "( j - l ) ;
y( i ) a*coef f ; 者ヒ he dat a f or ヒhe f i t ヒed cur ve
er r 2 er r 2 + ( x( i ,2) - y( i ) ) *( x( i ,2) - y( i ) ) ;
者ヒ he squar e er r or
者 pl ot t i ng
pl ot (x ( : , 1) , x ( : ,2) , , ro' , x ( : ,1) , y, , b- ' ) ;
% or i gi nal dat a ar e r ed o' s
告 f i t t ed dat a i s bl ue sol i d l i ne
xl abel ( , x' ) i
yl abel ( , y' ) ;
st r or der set st r ( num2st r ( no) ) ;
者 conver t t he or der of cur ve 七 o a st r i ng
巷 same as get val i f you have not cl ear ed i t
者 set st r assi gns t he st r i ng t o st r or der
t i t l est r [ ' pol ynomi al cur vef i t or or der ' ,
s七 r or der , ' of f i l e' , f i l e) i
者 t he t hr ee dot s at t he end ar e cont i nuat i on mar ks
者 t he t i t l e wi l l have t he or der and t he f i l e name
t i t l e ( t i t l est r )
l egend( ' or i gi nal dat a' , ' f i t t ed dat a' ) ;
er r st r l num2st r ( er r 2) ;
er r st r 2 [ ' squar ed erど or er r st r l ]
gt ext ( er r st r 2) ;
告 t hi s pl aces t he st r i ng er r st r 2 whi ch i s obt ai ned
者 by combi ni ng t he st r i ng ' squar ed er r or ' wi t h
も t he st r i ng r epr esent i ng t he val ue of t he er r or ,
者 wher ever t he mouse i s cl i cked on t he pl ot .
者 movi ng t he mouse over t he f i gur e you shoul d
告 see l ocat i on cr oss- hai r s
cl ear st r or der t i t l est r er r st r l er r st r 2 a y x i j
cl ear NDATA no coef f XY2 f i l e er r 2 get val
gr i d
者 Thi s f i ni shes t he exer ci se
Run theprogra四 byfirst runni ngMATLAB in由巳 di rectorywhere these削 es are. or
addi ng the p柚 to 1∞ate 白 白 血 At 白 e command prompt type pr oLP耐 . The
programshoul d EXEcute requi ri ng user i nput through the tile sel ecti on bOLi nput
di al og, and final1y di spl ayi ngFi gure 1.11. 百 leappe釘 ancemay be slightly di fferent
dependi ng on恥 pl aぜ' ormMATLAB is bei ngrun.
M Thi s fi ni shes the MATLAB secti on of the chapter. Tl msecti on lIas i ntroduced
ATLABin a robust manner. A broad mng巴 of pr ogmmmi ng experi ence has been
ml tl ated in this chapter- AH mw commands have hen i derui ncd wi th a bdef
expl anati on in白巴 comments. It is i mportant 白 紙 youusetheOPPOl tuni ty totypein
出 ECodE yoursel f;That is theonl y way theuseof MAT1AB will become fami l i ar. The
practi cea1sowi l l l eadtofewer syntax erro瓜 Thewri ti ngof code will si gni fi cantl y
i mprove your abi1i ty to debug and troubl eshoot. While 白 is chapter empl oyed a
Pol ynomi al curve fIt of order 3 of fi l eXY2.dat
Fl gure1.11 Origlnal andfllleddala.
t¥2J!.: ;l; ' f. l 草 山 記 ω」i 44uuム 凶 1uωi 即 位 k J t : i ム 幽 必 泌 叫 抽 融 柏 崎 州 附
seoarate secti on on the use of MATLAB out of n町 民 sity,sub叫 uent c hapmMe
CMa伽 i zedby a compl 仰 向 rati onof the us関 m巴 叩of MAT 官 .TLAB
opti mi匂za低tiぬon.
uruuz;ztm叩: ; 官官 a官lrι; 叩 ::;t;
mathemati cal modeJ.
1.2 I denti fy several possi bl eopti mi zati onprobl ems rel atedωan ai rcraft. For each
probl emi denti fy all め e di sci pl i nes 白 紙 wi l l hel p establ i sh the mathemati cal
1.3 Id 由巴 nt仲 s総ev 刊 巴 叫 possi bl e 0 叩pti mi za
p戸m帥州 0 油blふふ i耐d白e凶 匂 a叫11 t 侃he d 血i也s 叫ci増pl i n巴郎s 削 w叫 illけ凶山川 h 恥刷 1 惚叫叫 el如p es酬 i凶sl ぬh 恥 ma 闘la伽 na 削at山i比L刷
1・4 I denti fy severa1possi bl 巴 opti mi zati onprobl ems rel atedtoa mi crosysternused
for control . ForBach probl emi denti fyall thEdi scI Pl i nes that wi l l hel pestabl i sh
出 emathemati cal model .
1.5 Def i ne a probl emwi t1I respect toyour i nvestment inthestock market. Descri be
血 enatureof themathemati cal modd-
1.6 Def i ne 由巳 probl emand establ i sh 由巳 mathemati ca1 model for the I - beam
hol di ng up an i ndependent si ngl e-fami l y home.
1. 7 Def i ne the probl emand i denti fy the mathemati cal model for an opt i mum
overhangi ng traffic l i ght.
1.8 Def i ne 血 e probl emand i denti fy a mathemati ca1 model for schedul i ng and
ooti mi zati onof thedai l yrouti neactivity.
ωD叫ne 出 e p州 e mfor a l ami n
命 i灼v叩 gi venaspe 配:ei出白 c p 卯ump.
1・10 Def i ne a chemi cal engi neeri ng probl emto mi x v釘 i ous mi xtures of 1i mi ted
avai l abi l i ty tomake speci fi edcompounds tom田 t speci fi edd巴 mands.


Thi s book i ncl udes a l argenumber of cxampl es wi th t wo vari abl es. Two- vari abl e
probl ems can be di spl ayed graphi cal l y and th巴 sol uti on obtai ned by i nspecti on.
Al ong wi th the useful ness of obtai ni ng the sol uti on wi thout appl yi ng
mat h巴mati cal condi ti ons, thegraphi cal representati onof theprobl emprovi des an
opportuni ty to expl ore the geometry of many of the numeri cal techni ques we
exami ne later. Thi s is necessary for i ntui ti vel y understandi ng theal gori thmand its
progress toward the sol uti on. I n practi cal si tuati ons whi ch usual l y i nvol ve over
t wo vari abl es, thereis no opportuni ty for neat graphi cal correl ati onbecause of the
Ii mi tati ons of graphi cal representati on. It wil1be necessary torevi ewthenumbers
to deci de on convergence or thel ack thereof. What happens wi thmany vari abl es
is an extensi onof thegeometri c features that areobserved inprobl ems i nvol vi ng
t wo vari abl es. A good i magi nati on is an ess巴 ntial tool for these probl ems wi th
many vari abl es.
Contour pl ots provi de the best graphi cal 陀 presentati on of the opti mi zati on
probl emint wo vari abl es. The poi nts on any contour (or curve) have thesame val ue
of the functi on. Several so抗warepackages areavai l abl e to createand di spl ay these
plots. MATLAB. Mathemati ca , Mapl e, and Mat hcad are among 出 e general -puqJ Ose
softwarepackages 白 紙 candrawcontour plots. Inthis book, we will use MATLAB for
graphi ca1opti mi zati on.


The standard format for opti mi zati on probl ems was establ i shed in Chapter 1. lt is
rein町 oduc叫 herefor conveni ence:
Mi ni mi ze fl.. xJ , X2, . • • , Xn)
Subj ect to: h,(x" X2, • • • , Xn) = 0
(x,・X2・・.. ,x, , ) =O
ht(XI ,X2・・.. . X,,) =0
gl ( X"X2・・... Xn) 三 o
g2(XI . X2・・.., Xn) SO
gl l l (X"X2・・.. . x, , ) SO
x: gxi 三 X7.
i =I ,2,..., n
In山 is chapter, whi l eadheri ngtotheformat. thenecessi tyfor zeroon theri ght-hand
sideis rel axed. 百 is is bei ngdone for conveni ence and10aidcomprehensi on. 百 le
ri ght-hand side can also have numeri cal val ues other 1han zero. Th巴 first exampl e
chosenfor illustrationis a si mpl eonEusi ng el ementary functi ons whose gmp12i cal
natureis wel l known. Thi s si mpl eexampl e will permi t exarni nnti onof theMATLAB
code白 at will gencrate曲 ecurves andthesolution. It will alsoal l owus todefi ne山 E
format for thedi sp1ayof sol uti ontothegraphi cal opti mi zati onprobl cm.
2.1.1 Exampl e 2.1
The first exampl e, Exampl e 2.1, will have two equal i tyconstrai nts andtwo i nequal i ty
Mi ni mi ze fl.. XI'
X2) = (XI -
+ (X2-
2)2 (2.5)
Subj cct to: hl (xl . X2): 2x
1 + x2= 8 (2.6a)
XU: (XI -
+ (ゐ
- 4) 2=4 (2.6b)
gl (X" X2): Xl + x2S 7 (2.7n)
2(x, . XU: ー  
o SXI S 10; OSX2 S 10 (2.8)
Intheabove dcfmi 1i on, we have two straight lines. two circles, nnda parabol a. Note
that two equal i ty cons回 ints and two vari abl es i mpl y 伽 t 出 E陀 is no scope for
出 凶 zation. 百 leprobl emwill be detcrmi nedby出 巴 司 uali1yconstrai nts provi dedthe
twocons回 ints areI i nearl yi ndependent, whi ch is trueinthis exnr叩 le. This exampl e
wasα 凶 1ed10hel punderstand由巳 codci nMA' 礼 AB山 at will beusedtodrawgraphi cal
sol uti ons inthis book.
Fi gure2.1iII ustrates thegraphi cal sol uti onto白 i sprobl em. 百 lefigurealsodi spl ays
addi ti onal i nformati on, related to the i nequal i ty cons佐 aints,pl aced on it after the
principal graphi c i nformati onwas generatedand di spl ayed.
2.1.2 For mat for the Graphl cal Dl spl ay
The graphi cal sol uti on to Exampl e 2.1, as seen in Fi gure 2.1, is generated usi ng
MATI . AB [1.2] ex∞:pt for theidentifi 伺 tionof thei nequal i tyconstraints. Al l i nequal i ty
constrai nts aredi sti ngui shedby hashmarks. The hashedsi deindicates theinfeasible
region. InMA'τ 1. AB5.2,出 ereisno featuretoinsert thesemarks throughprogrammi ng.
An add-onprogramcalledMat dr awwas usedtocreatethehashI i nes inthefigure. It
canbe downl oaded f romtheMathworks site. Begi nni ngwi 山 Versi on5.3 thereis a
figure(plotedit) 巴ditor 白 紙 will al l owyou toinsert addi ti onal graphi c el ements to出 E
fig町 e. 百 lestudent canalsop叩 cil in由 民 ehashtines after obtai ni ngthepri ntout It is
tobe noted出 at thegraphi cal sol uti onis i ncompl eteif thei nequal i ty constrai nts are
not di sti ngui shed, or if thefeasibleregi onhas not beenestabl i shedinsome manner.

2 3 4 5 6
円 guro2.1 Graphical solutionfor Exampl e2.1.
s 10
In the graphi cal di spl ay of sol uti ons, onl y 血 e obj ecti ve functi on is drawn for
several contours sothat 白 edi recti onfor 白 emi ni mumcan be i denti fi ed. Each cqual i ty
constr必 nt is drawn as a si ngl ecurve for 出 enumeri cal val ueon theri ght-handside.
They shoul dbe i denti fi ed. Each i nequal i tyconstraI nt is si mi l arl ydrawn for a val ueon
出 eri ght-handside. They need tobe i denti fi cdtoo. The hash marks ar町 l rawn/penci l ed
inon thefi gurc or a pri ntout of 出 efi gu陀 . The extent of thedcsi gnregi onisestabl i shed
and thesol uti oni denti fi edon thefi gure. The rel evant regi oncan be zoomed for better
accuracy. 百 四 datafor 白 epl ot areobtai nedusi ngMATLAB' S matri x operati ons after
establ i shi ng the pl ol ti ng mesh. This expl oi ts MATI . AB' S nal ural speed for matri x
operati ons.
MATLAB poss凶 s a powerf ul vi sual i zati on engi ne that permi ts the sol uti on of 白 e
two-vari abl e opti mi zati on probl emby i nspecti on. There are three ways to l ake
advantage of the graphi c features of MATI . AB. 百】 e first is l he use of MATI . AB'S
hi gh-I evel graphi ngrouti nes for data vi sual i zati on. This will be山巴 pri mary way to
sol ve graphi cal opl i mi zati on probl ems in this book. Thi s wil1 al so be 出 e way 10
i ncorporate graphi cal expl oral i on of numeri cal techni ques in 出 e book. For mo隠
preci se control over the di spl ay of data , MATLAB al l ows us巴 r i ntcracti on through
programmi ng usi ng an obj ect-ori ented systemi denti fi ed in MATI . AB as Handl e
Graphi cs. The thi rduseof the MATLAB graphi cs engi ne is touse theHandl e Graphi cs
systemtodevel op a Graphi cal User I nterface(GUI ) for theprogramor m-fi l e. This
creal es a facility for sophi sl i cateduser i nteracti on. MOSl of thepl otti ngneeds can be
mel by thehi gh-I evel graphi cs functi ons avai l abl einMATLAB.
2. 2. 1 MATLABHi gh- Level Gr aphl cs Funct l ons
τ'here are three useful wi ndows duri ng a typi cal MATLAB scssi on. The first is 白 e
MATLAB Command wi ndowthroughwhi ch MATLAB recei ves i nstructi onand di spl ays
al phanumeri c i nformati on. 百 le second wi ndowis the texl -edi tor wi ndowwhere
m- fiI es are coded. The third is the Fi gure wi ndowwhere the graphi c el ements are
di spl ayed. There can be mor e白 anone fi gurewi ndow. The fi gurewi ndowis出巴 targel
of thehi gh-l evel graphi cs functi ons.
The graphi cs functi ons in MATLAB al l owyou to pl ot in 2D or 3D. They al l ow
contour pl ots in2D and 3D, mesh and suばaωpl ot s,bar, area, pi echarts, hi stograms,
ani mati on, and gradi ent plots. Subpl ots can al sobe di spl ayedusi ngthesefuncti ons.
In addi ti on they permi t operati on wi thi mages and 3D model i ng. They al l owbasi c
control of thc appear阻 ce of the pl ot through col or, line styl e, and markers,蹴 is
ranges, and鎚 pect ratioof thegraph.τ 'hey permi t annotati onof thegraph inseveral
ways. Some of thesefuncti ons will be used inthenext secti onwhen we devcl opthe
m-fi l efor 山 efirst exampl e. The fol l owi ngi ntroduces some more useful I nf orm姐 on
about theuseof MATLAB・8 hi gh-I evcl graphi cs functi ons.
The t wo mai n graphi cal el ements 白 紙 arc typi cal l ycontrol l edusi ngthehi gh-I evel
graphi cs functi ons arethefi gu陀 andtheaxes. Usi ng Handl e Graphi cs you can control
most of theother graphi cal el ements, whi ch i ncl udeel em巴nts usedintheGUI. These
other el ements are typi cal l y chi l drcn of 山 e axes or fi gure. The fi gure functi on or
command creates a fi gurewi ndowwi tha number starti ngat one, or will creat巴 anew
fi gure wi ndowi ncrementi ng the wi ndowcount by one. Normal l y all graphi cs
functi ons aretargetedtothec町 rent figur芭 wi ndow,whi ch is sel ectedby cl i cki ngit
wi th the mouse or executi ng the command fi gure (number) , where number is the
number of thefi gure wi ndow山 al will have l he focus. AIl commands arei ssuedin
command wi ndow. Graphi cs functi ons or commands will automati cal l y create a
wi ndowif none exists.
Invoke hel p commandname at the MATLAB prompt to knowmor e about the
functi ons or comrnands. The wor d handl e app回 路 on m加 y pl aぽorms and inmany
appl i cati ons, parti cul arl ythosethat deal wi thgraphi cs. They arewi del y preval ent in
obj ect・ori ented programmi ng practi ce. MATI . AB・s vi sual i zati on systemis obj ect
ori ented. Most graphi cal el ements areconsi deredas obj ects. The “handl e" inMATI . AB
is a sy印 刷 oftwarecreated number 白 紙 can i dent汀'y thespeci fi c graphi c obj ect . If
this handl e is avai l abl e,山 enproperti es of theobj ecl such as linesize, marker type,
col or, and soon can be vi ewcd, set, or reset if neccssary. InMATLAB, Handl e Graphi cs
is 出 eway tocustorni ze出 egraphi cal el ements. Inhi gh-I evel gr叩 hi cs functi ons 出 s
is used ina l i mi tedway. Inthis chapter, we will use it ina mi ni mal way tochange出 E
charactcri sti cs of some of thegraphi cal el ements on thefigure.
To understand the concept of "handl cs" we wi l l run 出 e fol l owi ng code
i nteracti vel y. It deal s wi thcreati nga pl ot, whi ch was i ntroduced inChapter 1. The
bol dface comment s in the code segment i ndi cate the newfeal ures that are bei ng
emphasi zed.
Sta武 MATLABand i nteracti veLyperf ormthefol l owi ng:
>>X O: pi / 40: 2*pi ;
> > y X. 合 si n(x) ;
>> pl ot ( x,y,' b- ' ) ;
>> gr i d;
>>h pl ot ( x,y ,' b- ' ) 者 h i s t he handl e t o t he pl ot
> > 告 a new pl ot i s over wr i t t en i n t he same f i gur e wi ndow
> > 苦 a numer i cal val ue i s assi gned t o h
> > 者 We can use t he var i abl e name h or i t s val ue
> > 告 t o r ef er t o t he pl ot agai n
>> set ( h,' Li neWi dt h' ,2) ; も t hi s shoul d make your pl ot
> > 告 t hi cker bl ue
>> set ( h,' Li neWi dt h' ,3,' Li neSt yl e' ,' :', ' Col or ' ,' r' )

















> > 者 The handl e i s used t o r ef er t o t he obj ect
> > も whose pr oper t y i s bei ng changed
> > 者 Usual l y Pr oper t y i nf or mat i on occur s i n pai r s of
> > 者 pr oper t y- namej pr oper t y- val ue
> > も pr oper t y- val ue can be a t ext st r i ng or number
>> get ( gca) も t hi s wi l l l i st t he pr oper t y of t he axes
> > 者 of t he cur r en七 pl ot . Not e t her e ar e a si gni f i cant
> > 者 amount of pr oper t i es you can change t o cust omi ze
> > 者 t he appear ance of t he pl ot
>> set ( gca,' yt i ck' , [- 5, ー 2. 5,0,2. 5,5] )
> > 告 you have r eset t he yt i ck mar ks on t he gr aph
>> set ( gca,' Font Name' ,' Ar i al ' ,' Font Wei ght ' , ...
>> ' bol d' ,' Font Si ze' ,14)
> > 者 Changes t he f ont used f or mar ki ng t he axes
>> set ( gca,' Xcol or ' ,' bl ue' )
> > も changes t he x- axi s t o bl ue
>> % concl udes t he demonst r at i on of handl es
Fr omthe sessi onabove it is evi dent 白 紙 to fi ne-tune thepl ot you create, you
need to first i denti fy the obj ect handl e. You use the handl e to access 白 eobj ect
prop凶 y. You change a property by setti ngits val ue. We will seemor e graphi cs
functi on as we obtai n the graphi cal sol uti on to Exampl e 2.1 in 由巳 fol l owi ng
sectl on.
2. 2. 2 Exampl e 2.1ー- Graphi cal Sol uti on
Fi gure 2.1 is thegraphi cal representati onof Exampl e 2.1. The range for thepl ots
matches thesi deconstrai nts for theprobl em. The i ntersecti onof thet wo equal i ty
constrai nts i denti fi es t wo possi bl 巴 sol uti ons to theprobl em, approxi matel y (2.6,
2.8) and (1, 6). The i nequal i ty constrai nt g2 makcs the poi nt (2.6, 2.8)
unacceptabl e. Poi nt (1, 6) is acc巴 ptabl e wi th respect to both constrai nts. Thc
sol uti on is therefore at (1, 6). Whi l e the obj ecti ve functi on was not used to
determi ne thc sol uti on, contours of theobj ecti vefuncti onaredrawn i ndi cati ng白巴
di recti onfor 出 emi ni mumof theobj ecti v巴 functi on. The sol uti oncan be i denti fi ed
by i nspecti onof theass巴 mbl edpl ots wi thout reference to thetermi nol ogy or the
techni qucs of opti mi zati on. Recal l , however. that this is onl y possi bl ewi thone or
t wo desi gnvari abl es intheprobl em.
The code for 血 is ex創 npl ei nvol ves sixm-fi l es: a scri pt m-fi l e白 紙 will inc¥ude
the MATL AB statements to create and di spl ay 出 edata, and fi vem- fi 1国一 onefor
cach functi on i nvol ved in Exampl e 2. 1. We begi n wi th the script m- f i l e-
Ex2_1. m.
File: Ex2 1. m
者 Chapt er 2: Opt i mi zat i on wi t h MATLAB
% Dr . P. Venkat ar aman
者 Exampl e 2_1 ( Sec 2. 1- 2. 2)

も gr aphi cal sol ut i on usi ng MATLAB ( t wo desi gn var i abl es)
毛 t he f ol l owi ng scr i pt shoul d al l ow t he di spl ay
苦 of gr aphi cal sol ut i on

も Mi ni mi ze f ( x1,x2)
者 h1( x1,x2)
者 h2( x1,x2)
告 gl ( X1,x2)
告 gl ( x1,x2)

(x1・ 3) **2 + (x2・ 2) **2
2x1 + x2 8
( x1- 1) ^2 + ( X2- 4) ^2 4
x1 + x2 <= 7
x1 - 0. 25x2^2 <= 0. 0
% 0 <= x1 <= 10 0 <= x2 <= 10

も NOTE: The hash mar ks f or t he i nequal i t y
も const r ai nt s must be det er mi ned and dr awn out si de
者 of t hi s exer ci se and on t he pr i nt out
者 ー ー ー ー ー - - - ・ ・ 白 ー ー ー ー ー ー ー ー ー ー ー - - ー ー ー ー ー ー ー ー ー ー ー ー ー ー " ・ ー ・ " ー ー ー ー ー ー - -
x1=0: 0. 1: 10i 者 t he semi ・col on at t he end pr event s
% t he echo
x2=0: 0. 1: 10i 者 t hese ar e al so t he si de const r ai nt s
者 x1 and x2 ar e vect or s f i l l ed wi t h number s st ar t i ng
苦 at 0 and endi ng at 10. 0 wi t h val ues at i nt er val s
告 of 0. 1
[X1 X2] meshgr i d( x1,x2) i
者 gener at es mat r i ces X1 and X2 cor r espondi ng t o
% vect or s x1 and x2. I t i s a mesh of x1 and x2 val ues
者 at whi ch t he f unct i ons wi l l be eval uat ed
f 1 obj _exl ( X1,X2) i 告 t he obj ect i ve f unct i on i s
者 eval uat ed over t he ent i r e mesh and st or ed i n f 1
者 MATLAB wi l l comput e t he val ues f or t he obj ect i ve
も t hr ougha f unct i on m田 f i l e cal l ed obj _exl . m
i neq1 i neconl ( Xl ,X2) i 巷 t he i nequal i t y gl i s
者 eval uat ed over t he mesh
i neq2 i necon2( Xl ,X2) i 者 t he i nequal i t y g2 i s
も eval uat ed over t he mesh
eq1 = eqcon1( Xl ,X2) i 者 t he equal i t y h1 i s eval uat ed
者 over t he mesh
eq2 = eqcon2(X1,X2) i 告 t he equal i t y h2 i s eval uat ed
者 over t he mesh
[C1,han1] cont our ( χ1, x2, i neq1, [7,7) , ' r- ' ) i
も a si ngl e cont our pl ot of gl ( or i neql ) i s dr awn f or
者 t he val ue of 7 i n r ed col or as a cont i nuous l i ne
も dupl i cat i on of t he cont our val ue i s necessar y
も f or dr awi ng a si ngl e cont our
者 han1 i s t he handl e t o t hi s pl ot . Thi s handl e
者 can be used t o change t he pl ot di spl ay char act er i st i cs
告 Cl cont ai ns t he val ue of t he cont our
cl abel ( C1,han1) i
も l abel s t he cont our wi t h t he val ues i n vect or C1
者 cont our ( x1,x2,i neq1,[7,7] ,' r- ' ) wi l l dr aw t he cont our
告 wi t hout l abel i ng t he val ue
hol d on 告 al l ows mul t i pl e pl ot s i n t he same f i gur e
告 wi ndow
gt ext ( , gl ' ) i
も wi l l pl ace t he st r i ng ' gl ' on t he pl ot at t he spot
苦 sel ect ed by a mouse cl i ck. Thi s i s a t ext l abel
者 t hi s pr ocedur e i s r epeat ed f or r emai ni ng
も const r ai nt s
[C2,han2] cont our ( x1,x2,i neq2, [0,0] ,' r- ー , ) i
cl abel ( C2,han2) i
gt ext ( ' g2' ) i
[C3, han3] cont our (x1, x2, eq1, [8,8] , ' b- ' ) i
cl abel ( C3,han3) i
gt ext ( , h1' ) i
[C4,han4] cont our ( x1,x2,eq2, [4,4] , ' b- - ' ) i
cl abel ( C4,han4) i
gt ext ( , h2' ) i
[C,han] cont our ( x1,x2,f 1,' g' ) i
告 cont our of ' f ' i s dr awn i n gr een col or . The number
苦 of cont our s ar e deci ded by t he def aul t val ue
cl abel (C, han) i
xl abel ( ' x1 val ues' ,' Font Name' ,' t i mes' ,' Font Si ze' ,12,.
' Font Wei ghヒ , ,' bol d' ) i も l abel f or x- axes
yl abel ( ' x2 val ues' ,' Font Name' ,' t i mes' ,' Font Si ze' ,12,
' Font Wei ght ' ,' bol d' ) i
gr i d
hol d of f
The functi onm- fiI es are:
凪鑑on1. m
f unct i on r et val i necon1( X1, X2)
r et val X1 + X2i
も X1, X2 ar e mat r i ces
者 r et val i s t he val ue bei ng r et ur ned af t er t he
告 comput at i on
者 Si nce X1 and X2 ar e mat r i ces r et va1 i s a1so a
告 mat r i x
者 By t hi s way t he ent i r e i nf or mat i on on t he mesh
者 i s gener at ed by a si ng1e cal l t o t he f uncヒi on
者 i necon1
凪 血 血 2 . m
f unct i on r et va1 i necon2( X1,X2)
r et val Xl - 0. 2S* X2. ^2i
者 Not e t he use of ^ oper at or f or el ement by e1ement
告 oper at i on. That i s each e1ement of t he X2 mat r i x
者 i s squar ed. Wi t hout t he dot t he i mp1i cat i on i s a
者 mat r i x mul t i pl i cat i on - bet ween mat r i ces whose
者 i nner di mensi ons must agr ee. Si mi l ar oper at or s ar e
告 def i ned f or el ement by el ement mul ヒi pl i cat i on
者 and di vi si on
盤 姐 L m
f unct i on r et val eqcon1( X1,X2)
r et val 2. 0*Xl + X2i
魁 迎 且 2. m
f unct i on r et val eqcon2( X1,X2)
r et val (X1 - 1) . *( X1 ・ 1) + (X2 - 4) . * (X2 - 4) i
曲 U 叫 掴
f unct i on r et val obj _ex1( X1,X2)
r et val = (X1 - 3) . *( X1 - 3) +( X2 - 2) . 合 (X2 ・ 2) i
2. 2. 3 Di spl ayl ng t he Gr aphi cs
Al1of lhc files requi redfor graphi cal di spl ayof theprobl emhave been crcal cd. Inthe
Command wi ndow, l ypeaddpat b fol l owedby出 ecompl el e pathfor thcdi rectory山 at
hol ds these files. At the prompt, type thc name of thc script file for the exampl e
wi thout the. meXl ensi on
>> Ex2 1
The first contour pl ot will appear. Move themouse over to白 epl ot and a cross hai r
appears. Cl i cki ngon thepl ot wi1l pl ac巴 thestri ng"gl " at thecross hair. The second
contour pl ot shoul dappear and thereis a pause topl ac巴 出 etext l abel . After thefour
cons住 aints,出 eobj ecti vefuncti onispl ottedfor several contour val ues. Fi nal l ythepl ot
shoul d app伺 r ωi nFi gure 2.1 wi thout thehashmarks (andwi 山 your choi ce of 白 B
l ocati onfor constrai nt labels). You can insert thehashmarks throughedi ti ngthepl ot
di rectl y ( Versi on5.3 onw釘 ds). 明 記 sol uti onfor theprobl emis at ( 1 , 6) , where four
pl ots i ntersect . The val ueof theobj ecti vefuncti onis 20鉱 山 esol uti on.
If thepl ot is acceptabl e, you canpri nt thei nformal i onon thefi gureby expo凶 ngit
(usi ng an appropri ate extensi on) 10 a file and later i ncorporati ng it in another
document . You can al so send it to 出 epri nter f romIhe pri nt command on theFi l e
menu. Typi ng hel p pri nt intheCommand wi ndowshoul dI i st a set of commands you
can use tosave曲 efile.



2 4 6
s 10
>> pr i nt - depsc2 pl ot _ex_2_1. eps
wi1l c路 ate the l evel 2 col or postscri pt file cal l ed pl ocex_2_l. eps in the worki ng
di rectory. A compl ete pathname shoul dsavethefileto白 especi fi eddi rectory. You
can then execute standard operati ng systemcommands 10 dump the filc 10 出 e
appl i cabl 巴 printcr. You can al so save the fi gure (ex蜘 si on .fig) and later open it in
MATLAB usi ngthefilc open command f rommenu b釘 ortool bar.
2. 2. 4 Cust oml zl ng t he Fl gure Fl gure2. 2 Customi zed両日 U悶 forE姐 mpl e2.1.
Inthis secti onthc basi c fi gureand pl ots creatcdabov巴 wilI be custorni zcdusi ngthe
Handl e Graphi cs commands first expl o陀 di nS巴cti on2.2. 1. Al l ofthe pl ots descri bi ng
白 econstrai nts will have a l i newi dthof 2 poi nts. 百 leobj ecti vefuncti onwi l l have a
l i newi dthof 1 poi nt. The text l abel s wi1l be inthe"Ti mes" font. It wilI be inbol dface
wi tha font of si ze 14and in白 ecol or of thecons回 int . Thex叩 dY tickmarks will be
changed to establ i sh a broader grid. The x 叩 dy labe1s wi l l i ncl ude a subscri pt . A
two- rowtext wi1l be i nsertedat thepoi nt sel ectedthroughthemousc. The ncwfi gure
is shown inFi gurc 2.2.
First ∞py thefilein出 eprevi ous exerci seand rename it.羽 lerenamed filewill be
used ωmaket hen配 essarych佃 gesto出 ecode. Thesechanges inc1ude bothnewcode
as weIl as repl acement of exi sti nglines of code. In出 efoIl owi ng onl y白 echanges to
出巴 codearei1I ustrated. The ori gi nal code is shown in the italic styleand i denti fi ed
wi ththreeellipsis poi nts (. . .) at 出巴 begi nni ngof theline. The code tobe i nsertedor
repl acement code is shown innor m叫 styl 巴. If no repl acemcnt is apparent, thentheol d
code is used 10defi nethel ocati ons whe回 目 ew∞deis appended.
g t 儲 t ( ' gl ' ) ;
k1 gt ext ( , gl ' ) ; 告 k1 i s t he handl e t o t he t ext el ement
set ( k1,' Font Name' ,' Ti mes' ,' Font Wei ght ' ,' bol d' ,
, Font Si ze' ,14,' Col or ' ,' r ed' )
. . cl abel ( C2,han2}i
set ( han2,' Li neWi dt h' ,2)
k2 = gt ext ( ' g2' ) ;
set ( k2,' Font Name' ,' Ti mes' ,' Font Wei ght ' ,' bol d' ,
, Font Si ze' ,14,' Col or ' ,' r ed' )
cl abel ( C3,han3) ;
set ( han3,' Li neWi dt h' ,2)
k3   gt ext ( ' h1' ) ;
set ( k3,' Font Name' , ' Ti mes' ,' Font Wei ght ' ,' bol d' ,
' Font Si ze' ,14,' Col or ' ,' bl ue' )
{Cl ,hanl } cont our ( xl ,x 2,i neql , {7,7},' r - ' };
set ( han1,' Li neWi dt h' ,2) ; 者 set s 1i ne wi dt h of t he gl
も cont our t o 2 poi nt s
cl abel ( C4,han4) ;
set ( han4, , Li neWi dt h' ,2)
k4   gt ext ( ' h2' ) ;
set ( k4,' Font Name' ,' Ti mes' ,' Font Wei ghヒ ' bo1d' ,
, Font Si ze' ,14,' Co1or ' ,' b1ue' )
c: l abel (C, han) ;
set ( han,' Li neWi dt h' ,1)
.. xl abel ( , xl val ues' , , Font Name' , ' t i mes' ,
x1abe1( ' x 1 va1ues' ,' Font Name' ,' t i mes' , , Font Si ze' ,12,.
, Font Wei ght ' ,' bo1d' ) i
y1abe1( ' x 2 val ues' ,' Font Name' ,' ti mes' ,'
Font Si ze' ,12, ' Font Wei ght ' ,' bo1d' ) i
set ( gca, , xt i ck' , [0 2 4 6 8 10] ) 者 set xt i cks
seヒ ( gca,' yt i ck' ,[0 2. 5 5. 0 7. 5 10] ) 昔 set yt i cks
k5 gt ext ( {' Chapt er 2: Examp1e 1' ,
' pr et t y gr aphi cal di spl ay' })
者 t he above gt ext descr i bes a st r i ng ar r ay
も st r i ng ar r ay i s def i ned usi ng cur l y br aces
set ( k5,' Font Name' ,' Ti mes' ,' Font Si ze' ,12,
, Font wei ght ' ,' bo1d' )
!(xI' X
) 町   一 中 川 ー cos(qx
) -1- c + d
wi th
a = 1, b = 2, c = 0.3, d = 0.4, p = 3π ,q=4π
Fi gurcs 2.3-2. 7ar宮 山 egraphi cal di spl ay of sol uti ons 泊 出 is section. The fi gures in
thebook bei ngres凶 ctedtobl ack and whi te will not ∞nvey也 ei mpact of col or i mages
you will see on the screen. Thcre are two files associ atcd wi ththeplots: thescript
m- fiI e (ex2.-2. m) and thc functi onm- fiI e (obLex2. m).
盟主. . z. m
者 Chapt er 2: Opt i mi zat i on wi t h MATLAB
者 Dr . P. Venkat ar aman
者 Examp1e 2. 2 Sec. 2. 3. 1

( at t he end)
c1ear C C1 C2 C3 C4 h h1 h2 h3 h4 k1 k2 k3 k4 k5
壱 get r i d of var i ab1es f r omt he wor kspace
Run theprograrntosec 曲 efi gureinFi g山 宮 2.2. Once thepl ot is customI zed toyour
sati sfacti on, you can make it 白巴 standardfor other pl ots you will pr吋 U四 i nMA礼As .
Thi s filecoul dal sobe a templ atefor general contour plotting. Note白 at a1l functi on
i nformati onis obtai nedthrough functi on m- fiI es whi ch arecoded outsi dethescript
filc. Sctti ng up ncwprobl ems or exampl cs onl y requi res addi ngl changi ng new
functi onm- fi1ω.
間 11ω I AbclodConl 凹 g
defaull cGlcr
The f01l 0wi ng addi ti onal ex創 npl es will servetoiIl us回 tebo白 opti mi zati onprobl ems鎚
wel l asaddi ti onal gmphi cal fcatures of M A TLA白血 at will be useful ind巴 vel opi nggraphi cal
sol uti ons toopti mi zati onprobl ems. 明 lCgrョphi cal routi nes inMATLAB arepowerful 叩 d
回 SYtouse. They cangrョ , phi cal l ydi spl aytheprobl ems 副 総 vem1ways wi 白 verysi mpl c
∞mm阻 ds. The useful di spl ayis, however, detcnni nedby血 euser. The first exampl e in
this secti on, Exampl e 2.2, is a probl emin uncons回 i ned opt泊 前 回 tion. 明 記 second
cxampl e is a structum1engi neeri ngprobl cmof reasonabl c compl cxi ty. 百 lC白 凶 exar即 Ic
demons回 飽 sopti mi zati oninthcareaof heat Iransfcr dcsi gn.

2.3.1 Exampl e 2. 2
Thi s cxampl c iII us位置 tes scvcral di ffcrent ways of gmphi caUy di spl ayi nga functi onof
two vari abl es. The probl emwas uscdtoiII usl rategl obal opti mi zati oninReference 3.
The si ngl c obj ccti vefuncti onis
-0. 5 O
val u鎚
円 gu開 2.3 Filledconl ours withcol orbar . Exampl e2.2.
2DCon随 町
wi thGrndi .nl V,酎: 10悶

val ues
-1 X:! V剖 ues
0.5 o
- 0.5
3Dconl our wilh∞lorbar . Ex創 叩 le2.2.
者 t he obj ect i ve f unct i on i s
告 eval uat ed over t he ent i r e mesh
者 f i l l ed cont our wi t h def aul t col or map
者 hel p gr aph3d gi ves you t he choi ces f or col or map
[C1,han11 cont our f ( x1,x2,f 1,
[0 0. 1 0. 6 0. 8 1. 0 1. 2 1. 5 1. 8 2. 0 2. 4 2. 6 2. 8 3. 01) ;
Fl gure2. 5
f 1 obj _ex2( X1,X2) ;
1臨書y' t.
t i cks
t i cks
者 speci f i c cont our l evel s i ndi cat ed above
cl abel ( Cl ,hanl ) ;
col or bar 告 i l 1ust r at es t he def aul t col or scal e
set ( gca,' xt i ck' , [- 1 - 0. 5 0. 0 0. 5 1. 01) も cust om
set ( gca, ' yt i ck' , [・ 1 - 0. 5 0. 0 0. 5 1. 01) 者 cust om
gr i d
xl abel ( ' X_1 val ues' ,' Font Name' ,' t i mes' ,
' Font Si ze' , 12) ;
% l abel f or x- axes
yl abel ( ' x 2 val ues' ,' Font Name' , ' t i mes' ,
告 gr aphi cal sol ut i on usi ng MATLAB ( t wo desi gn var i abl es)
も Unconst r ai ned f unct i on i l l ust r at i ng gl obal mi ni mum
も Exampl e wi l l i nt r oduce 3D pl ot s , 3D cont our s , f i l l ed
も 2D cont our s wi t h gr adi ent i nf or mat i on
者・ーーーー・・・・・・ーーー・ー・ーーーーーーーーーー- - - - - - ・ーーーーーーーーー- - - - - - ーー- -
x1=・ 1: 0. 01: 1; も t he semi - col on at t he end pr event s
" t he echo
x2=- 1: 0. 01: 1i 巷 t hese ar e al so t he si de const r ai nt s
者 x1 and x2 ar e vect or s f i l 1ed wi t h number s st ar t i ng
苦 at - 1 and endi ng at 1. 0 wi t h val ues at i nt er val s of
者 0. 01
Contour wlthgradlent vectors: Exampl e 2.2. 円 gure2. 4
[Xl x21 = meshgr i d( x1,x2) ;
者 gener at es mat r i ces X1 and X2 cor r espondi ng
者 vect or s x1 and x2
者 r emi nder   T 工 ~ i s case sensi t i ve
CO!U' SC Mesh P101
ωl ormap・目。 1


val ues
-1 -1
val u凶
Fl gure2.6 3Dmesh plot wlthcolorbar: Exampl 自 2.2.
, Font Si ze' ,12) ;
t i t l e( {' Fi l l ed Label l ed Cont our ' ,' ...
def aul t col or map' },' Font Name' , , ヒ i mes' ,' Font Si ze' ,10)
者 a new f i gur e i s used t o dr aw t he
も basi c cont our pl ot super i mposed wi t h gr adi ent
告 i nf or mat i on
告 al so i nf or mat i on i s gener at ed on a coar ser mes h t o
者 keep t he f i gur e t i dy. gr i d i s r emoved f or cl ar i t y
f i gur e 告 a new f i gur e wi ndow i s dr awn
y1 - 1: 0. 1: 1. 0;
y2 ー1: 0. 1: 1;
[ Y1,Y2] meshgr i d( y1,y2) ;
f 2 obj _ex2( Yl ,Y2) ;
[C2,han2] cont our ( y1,y2,f 2,
[0 0. 5 0. 75 1. 0 1. 5 1. 75 2. 0 2. 5 3. 0] ) ;
Coa問。 SurfacePlot
。 0.5

心 5
val ues
-1 -1


a v


Fl gure2.7 3Dsu巾 ωpl ot withdefault col orbar : Exampl e 2.2.
cl abel ( C2,han2)
[ GX, GY] gr adi ent ( f 2,0. 2) ;
% gener at i on of gr adi ent i nf or mat i on see hel p gr adi ent
hol d on 者 mul ヒi pl e pl oヒs on t he same f i gur e
qui ver ( Y1,Y2,GX,GY) ;
者 dr aws t he gr adi ent vect or s at gr i d poi nt s
も see hel p qui ver
hol d of f
set ( gca,' xt i ck' , [ - 1 - 0. 5 0. 0 0. 5 1. 0] )
set ( gca,' yt i ck' , [ - 1 - 0. 5 0. 0 0. 5 1. 0] )
xl abel ( ' x_1 val ues' ,' Font Name' ,' t i mes' ,
, Font Si ze' , 12) ;
yl abel ( ' x 2 val ues' ,' Font Name' ,'ヒ i mes' ,
' Font Si ze' , 12) ;
t i t l e( {' 2D Cont our ' ,' wi t h Gr adi ent Vect or s' },
, Font Name' ,' t i mes' ,' Font Si ze' ,10)
も A f i nal exampl e of cont our i nf or mat i on i s a 3D
% cont our pl ot - or a st acked cont our pl ot
f i gur e
col or map( spr i ng)
cont our 3( x1,x2, f 1,
[0 0. 3 0. 6 0. 8 1. 0 1. 5 1. 8 2. 0 2. 4 2. 6 2. 8 3. 0] ) ;
set ( gca,' xt i ck' , [- 1 ・ 0. 5 0. 0 0. 5 1. 0] )
set ( gca, ' yt i ck' , [- 1 - 0. 5 0. 0 0. 5 l . 0])
も change col or map and set col or bar
者 changi ng col or f or di spl ay i s easy by sel ect i ng
者 among some col or maps pr ovi ded by MATLAB
col or bar
xl abel ( ' x 1 val ues' ,' Font Name' ,' t i mes' ,
, Font Si ze' , 12) ;
yl abel ( ' x 2 val ues' ,' Font Name' ,' t i mes' ,. . .
, Font Si ze' , 12) ;
t i ヒl e( {' St acked (3D) Cont our ' ,: col or map - spr i ng' },
, Font Name' ,' t i mes' ,' Font Si ze' ,10)
gr i d
苦 ヒ he next t wo f i gur es wi . l l di spl ay 3D pl ot s
者 ヒ he f i r st i s a mesh pl ot of t he f unct i on
者 once mor e coar se dat a i s used f or cl ar i t y
f i gur e
col or map (cool ) 苦 anot her col or map
mesh (y1, y2, f 2) 者 usi ng i nf or mat i on gener at ed ear l i er
set ( gca, ' xt i ck' , [- 1 - 0. 5 0. 0 0. 5 l . 0])
set ( gca,' yt i ck' , [- 1 ・ 0. 5 0. 0 0. 5 1. 0] )
col or bar
xl abel ( ' x 1 val ues' ,' Font Name' ,' t i mes' ,
, Font Si ze' , 12) ;
yl abel ( ' x_2 val ues' ,' Font Name' ,' t i mes' ,. . .
, Font Si ze' , 12) ;
t i t l e( {' Coar se Mesh Pl ot ' ,' col or map - cool ' },
, Font Name' ,' t i mes' ,' Font Si ze' ,10)
gr i d
者 ヒ he f i nal pl ot i n t hi s ser i es
者 sur f ace pl ot wi t h def aul t col or map
f i gur e
col or map( j eヒ}
sur f ( y1,y2,f 2) 告 usi ng ol d i nf or mat i on
col or bar
xl abel ( ' x 1 val ues' ,' Font Name' ,' t i mes' ,.
, Font Si ze' , 12) ;
yl abel ( ' x_2 val ues' ,' Font Name' ,' t i mes' ,
, Font Si ze' , 12) ;
set ( gca, ' xt i ck' , [- 1 - 0. 5 0. 0 0. 5 l . 0])
set ( gca,' yt i ck' ,1- 1 - 0,5・0. 0 0. 5 l . 0])
t i t l e( {' Coar se Sur f ace Pl ot ' , ‘ col or map - j et /
def aul t ' },' Font Name' ,' t i mes' ,' Font Si ze' ,10)
gr i d
Q祉 s謡 . m
f unct i on r et val obj _ex1( X1,X2)
も Opt i mi zat i on wi t h MATLAB
者 Dr . P. Venkat ar aman
者 Chapt er 2. Exampl e 2. 2

も f ( x1,x2) ョ a*xl ""2 + b*x2^2 ・c合 cos ( aa*x1) ー
も d*cos ( bb*x2) . . .
も + c + d
a ' " 1; b 2・ c 0. 3; d ' " 0. 4; aa
3. 0*pi ; bb 4. 0*pi ;
者 not e mat r i x oper at i ons need a dot oper at or
r et val a*X1. *X1 + b*X2. *X2 - c*cos( aa*X1) ー
d*cos( bb*X2) + c + d;
The brief comments inthecode shoul dprovi deanexpl anati onof what you seeon
由巳 figure. Fi gures canbe further custorni zedas secnin血 eprevi ous section. Froman
opti rni zati onperspecti ve, Fi gure2.4provi des thc best i nfonnati onabout thcnatureof
theprobl em. The 2D contour curves identifythenei ghborhood of the10伺 l rni ni mum.
百 legradi ent vectors i ndi catc 山巴 di recti onof 出 efuncti ons steepest risea1出巴 poi nt ,
so戸 松 s and val l cys canbc di sti ngui shcd. The contou悶 thernsel ves canbecol ored
wi thout bei ngfillcd.
Thc qui ver plp1shown in Fi gure 2.4also provi des a mechani sm10i ndi cate 白 e
feasible regi on when deal i ng wi th i nequal i ty constrai nts si ncc they i ndi catc 出 e
di recti oninwhi ch由巳 constrai nt functi onwiII incr,伺 se. If several functi ons arebci ng
合 司 wn,白 enthc cI utter produ田 dby 白 earrows may dぜfuse白 ecI arity. The user is
encouragcd10usethe.powerful graphi cal features of MATLAB tohisbenefi t at all ti mes
wi thout l osi ngsight of 出 eobj ecti veof his cffort. MATLAB graphi cs has many more
features thanwill becoveredin出 is chaptcr. The expos山 冠 inthis chapter shoul dbc
sufficient for 血 E隠 晶 derωconfi dentl yexpl oremany other useful graphiω ∞mmands.
Thc 3D mesh and surfaceplots have l i mi teduseful ness. Theseplots canbeuscdto
rei nforcesome of thefeatures foundinFi gure2.4. The i nfonnati onintheseplots may
bei mproved by choosi nga camera angl e白 紙 emphぉl ZCS釦 measpect of thegraphi 回 I
descri pti on. Thisexpl orati onisleft to出 C陀 ader asanexerci se. Usi nghel pvi ewinthe
MATLAB Command wi ndowshoul dget you startcdin出 is di 陀 ction.
2. 3. 2 Exampl e 2. 3
百 lenext exampl e is a compl ex one f romstructural engi neeri ngdesi gn白 紙 is rel evant
inci vi l l mechani cal l aerospacecngi n田 ri ngappl i cati ons. It appeared as a probl cmin
Refercnce 4. It is dcvel oped indetail here. The probl emis toredesi gnthe basi c tall
fl agpol c invi ewof thephenomenal i ncreasc inwi nd spceds duri ngextreme weather
condi ti ons. In recent catas佐' Ophi c events. the wi nd sp田 ds in tomadocs have bccn
measured at over 350 mi l es per hour. Thesehi ghspecds app伺 r tobe thenor mrather
曲 叩 佃 unusual cvent .
Desl gn Pr obl em: Mi ni mi ze the mass of a standard IO- m tubul ar fl agpol c to
wi thstandwi nd gusts of 350 mi l es per hour. The fl agpol ewi¥l be made of structural
steel . Use a factor of safetyof 2.5for 白 c structuraJ desi gn. 百 lCdefl ecti onof thetop
of thefl agpol eshoul dnot exceed 5 cm. The probl emis descri bedinFi gure 2.8.
Mat hemat i cal Model : The mathemati cal modcl is devel oped in detail for
compl etencss and to provi de a rcvi ewof useful s凶 ctural (5) and aerodynami c
rel ati ons [6). The rel ati ons areexprcssedinori gi nal symbol s rather thaninstandard
format of opti mi zati onprobl ems toprovi de an i nsi ght intoprobl emformul ati on.
Desl gn Paramet ers: Thc stnl ctucal stcel [5] has thefol1owi ng materi al constants:
E (modul us of clasticity): 200 E叫 9Pa
σ011 (al l owabl c nor maJ s岡崎) : 250 E+06 Pa
1 1(al l owabl eshcar stress): 145 E+06 Pa
8 m
円 gure2.8 Fl agpol edeslgn: Exampl a 2.3.

s8ctl on A, ・A
'Y(materi aJ densi ty): 7860 kgl m
FS (factor of safety): 2.5
g (gravi tati onal accel erati on) = 9.81 ml s2
For theacrodynami c cal cul ati ons thefol l owi ngareconsi dered:
p (standardair densi ty): 1. 225kgl m
( 合唱 coeffi ci ent of cyl i nder): 1.0
Wp (flagwi nd l oadat 8 m): 5000 N
Vw (wi nd speed): 350 mph (156.46 ml s)
Thegeome凶 cparameters arな
Lp: thel ocati onof flagwi nd l oad(8m)
L: l eng白 of thepol 巴( 10m)
OlIl l : permi Ued defl ecti on(5cm)
Desl gn Vali俗 bl es: The desi gnvari abl es shown inFi g2.8are
do: outsi dedi ameter (XI ) [ Note: x's arenot usedin山 emodel ]
dj : i nsi dedi ameter (.x2)
Geomet ri c Re伺 uons: The fol1owi ng rel ati ons will be useful inlater caJ cul ati ons:
A: areaof cross却 cti on= 0. 25吋 *   d ~ - d f )
1: di ame甘 ical moment of inertia=π * d ~ - dt)/ 64
Qlt: first momcnt of areaabove theneutral 蹴 is di vi dedby thi ckness
= d ~ + dodi + dr ) / 6
Obj ecti ve Functi on: The obj ecti ve functi on is the w巴 i ght of the I O- muni form
fl agpol e:
Wei ght:j{xl' X
): L * A * 'Y* g
Constra的 t Functi ons:・ 百 le wi nd l oad per uni t l ength ( F
) on the fl agpol e is
caI cul atedas
Fo = 0.5* P * ~ * C
* d
Thc bendi ng moment at thebase of thepol edue to白 is uni f ormwi nd l oadon theenti 問
pol eis
Mw=0. 5 * Fo・L * L
The bendi ng moment due tothewi nd l oadon thef1ag is
Wn"' L_
F- " " " P A JD
Bendi ng (nonnal ) stress at thebase of 白 epol eis
σ'bcnd= 0.5ホ ( Mw+M
) 場 di l
Nonnal s位 ess due tothewei ght is
σweight = 'Y事 g 地 L
Tota1 nonna1 sU芭 sses to be resi sted for desi gn is 白 e sumof 由巳 nonna1 stresses
comput ed above. Incorporati ng the factor of safety and the a1l owabl e stress f rom
materi a1va1ues, thefirst in叫 ua1i ty∞nsU湿 int can be set up as
  σ +σwci ght S σnU
I FS (2.11)
The maxi mumshear l oadinthecross secti onis
s= WF +F
' " L
The maxi mumshear stress inthepol eis
τ=S * QI (I 取 t)
The second i nequa1ity constrai nt bωed on handl i ng由 e油 田 r stresses inthef1agpol e
, , τ τ (2.12)
百 le third practi ca1coostrai ot is based 0 0 the def1ecti oo of the topof 血 e pol e.τ 'hi s
d巴fl ecti oodue toa uni f onn wi nd l oadon thepol e is
ow= Fo 01<L
/(8場 E * 乃
The def1ecti onat thetopdue tothef1ag wi nd l oadat ら is
= (2ゆ W
'" L
_ W

L ' " L ' " Lp)/(E'"乃
The thi rdcoostrai ot transl ates to
C3( X

) ow+ O
S O.
(2. 13)

To di scourage sol uti ons where d
< dj , we wi l l i ncl udea geometri c constrai nt:
84( X
, X
) d, 。ー叫ミ 0. 001 (2.14)
Si de Constrai nts:・ 百 lis defi nes thedesi gnregi onfor thesearch.
2 c mS d
S 100 cm; 2 c mS d
S 100 c m (2.15)
MATLAB Code: The m-fi l es for this exampl e are gi ven bel ow. An i mportant
observati onin出 is probl em, and structural engi neeri ngprobl ems inparti cul ar, is 白巴
order of magni tude of the quanti ti es in the constrai ni ng 珂 uati ons. 百 le stress
coos釘泊 nts areof 出 eorder of 108+06, whi l e thedi spl acemeot l enns areof 山 eorder
of 108-02. Most numeri ca1techni ques s凶 ggl etohandl e this range. Typi ca1ly, such
probl ems 0田 d to be nonna1i zed before bei og soI ved' It is essenti a1 in appl yi ng
numeri cal techni ques used inopti mi zati on.
Thi s exampl e is pl ottedint wo p紅 15. Inthefirst part, each i nequa1ityCOI 首 位 置 int is
i nvesti gateda1one. Two curves areshowlI for each cons甘 凶 nt. Thi s avoi ds clutter.τ 'he
si defor thel ocati onof thehash mar k 0 0 theconstrai nt is detenni ned by drawi ng these
C町 Vωi n col or. 百 le bl ue col or i ndi cates the feasi bl e di recti on. Al tematel y, qui ver
pl ots can be used. The second part is 山 econsol i datedcurve shown. The consol i dated
curve has to be dr awn by removi ng the comment s on some of the code 加 d
comment i ng the∞de出 at is not needed. Si nce出 eopti ma1sol uti oncannot be c1earl y
establ i shed,山巴 zoomfeatureof MATLAB is used tonarrowdown thesol uti on.
皐芯2. : . J . . . . 且ー ( t he mai n scr i pt f i l e)
も Chapt er 2: Opt i mi zaヒi on wi t h MATLAB
も Dr . P. Venkat ar aman
も Exampl e 2. 3 Sec. 2. 3

も gr aphi cal sol ut i on u6i ng MATLAB ( t wo desi gn var i abl es)
苦 Opt i mal desi gn of a Fl ag Pol e f or hi gh wi nds
告 Ref . 2. 4
者ーー' ・ーーー・岨ー・ーーーー- - - - ・ーー・ーーー・ーー・ー司司・・ーーーー- - - - - ーーーーー・・"
も gl obal st at ement i s used t o shar e same i nf or mat i on
% bet ween var i ous m- f i l es
も- ー- - - - - - - - - - ーー・・・ーーーーーーーーー" ・・ー- ーー・ーーー- - - - ーーーーーーーーーーー"
も I ni t i al i ze val ues
ELAS 200e+09i も Modul us of el ast i ci t y - Pa
SI GALL = 250E+06i % al l owab工e nor mal st r ess - Pa
TAUALL = 145e+06i 告 al l owabl e shear st r ess - Pa lI l i 叩 肺 判 明 判 明 附 問 問 削 7岬 榊 糊 州 側 刷 叫 州 出 印 刷 山 品 川 III 剛 山 市 酔 一 一 一 1
GAM= 7860;
FS 2. 5;
GRAV 9. 81;
者 ー ー ー ー ー ー ・ ー ー ー ー ー ー ー ー ー ー ー -
R H O l . 225;
CD l . 0;
FLAGW 5000;
SPEED 156. 46;
者 ー ー ー " ー 同 情 ー ・ ー ー ー ー ー ー ー ー ー -
LP 8;
L 10;
DELT 0. 05;
告 ー - ー ー ・ ー ー ー ー ー ー ー ー ー ー ー ー -
gl val SI GALL/ FS
g2val TAUALL/ FS
g3val DELT
g4val 0. 001
者 densi t y of mat er i al ・ kg/ m3
% f act or of saf et y
告 gr avi t at i onal accel er at i on
者 densi t y of ai r ・ kg/ m3
% dr ag coef f i ci ent
者 concent r at ed l oad on f l ag - N
老 町 I / S
者 l ocat i on of dr ag l oad on f l ag - m
も l engt h of pol e
者 al l owabl e def l ect i on - m
も r i ght hand si de val ues
告 f or t he const r ai nt s
者 - - 喧 ・ - - ー ・ ・ ー ・ ー ー ー ー ー ー ー ' ー ー ー ー ー ー ー ー ー 司 ー ・ ー ー ー ー ー ー ー ー ー ー ー - ー ー ー ー ー - - - - -
x工田 0. 02: 0. 01: 1; 者 t he semi - col on at t he end pr event s
告 t he echo
x2=0. 025: 0. 01: 1; 告 t hese ar e al so t he si de consヒr ai nt s
t x1 and x2 ar e vect or s f i l l ed wi t h number s
も not e a way t o avoi d x1 x2
[ X1 X2] meshgr i d{x1,x2) ;
宅 gener at es mat r i ces X1 and X2 cor r espondi ng t o
も vect or s x1 and x2
f 1 obj _ex3( X1,X2) ;
者 t he obj ect i ve f unct i on i s eval uat ed over t he ent i r e
者 mesh
者 Const r ai nt s ar e eval uat ed
i neq1 i neq1_ex3{X1,X2) ;
i neq2 i neq2_ex3( X1,X2) ;
i neq3 i neq3_ex 3 {X1,X2) ;
i neq4 i neq4_ex3( X1,X2) ;
[ C1,han1] cont our {xl ,x2,f 1, [0,10000,50000,
100000, 150000, 200000, 250000, 300000] ,' g- ' ) ;
cl abel ( C1,han1) ;
set ( gca, ' xt i ck' , [0 0. 2 0. 4 0. 6 0. 8 l . O] )
set ( gca, ' yt i ck' , [0 0. 2 0. 4 0. 6 0. 8 1. 0) )
xl abel ( ' out si de di amet er ' ,' Font Name' ,' t i mes' ,
, Font Si ze' ,12) ;
yl abel {' i nsi de di amet er ' ,' Font Name' ,' t i mes' ,
, Font Si ze' ,12)
gr i d
告 hol d on
f i gur e も a new f i gur e wi ndow
cont our {x1,x2,i neq1, [ gl val ,gl val ] ,' r - ' ) ;
hol d on
も dr aw anot her cont our at 10者 t he const r ai nt boundar y
cont our {x1,x2,i neq1, [ O. l *gl val ,O. l *gl val ] , ' b- ' ) ;
set {gca, ' xt i ck' , [0 0. 2 0. 4 0. 6 0. 8 l . O] )
set {gca, ' yt i ck' , [0 0. 2 0. 4 0. 6 0. 8 l . O] )
xl abel {' out si de di amet er ' ,' Font Name' ,' t i mes' ,' . . .
Font Si ze' ,12) ;
yl abel {' i nsi de di amet er ' ,' Font Name' ,' t i mes' ,' .
Font Si ze' ,12) ;
hol d of f
gr i d
も t he f ol l owi ng code may be usef u1 f or t he consol i dat ed
者 f i gur e. Not used her e _ _ uncomment bel ow
者 [C2,han2] cont our {x1,x2,i neq1, [ gl val , gl val ] ,' r - ' ) ;
者 cl abel {C2,han2) ;
者 set {h2,' Li newi dt h' ,1)
苦 k2 gt ext ( ' gl ' ) ;
者 set {k2,' Font Name' ,' Ti mes' ,' Font Wei ght ' ,' bol d' ,.
者 Font Si ze' ,14奄 , , Col or ' , , r ed' )
f i gur e
cont our ( x1,x2,i neq2, [ g2val ,g2val ] ,' r - ' ) ;
hol d on
cont our ( x1,x2, i neq2, [0. 1*g2val , 0. 1 *g2val ] , , b- ' ) ;
set {gca,' xt i ck' ,[O 0. 2 0. 4 0. 6 0. 8 l . 0) )
set {gca, ' yt i ck' , [0 0. 2 0. 4 0. 6 0. 8 l . 0) )
xl abel {' out si de di amet er ' ,' Font Name' ,' t i mes' ,
, Font Si ze' ,12) ;
yl abel {' i nsi de di amet er ' ,' Font Name' ,
' t i mes' , ' Font Si ze' ,12) ;
hol d of f
gr i d
者 [C3,han3] cont our ( x1,x2,i neq2, [ g2val ,g2val ] ,' r - - ' ) ;
者 cl abel {C3,han3) ;
も set ( h3,' Li neWi dt h' ,1)
者 k3 gt ext ( ' g2' ) ;
者 seヒ ( k3,' Font Name' ,' Ti mes' ,' Font Wei ght ' ,' bol d' ,
も, Font Si ze' ,14,' Col or ' ,' r ed' )
f i gur e
cont our ( x1,X2,i neq3, [ g3val ,g3val ] ,' r - ' ) ;
hol d on
cont our ( xl ,x2,i neq3, [ 0. 1*g3val ,0. 1*g3val ] ,' b- ' ) ;
set ( gca, ' xt i ck' , [0 0. 2 0. 4 0. 6 0. 8 1. 0] )
set ( gca, ' yt i ck' , [0 0. 2 0. 4 0. 6 0. 8 1. 0] )
xl abel ( ' out si de di amet er ' ,' Font Name' ,' t i mes' ,
, Font Si ze' ,12) ;
yl abel ( ' i nsi de di amet er ' ,' Font Name' ,' t i mes' ,
, Font Si ze' ,12) ;
hol d of f

者 [ C4,han4] m cont our ( x1,x2,i neq3, [ g3val ,g3val ] ,' b- ' ) ;
者 cl abel ( C4,han4) ;
者 set ( h4,' Li neWi dt h' ,1)
告 k4 = gt ext ( ' g3' ) i
も set ( k4,' Font Name' ,' Ti mes' ,' Font Wei ght ' ,' bol d' ,
も, Font Si ze' ,14,' Col or ' ,' bl ue' )
f i gur e
cont our ( x1,x2,i neq4, [ g4val ,g4val ] , ' r - ' ) i
hol d on
cont our ( x1,x2,i neq4, [ 0. 001*g4val ,0. 001*g4val ] ,' b- ' ) i
set ( gca, ' xt i ck' , [0 0. 2 0. 4 0. 6 0. 8 1. 0] )
set ( gca,' yt i ck' , [0 0. 2 0. 4 0. 6 0. 8 1. 0] )
xl abel ( ' out si de di amet er ' ,' Font Name' ,' t i mes' ,
, Font Si ze' ,12) i
yl abel ( ' i nsi de di amet er ' ,' Font Name' ,' t i mes' ,
, Font Si ze' ,12) ;
hol d of f
gr i d
も [ C5,han5] cont our ( xl ,x2,i neq4, [ g4val ,g4val ] ,
者 , b- - ' ) i
も cl abel ( C5,han5) i
も set ( h5,' Li neWi dt h' ,1)
者 k5 gt ext ( ' g4' ) i
者 set ( k5,' Font Name' ,' Ti mes' ,' Font Wei ght ' ,' bol d' ,
者, Font Si ze' ,14,' Col or ' ,' bl ue' )
も t he equal i t y and   const r ai nt s ar e
も not wr i t t en wi t h 0 on t he r i ght hand si de.
も I f you do wr i t e t hemt hat way you woul d have t。
者 i ncl ude [0,0] i n t he cont our commands
1neal ex3..... m ( t he f i r st const r ai nt )
f unct i on r et val i neq1_ex3( Xl ,X2)
者 gl obal st at ement i s used t o shar e same i nf or mat i on
, bet ween var i ous m- f i l es
AREA 0. 25* pi *( X1. ^2 - X2. ^2) i 者 mat r i x
I NERTI A = pi *( X1. ^4 ・ X2. ^4) ! 64i 者 mat r i x
MW 0. 25*FD*L*L. *X1. ! I NERT1Ai
MF 0. 5*FLAGW* LP*Xl . ! I NERT1Ai
r et val MW+ MF + SI GWi 者 SI GWi s added t o al 1 mat r i x
% el ement s
1nea2 ex3. m ( t he second const r ai nt )
f unct i on r et val i neq2_ex3( X1,X2)
者 gl obal st at ement i s used t o shar e same i nf or mat i on
者 bet ween var i ous m- f i l es
AREA 0. 25* pi *( X1. ^2 - X2. ^2) i
I NERTI A pi *( X1. ^4 - X2. ^4) ! 64i
8 FLAGW+ ( FD 合 L) i
o ( X1. *X1 + X1. *X2 + X2. *X2) ! 6. 0i
r et val 8. *0. / 工NERTI A;
i neaヨ白1t : L. m ( t he t hi r d const r ai nt )
f unct i on r et val i neq3_ex3( X1,X2)
者 gl obal st at ement i s used t o shar e same i nf o幻nat i on
者 bet ween var i ous m- f i l es
AREA 0. 25* pi *( Xl . ^2 - X2. ^2) ;
I NERTI A pi *( Xl . ^4 ・ X2. ^4) / 64;
dw FD*L^4. / ( 8*E凶 S合 I NERTI A) ;
df ( 2. 0合 FLAGW*L^3 - FLAGW*L*L*LP) . / ( ELAS*工NERTI A) ;
r et val dw + df ;
i n自 白 4 ex3. m ( t he f our t h const r ai nt )
f unct i on r et val i neq4_ex3( Xl ,X2)
r et val Xl ・ X2;
Fi gure 2. 9di spl ays thegraphi cal or consol i datedsol uti onto由巳 probl cm. 百 l ecodc
is avai l abl e泊 l l eabove (by comment i ng thecodes 白 紙 creal c thenewfi gure
wi ndows and removi ng thecomment s on thecodc白 紙 is currcntl ycommentcd). Thc
opti mal sol uti on is not very c1ear. Fi gu問 2.10 is obtai n巴d by zoomi ng in near 出 e
nei ghborhood of 0.6. Sec hel pzoomfor in附 ucti onon its use. lt can be achi cvedby
F1llgPolc戸。 blcm
0.81- φ




0.2 0.4 0.6 0.8
outsidedi amel er
Fl gure2.9 Graphl cal solulion: Exampl e 2.3.
0.58 0.6 0.62 0.64 0.66 0.68 0.7 0.72 0.76 0.8
outsidedi amCl cr
Fi gure 2.10 Graphi cal solutlon(zoomed): 臥 :ampl e2.3.
typi ngzoomat theworkspace prompt and usi ngthemouse todraga rectangl earound
曲 e regi on that n民 ds to be enl arged. Fr omFi gure 2. 10 出 c sol uti on is around the
outsi dc di ameter of 0. 68 m胡 d tllc i nsi dedi ameter of 0. 65 m. Typi ng zoomin出巴
workspace agai nwill toggl e白 白 伊 回 back tononnaJ state. InFi gure 2. 10, 山 tick
marks arepl acedon thefi gurethrough出 ecommand wi ndowfor bcttcr inte中 肉 tati on
of thesol uti on.
The graphi cs in 由 e above ex副 npl e were created usi ng the same statemenl s
encountered previ ousl y. Col or contours were used toestabl i shthefeasi bl eregi on. τhe
zoomfeaturewas empl oy吋 toobtai na bettcr esti matc of thesol uti on.
2. 3. 3 Exampl e 2. 4
This exampl e is 合u m出 earcaof heat transfer. The probl emis todesi gna tri angul ar
fin of 白巴 smal l est vol ume that wi l l at least del i ver speci fi cd fin effi ci enci es. The
graphi cal t i回 tu陀 of this code is very si mi l ar to Exampl e 2.3. ln 山 is cxampl e, the
i nequal i tyconstrai nts arecomput ed and retumed f roma si ngl efuncti l erather
山 an separatc files consi deredin the previ ous exampl e. Another ncwfeal ure in this
exampl e is to i nvoke speci al mathemati cal functi ons, the sesseI functi ons that a問
avai l abl einMATLAB. 百世 sexampl e al soiI1ustr官 民 s a probl emwhere theopti mi zati on
probl emcan be adequatel y defi ned but the sol uti on is easi l y determi ned f romthe
equal i ty and si de cons位 aints. Inother words, the probl emcan easi l y accommodat e
addi ti onal demandi ng constrai nts. Inl argecompl cx mathemati cal model s wi thmany
dcsi gnvari abl cs it is not easy toensure that at least one of thc i nequal i ty constr瓜 nts
I S active.
Des勿 'nPr obl em: M凶 凶 ze由巳 amount of materi al used in出 edωi 伊 of ascri cs
of i denti cal tri angul ar fins that cover a gi vcnareaand operateat or abovc thespeci fi ed
cffi ci cnci ω.
Mat hemat l cal Model : Thi s type of probl emshoul d be abl e to accommodat e
several dcsi gn vari abl es. In this secti on,出 e probl emis set up to have t wo desi gn
vari abl 邸 . 百 1efinmateri al is al umi num.
Desi gn Parameters: For al umi num
h = 50 W/ m
[convecti oncoeffi ci cnt]
k = 177 W/ m- K [thermal conducti vi ty)
N = 20 [ number of fins]
W=O. 1 m [wi d山 of thefins)
Fi ngap is th巴 samc as thebase l engthof thetri angul ar fin.
Desi gn Vari abl es
b: base of thetri angul ar fin
L: hei ght of thc tri angul ar fin
Fi gurc 2.11 iJl us佐 atω 山 egeom巴町 yof 自 己 fins.
Geomet ri c Rel atl ons: Th巴 fol l owi ngaresome of th巴 areacal cul ati ons that areuscd
later inthedcvel opmcnt of 山 ccons佐 aints:
= ( 2N- 1)*b叫 V footpri nt of 出 efinand gap
Ac = 2*W*[L2+ (b!2)2] 112 finareafor heat transfer
Ab = ( N- l)*b*W gap ar伺
At = N*Ac + Ab total areafor heat transfer
Obj ecti ve Functi on: The total vol ume of ma回 ial for thefinis
j(b, L) = O.5*N*W*b*L (2.16)
b b
FJ gure2.11 Flndesl gnfor heat transfer: Exampl e 2.4.
Constrai nt Funct, 幻 ns: Thc heat transfer equati ons for the fin arc avai l abl e in
Reference 7. Fi ns aretypi cal l ymount ed on a speci fi ed紅 白 . 明 1efirst cons回 int is one
on area:
h(b, L): Af = 0. 015m
Note: 山 is constrai nt essenti al l y fi xes theval ueof b. In山 egraphthis constrai nt wil¥
be a strai ght lineparal l el to出 cLaxi s. l f 血 is wcre tohappen ina probl cmwi thscveraI
desi gnvari abl es, it woul d be prudent toel i mi nate出 is vari abl c f romthemathemati cal
model by i denti fyi ngit as a desi gnpar佃 leter.
百 eeffi ci ency of a si ngl efincan be establ i shedぉ
ηr = (l / mL) 1,(2mL)l I
m= (2h / , 肋 )112
叩 d出 el' s areBessel equati ons of thefirst ki nd. The first i ncqual i tyconstrai nt is
g,(b,   : η f ~ 0. 95
The overal l cffi ci ency for theheat transfer is
= l - N*叫ん 4
) ( ト 叩
The final i nequal i tyconstrai nt is
g2(b, L): TJ
0. 94
Si de Constrai nts: The si decons住 ai nts 釘 芭
0. 001 SbSO. ∞5
0.01 S L S 0. 03
MATLAB Code: Si mi l ar to 由 e previ ous exampl e. the hashmarks on the i nequal i ty
cons佐 aint canbe establ i shedby pl otti ngt wo vaI ues of theconstrai nt: thelimit val ue
and a val uehi gher. Once出巴 yhav巴 beendetermi ned. thentheconsol i datedfi gurecan
be establ i shed. Th巴 codeshown herewill drawaI l thefuncti ons inseparatewi ndows.
For theconsol i datedplot this code necds to be edited. It is left to thestudent 錨 m
exercl se.
盟 主 ム mThe mai n script file
もChapt er 2: Opt i mi zat i on wi t h MATLAB
% Dr . P. Venkat ar aman
% Exampl e 2. 4 Sec. 2. 3

者 gr aphi cal sol ut i on usi ng MATLAB ( t wo desi gn var i abl es)
者 Mi ni mumf i n vol ume f or ef f i ci ent heat t r ansf er
も mat er i al Al umi num
者 ー ー ー ー ー ー - - - - - - - - - - - ・ ・ ・ ・ ・ - - - - - - ー ー ー ー ー ー ・ ー ー ー ー ー ー ー - - ーーーーーーーー
者 gl obal st at ement i s used t o shar e same i nf or mat i on
者 bet ween var i ous m- f i l es
gl obal N H K W AREA
者 ・ " ・ - - - - ー ー ー 司 - - ーーーーー- ーーーーーーーーーーーーーーーーーーー一一ーーー・・ーー回・ー- -・
も 工 ni t i al i ze val ues
N 20 告 number of f i ns
W 0. 1 者 wi dt h of f i ns
日 50. 0 者 convect i on coef f i ci ent W/ m*m
K 177. 0 者 t her mal conduct i vi t y W/ m- K
AREA 0. 015 者 avai l abl e f i n f oot pr i nt ar ea
者・・ーーーーーーーーーーーーーーーーーーーーーーーーーー・・ーーーーー- - - - - - - ーーーーーー・ーーー・
者 r i ght hand l i mi t s f or t he f unct i ons
h1val AREA;
gl val 0. 95;
g2val 0. 94;
もーーーーーーーー- - - ーーーーーー・ー・ーー・・ー・・ーー・・四ーー・ーー・・ーーーーーー・・ー・『ーー-
x1=0. 001: 0. 0001: 0. 005;
x2=0. 01: 0. 001: 0. 03; も x1 and x2 ar e vect or s f i l l ed wi t h
者 number s
[X1 X2] meshgr i d( x1,x2) ;
も gener at es mat r i ces X1 and X2 cor r espondi ng t o
も vect or s x1 and x2
f 1 = obj _ex4( X1,X2) ;
も t he obj ect i ve f unct i on i s eval uat ed over t he ent i r e
者 mesh
も Const r ai nt s ar e eval uat ed
! 配
eq1 eq1_ex4( X1,X2) ; 苦 t he equal i t y const r ai nt
[ i neq1, i neq2]   ,
者 i n t he above not e t hat t he t wo i nequal i t y const r ai nt s
者 ar e obt ai ned si mul t aneousl y. Keep i n mi nd
も t hat each const r ai nt i s a mat r i x
者 r eady f or t he pl ot s
[C1,han1] cont our ( x1,x2,f 1, [ 0. 00001, 0. 00002,...
0. 00004, 0. 00006, 0. 00008, O. OOOl ] ,' g・' ) ;
cl abel ( Cl ,han1) ;
set ( gca, , xt i ck' , [ 0. 001 0. 0015 0. 002 0. 0025 0. 003
0. 0035 0. 004 0. 0045 0. 005] )
set ( gca, ' yt i ck' , [ 0. 010 0. 015 0. 02 0. 025 0. 03 0. 125] ) ;
xl abel ( ' f i n l engt h' ,' Font Name' ,' t i mes' ,' Font Si ze' ,12) ;
yl 油 el ( ' f i n wi dt h' ,' Font Name' , ' t i mes' , ' Font Si ze' ,12)
gr i d
% hol d on
f i gur e も open a new f i gur e wi ndow
も p10t t i ng t he equal i t y const r ai nt
cont our ( x1,x2,eq1, [ h1val ,h1val ] ,' r - ' ) ;
set ( gca,' xt i ck' , [ 0. 001 0. 0015 0. 002 0. 0025 0. 003
0. 0035 0. 004 0. 0045 0. 005] )
set ( gca, ' yti ck' , [ 0. 010 0. 015 0. . 02 0. 025 0. 03 0. 125] ) ;
xl abel ( ' f i n l engt h' ,' Font Name' , ' t i mes' , ' Font Si ze' ,12) ;
yl abel ( ' f i n wi dt h' ,' Font Name' ,' t i mes' ,' Font Si ze' ,12)
gr i d
も pl ot t i ng t he f i r st i nequal i t y const r ai nt
f i gur e
cont our (x1, x2 , i neq1, [0. 968 , 0 . 968] , , r - , ) ;
hol d on
cont our ( x1,x2,i neq1, [ 0. 969,0. 969] ,' b- ' ) ;
set ( gca,' xt i ck' , [ 0. 001 0. 0015 0. 002 0. 0025 0. 003
0. 0035 0. 004 0. 0045 0. 005] )
set ( gca, ' yt i ck' , [ 0. 010 0. 015 0. 02 0. 025 0. 03 0. 125J ) ;
xl abel ( ' f i n l engt h' ,' Font Name' ,' t i mes' ,' Font Si ze' ,12) ;
yl abel (' f i n wi dt h' , , Font Nar陪, , ' t i mes' , , Font Si ze' ,12)
hol d of f
gr i d
者 pl ot t i ng t he second i nequal i t y const r ai nt
f i gur e
cont our ( x1,x2,i neq2, [ g2val ,g2val ] ,' r - ' ) ;
hol d on
cont our ( x1,x2,i neq2, [ 1. 01*g2val ,1. 01*g2val ] ,' b- ' ) ;
set ( gca,' xt i ck' ,[ 0. 001 0. 0015 0. 002 0. 0025 0. 003
0. 0035 0. 004 0. 0045 0. 005] )
set ( gca,' yt i ck' , [ 0. 010 0. 015 0. 02 0. 025 0. 03 0. 125] ) ;
xl abel ( ' f i n l engt h' ,' Font Name' ,' t i mes' ,' Font Si ze' ,12) ;
yl abel ( ' f i n wi dt h' ,' Font Name' ,' t i mes' ,' Font Si ze' ,12)
hol d of f
gr i d
nb. Le組担割 leobj ecti vefuncti on
f unct i on r et val obj _ex4( X1,X2)
者 vol ume of t he f i n
gl obal N H K W AREA
r et val 0. 5*N*W*X1. *X2
量ql J 雄. mThe equal i ty∞ 田 佐 直 int
f unct i on r et val eql _ex4( Xl ,X2)
も t he equal i t y const r ai nt on ar ea
gl obal N H K W AREA
r et val ( 2. 0合 N - 1) *W*Xl ;
旭 叫 The constrai nts
f unct i on [ r et l , r et 2]   ,
者 r et ur ns bot h t he i nequal i t y const r ai nt s
gl obal N H K W AREA
c 2*sqr t ( 2. 0) *sqr t ( 日 / K) *X2. / sqr t (Xl ) ;
r et l ( bessel i ( 1,c). . / ( ( 0. 5*c) . *bessel i ( O,c) ) ) ;
Ae 2. 0*W*sqr t ( ( X2. *X2 + . 25*Xl . *X1) ) ;
Ab ( N- 1) *W*X1;
At N*Ae + Ab;
Ar Af . / At ;
r et 2 ( 1. 0 - N *Ar . 合 (1 - r et 1) ) ;
The above script files shoul ddi spl ay each functi onina scparatefi gure wi ndow.
The i nequal i ty constnl i nt fi gures shoul dcontai nt wo contours wi th山 c bl uecontour
i ndi cati ng白 efeasi bl eregi on. Fi g山 な 2.12di spl ays a composi te pl ot The sol uti onis
detcnni nedby theequal i tyconstrai nt and thel ower Ii mi t on白 efi nl eng白 ぉ di scussed
previ ousl y.
1量産 l
. . ,
1.5 2 2.5 3 3.5
円 gure2. 12 Graphl cal solulion: Exampl e 2.4.
This secti on pr ovi dωa bri ef exposure to addi ti onal features of Handl e Graphi cs,
inc1udi ng devel opment of a GUI in MATLAB. The coverage is very modest and is
inc1uded hcre for thc sake of compl cteness as we have accompl i shcd our goal of
graphi cal opti mi zati on 泊 山 e previ ous secti ons. Thi s secti on can be avoi ded,
postponed, or emphasi zed for the proj ects in 也 e course, as it has no addi ti onal
i nformati on for graphi cal opti mi zati on. It does contai n reasonabl c i nfonnati on in
undcrstandi ngMATLAB graphi cs and programmi ng. The reader is encouragedtotake
somc ti megetti ngfami l i ar wi th出 eversi onof MATI . ABi nstal l edon his machi ne. There
are usual l y si gni fi cant enhancements in newcr versi ons of MATLAB. For exampl e,
MATLABV巴rsi on5. 3(出 is book is basedon Versi on5.2) al l ows you tocustomi ze your
plot and add annotati ons toit 白 roughaddi ti ona1edi ti ngt i回 1町 民 of 由 巳 自 仰 向 wi ndow.
You do not have tocustomi ze your fi gurethroughcode al one. Inthis secti on, we will
a1so be devel opi ng the GUI usi ngMATLAB Handl e Graphi cs so 白 紙 we have more
experi en白 wi 血 MATI . ABHandl e Graphi cs. MATI . ABdoωprovi dc a GUI creati ontool ,
gui de,出 at theuser is recommended toexpl ore.
Intheprevi ous secti ontherewere onl y two desi gn vari abl es. An cq回l I y spaced
vector was obtai nedfor eachvari abl c basedon由巳 si dc constrai nt. A m巴shgri dwas
出 engenerated. Al I functi ons werc eval uatedon this grid. Contour pl o15were drawn
for thefuncti ons representi ng出巴 obj ecti veand constrai nts. Except for theobj ecti ve
functi on whi ch di spl ayed several contours, the constrai nts were onl y drawn for a
si ngl eval uc. Thi s procedure suggests thepossi bi l i tyof automati on. User i nput will be
necessary for sel ccti ng出 巴 悶 ngc for thevari abl es as wel l as thc functi ons 出 at will be
pl otted.
Inthis sccti on, we will createpl otti ngfacility toi ncl udea GUI that wi l l
• Obtai n白 enumber of pl o15
. The range and i ncrcment of thedesi gnvari abl es
• Use a filesel ecti onbox/ utility出 at theuser will us巴 to i denti fy 出 efuncti onto
be pl otted
• Prompt theuser for thenumber of contour百 for each pl ot
・Al l owsel ecti onof contour l evel s
・Creatc thepl ot
百 leexcrci sewi l l requi rc t wo m-fi l es: one 10set up出 eGUI and山 eo白巴 r toassist in
thepl otti ng. The user will be abl etotweak thc pl ot intheCommand wi ndowafter it
is drawn. The fol l owi ngsubsecti ons pωsent a bri ef i ntroducti ontoHandl c Graphi cs
and GUI control s inVersi on 5.3.
2. 4. 1 Handl e Gr aphi cs
MATI.. AB Handl e Graphi cs refer to a col l ecti on of l ow-l evel graphi c routi nes that
actual l ygeneratc or carryout graphi cal changes you s田 i nt hcMA百 . ABfi gurewi ndow.
For themost part theserouti nes aretransparent 阻 dtypi cal l yusers nced not be awarc
of them. Infact , tounderstandopti mi zati onand toprogramthetechni ques inMATI.. AB,
it is not necessary to knowabout Handl e Graphi cs si nceusi ngthehi gher-l evel pl ot
commands such as pl ot and contour is suffi ci ent toget thej obdone. "Thesc commands
actual l y ki ck in scvcral of 白 e Handl e Graphi cs routi nes 10 di spl ay 白 e pl ot on the
fi gure. A notewor註ly feature of MATI ..AB graphi cs is 白 紙 it is i mpl emented 泊 佃
obj ect圃 ori entedmanncr. What this means is that most of the印 刷 es you seeon the
fi gurc, Ii ke axes, l abel s, tcxt , and lines, arc all obj ec15. This mcans that most of the 蜘
graphi cal i tems on the fi gurehave properti es 白 紙 C組 bechanged出 roughprogram
code. Intheprevi ous secti ons we used this tomani pul ate col or, font si ze, lin巴 wi d白 ,
and theIi ke.
Inorder to change出 eprope凶 es of obj ects, it is nec凶 sary to i denti fy or refer to
thcm. Obj ects inMATLAB arei denti fi edby a uni que number ref erredωas a handl e.
Handl es can be assi gnedtoall obj ec15when they arc crcatcd. Some obj ec15紅 巴 partof
other obj ccts. Obj ects that contai nother obj ec15arc referredtoas contai ner or parent
利 回 民 . There is a defi ni tc hi erarchi 叫 s回 cture among obj ects.τ 'he root of all
MATLAB graphi c 0切 回 15is 出 efi gureobj 田t . The fi gure0同ect for exampl e can hol d
sevcral GUI control s, whi ch areknown as ui control obj ec15. It 31socontai ns the蹴 es
obj 田 t . 百 巴 回 凶 obj ωt in t umhas the line, text, and i mage obj ect. Al l 0対ects in
MATLAB can be associ atedwi 白 aset of properti es 白 at is usual l y based on i15functi on
or useful ncss. Di fferent obj ect types have di fferent sets of properti es. Each of thcs巴
prope凶 es is descri bcdthrougha pai r of rel atedi nformati on(name, value). 百 efirst
el ement of this pai r is the namc for the property and the second is the val ue
correspondi ngto恥 property. When出 eo同ect is created, it i nheri ts all of thedefaul t
prope凶 es. For exampl e:
pl ot ( ヒ , si n( 2*ヒ) ,' - go' ,
, Li neWi dヒh' , 2,
, Mar ker EdgeCol or ' ,' k' ,
, Mar ker FaceCol or ' , [ 0. 49 1 0. 63] ,
, Mar ker Si ze' ,12,)
The last four 1i nes abovc represent four properti es of 出 e pl otted line that is bei ng
changed. The property narn凶 on山 e Icft can be easi l y understood. 百 leformat si ze
uni ts areinpoi nts. The col or val ue is a r owvcctor of 出 向 巴 col or val ues between zero
and onc representi ngthered, green, and bl ueval ues, respecti vel y. There aresevcral
oth巴r prope而 es of the pl otted line whi ch will be sct at thcir defaul t val ues. Thc
MATLAB onl i nercferenceshoul d Ii st all obj ec15and thcir prope凶 es.
In general . obj ect properti es cllnbe changed by usi ng theser functi onto change
their val ues. Si mi l arl y, obj ect properti es c佃 bercadby theuse of theget functi on. To
i nqui reabout theproperty of an 0附田t :
get ( handl e,' Pr oper t y Name' )
geヒ ( h_1,' Col or ' )
Intheabove, h_l is 出 ehandl e of theobj ect whose col or is desi rcd. Si mi l arl y,
seヒ ( handl e, ' Pr oper t y Name' , ' Pr oper t y Val ue' )
set ( h_l ,' Col or ' ,' r ' ) or set ( h_1,' Col or ' , [1 0 0] )
intheabove出巴 val ueof Col or for theobj cct represcnt叫 by thehandl c h_l is set 10
rcd in t wo ways. The sccond speci f i ωthe col or red through the vector val uc. It is
expectcdthat theobj ect i denti fi edthroughh_1 shoul dhave a Col or property. MATt. AB
wi l l i nf ormyou of thc error incase you areassi gni nga property/val ue伽 tisnot val i d
for theobj ect.
2. 4. 2 Graphl cal User I nterf ace
The most si gni fi cant featurc of software desi gn and devel opment today is the
obj ect-ori enteddesi gnparadi gm白 紙 was bri efl y iII ustrated in 出 eprevi ous secti on.
The combi nati on of theabove wi thevent-dri venprogrammi ng is botha natural and a
陀 qui redfcaturc incurrent softwarc programs. I nsteadof theprograms control l edby
a predetermi ned processi ng s叫 uence (batch processi ng), today' s processi ng is
govemed by the user through some dirl田 ted mouse cl i cks and buttons. These user
sel ecti ons cause certai nevents that i nstruct thesoftwaretocarryout some acti on. The
o句 民 俗 伽 1fire 血 e evenωar e usual l y referred 10 ぉ user inte巾 叩 el cmc附 . It is
i mportant to understand that thc same evcnts can be tri ggered by obtai ni ng user
responsc to sc隠 en prompts al the command linc (unattracti vc). Usi ng buttons to
achi eve the same resul t i5 more gl amorous al though it uses mor e compu旬 ti on叫
陀 sources. l nfact , for commerci al success it is n配 essarytoi ncl udethis mechani smof
user inte. racti oninthesoftware product . Al so. users expect theseel ements 10behave
in a standard way. There are many el ements 白 紙 the user can use to i nteract wi th
program5. Many appl i cati ons across several di sci pl i nes havc nowdefi neda mi ni mum
5tandardcol l ecti onof 出 eseel ements. MATLAB必 50provi dcs them. The organi zati on
of 山 eseel emenl s and their presentati ontotheuser ina graphl cal manner is i denti fi ed
as 出 egrョphl cal user i nterfa田 . A number of GUI - based t∞1s. 釦 ncti ons. 催 芭 avai l abl e
at theMATLAB site.
Thi s secti onprovi des onl y a bri ef i ntroducti onto出 eel ements used intheexampl e.
Th巴seel cments arepri mari l y usedthroughMATLAB Handl c Graphi cs programmi ng.
Consi der thecasc of a button白 at theuser can push tocause some acti on. referredto
as a push buttoninthc subsequent di scussi on. At theoutset. thereru官 two pri nci pal
requi remcnts 山 at arecxpectl 叫 of thepush button. Fi rst , the凶 er must be abl etosee
it. Second. whcn出 euser cl i cks on it (si ngl eversus doubl e cl i cks can be di sti ngui shed
inmany software devel opment codes) thebuttonmust 飽 spondwi 出 someexpected
acti on. InMA TLABthepush buttonis a graphi cal ot羽田t. It has many prope両 国 thatcan
be used todescri beits app回 ranceon thc screcn. InMA'τLAB. all graphi cal obj ects can
onl yappe釘 in a fi gurc wi ndow. Hencc, a fi gure wi ndowis essenti al to di spl ay thc
push button. 百 is fi gure wi ndowact sωa contai ner and can contai n many user
i nterface el cmcnts. Consi deri ng the presentati on of the button on thc scrccn, it is
ne∞ssary todcci de where thebuttonshoul dapp回 r on出 ef i gw沼 wi ndow,how bi git
shoul dbe, what l abel it shoul d have (typi cal l y i ndi cates 血巴 ki ndof acti onit is l i kel y
ωcause) , 加 d so on. Si ncc the push button is a graphi cal obj ect , much of these
prope凶 esc叩 beset if therc is a handl e avai l abl efor I hepush-buttonobj cct. Once the
graphl 伺 1appearance is tak
Hpush ui cont r ol ( Hf i g_1,' St yl e' , ' push' ,
' Uni t s' ,' i nches' ,

' Posi t i on' , [ 2. 5,0. 1,0. 7, . 2] ,
' Backgr oundCol or ' ,' r ed' ,
, St r i ng' , , EXI T' ,
' Cal l back' ,' cl ose' ) ;
Intheabovc Ii nes of codc. Hpush is thehandl e tothepush-buttonobj ect . It is created
by usi ngthefuncti onui control . Thi s is thefuncti onthat wi l l createtheuser i ntcrface
el ements. The same functi onis used topl aceseveral di 庄erent user i nterfaceel ements
through thesel ecti onof thestyleproperty. Intheexampl e bel ow. thesei ncl udelabel
boxcs and user edi tabl etexl boxes. Several parameters i denti fy Lhe typ巴 of 巴 l emcnt
UEat is createdand theproperti es UEat need to beset. Each propeny is set througha
propertynamel propertyval ue pair. Hf i ι l i n 恥 I i ncs of code abovc 児 島 内 to the
handl e of thefi gurewi ndowinwhi ch this el ement wiUbe pl aced. The typeof el ement
createdis a push-buttonel ement cstabl i sh吋 through恥 Styl eproperty s凶 ngname
‘push. " The Uni ts used to si zeand posi ti onthepush buttonis i nches. The Posi ti on
property is a r owvector of fo町 val ues gi vi ngtheLeft. Bottom. Wi dth. Hei ght of the
el ement intheUni ts scl ectedearlier. The defaul t l ocati onof theori gi nis thebottom
lcft comer of thefi gu問 wi ndow. The BackgroundCol or for thebuttonis sct tored. Thc
l abel that wil¥be on thefaceof thepush buttonis EXI T. Thi s is theTCXl property of
thepush-buttonobj ect . Thc Cal l Back property of thepush buttonis 白 estri ng“cl ose."
This stri ngwill be thc parameter intheeval functi on出 at is cal l edwhen出巴 pushl cl i ck
event is tri gge陀 d. MATLAB command cl ose wi l l be executed in the Command
workspace when the push button is p問 ssed. Thl s will cause l he fi gure wi ndow
di spl ayi ngthepush buttontoc1osc.
Onl y a f ewel ements areused in出 is sccti on. The Cal1Back stri ngs areal soqui te
s回 i ghばor w副 . Bef ore dcvel opi ng thc code. it is useful to observe 恥 hl erarchi cal
order in whi ch thegraphi cal obj ects in MATLADareorgani zed(reproducedf rom血 e
MATLAB documentati on) inFi gure 2.13. The root i mpl i es thestarti ngresourcefor aIl
graphi c di spl ays inMATLAB. Its handl e usual l y has a val ueof O. The root al soserves
as a contai ner for all of thefi gurewi ndows. Another exprcssi onfor 山 is rel ati onshi p
is that a fi gurewi ndowis thechi l dof theroot . Fr omthehi erarchy it isessenti al ωnot e
白 紙 出 e user i nterfaceel ements. ui conヒr ol or ui menu, c叩 onl y exi st wi 出 in a
Ul control Axes Ul menu
I mage Ll ne Pat ch Surf ace Text l i ght
F1gure 2.13 Hlerarchical slruclurelor MATl AB graphl cal objecls.
figurewi ndow. 80 al50the蹴 e5obj ect . Inthefol l owi ngcode eachinterfaceel ement
is first associ atedwi 白 ahandl e50出 創 出 epropertyof theel ement canbe陀 fen吋 to
if necessary. Thi s is especially trueof thetext entry boxes to i nvok巴 theCal l 8ack
s凶 ng5.
2.4.3 GUI Code
百 lefol l owi ngcodewil1generatetheGUI. The ui wai t 陀 quires thal 山 is wi ndowbe
cl osedbeforesubsequent processi ngcantakepl ace(or ui r esume beuSedin白 E
∞de) .
. Gl l 旦. . yl
も GUI f or Pl ot t i ng f aci l i t y
も Dど . P. Venkat ar aman Appl i ed Opt i mi zat i on Usi ng MATLAB

% Ch. 2 Sec. 2. 4

者 GUI i s cr eat ed wi t hi n a f i gur e wi ndow
告ーーー- - - - - - ーーーーーーーーーーーーーーーー- ーーーー司ーーーーーーー・・・・ーーーーーーーーーー
Hf i g_1 ' " f i gure; 者 Hf i g_1 i s t he handl e t o t he
も f i gur e wi ndow
set ( Hf i g_l , ' Col or' , [0. 3,0. 6,0. 5] , . . . も set col or
, Number Ti t l e' , ' of f ' , 告 no wi ndow t i t l e
, Name' , ' Set Range f or Desi gn var i abl es' ,
' Uni ts' ,' i nches' , 者 i nches used t o l ayout
% ot her cont r ol s
' Menubar' , ' none' ,
' Posi t i on' , [4,4,4. 4,1. 5]);
告 Posi t i on pr oper t y i s [l ef t , bot t om, wi dt h, hei ght] i n
者 Uni t s sel ect ed
Ht _1 ui cont r ol ( Hf i g_1,
, St yl e' , , t ext ' ,
' Uni ts' , ' i nches' ,
' posi t i on' , [0. 1,1. 1,2. 4, .2] ,
' St ri ng' ,' Number of Funct i ons t o Pl oヒ , ) ;
























, Backgr oundCol or ' ,' whi te' ,
, St ri ng' , ' ,
' Cal l back' ,' Np st r 2nUm( get ( Ht _2," St r i ng" ) ) i ' ) ;
HL X1L ui cont r ol ( Hf i g_l ,
' St yl e' ,' text' ,
' Uni ts' ,' i nches' ,
' Posi t i on' , [0. 1,0. 8,0. 5, .2] ,
' St ri ng' ,' x1( mi n) ' ) i
HT_XI L ui cont r ol ( Hf i g_l ,
' Styl e' ,' edi t ' ,
' Uni ts' ,' i nches' ,
' Posi t i on' , [0. 7,0. 8,0. 7, .2] ,
' Backgr oundCol or ' ,' whi te' ,
' St ri ng' ," ,
' Cal l back' ,' xl mi n st r 2num( get ( HT XI L,'
, St r i ng' , )) i ' ) j ー
HL_XI M ui cont r ol ( Hf i g_1,
' Styl e' ,' text' ,
' Uni ts' , ' i nches' ,
' Posi t i on' , [1. 5,0. 8,0. 6, .2] ,
' St ri ng' , ' i ncr emenヒ' ) i
HT X1M ui cont r ol ( Hf i g_l ,
' St yl e' ,' edi t ' ,
' Uni ts' ,' i nches' ,
' Posi t i on' , [2. 2,0. 8,0. 7, .2] ,
, Backgr oundCol or ' ,' whi te' ,
, St ri ng' , , ,
' Cal l back' ,' x1i nc st r 2num( get ( HT_X1M,'
, St r i ng' , )) j ' ) j
HL_X1U ui cont r ol ( Hf i g_1,
, St yl e' , , t ext ' ,
' Uni ts' ,' i nches' ,
' Posi t i on' , (3. 0,0. 8,0. 5, .2] ,
' St ri ng' ,' xl ( max) ' ) j
HT_X1U ui cont r ol ( Hf i g_l ,
' Styl e' , ' edi t ' ,
' Uni ts' ,' i nches' ,
' Posi t i on' , (3. 6,0. 8,0. 7, .2] ,
, BackgroundCol or' ,' whi te' ,
, St ri ng' , , ,
' Cal l bacl 屯 , ,' x1max st r 2num( geヒ ( HT_X1U,,
, Stri ng' , ) ) ; , ) ;
E江, _X2L ui cont r ol ( Hf i g_l ,
, Styl e' , , text' ,
' Uni ts' ,' i nches' ,
' Posi t i on' , [0. 1,0. 5,0. 5, .2) ,
' stri ng' ,' x2( mi n) ' ) ;
HT_X2L = ui cont r ol ( Hf i g_l ,
, St yl e' , , edi t ' ,
' uni ts' ,' i nches' ,
' Posi t i on' , [0. 7,0. 5,0. 7, .2) ,
' BackgroundCol or' ,' whi te' ,
, St ri ng' , , ,
' Cal l back' ,' x2mi n st r 2num( get ( HT_X2L,'
, St ri ng' , ) ) ; , ) ;
HL_X2M ui cont r ol ( Hf i 9_1,
, St yl e' , , text' ,
' Uni ts' ,' i nches' ,
' Posi t i on' , [ 1. 5,0. 5,0. 6, .2) ,
' Stri ng' ,' i ncrement ' ) ;
HT_X2M ui cont r ol ( Hf i g_1,
, Styl e' ,' edi t' ,
' Uni ts' ,' i nches' ,
' posi t i on' , [2. 2,0. 5,0. 7,.2) ,
' BackgroundCol or' ,' whi te' ,
' Stri ng' , ' ,
' Cal l back' ,' x2i nc st r 2num( get ( HT_X2M,'
, St ri ng' ,) ) ; ' ) ;
HL_X2U ui cont r ol ( Hf i g_1,
' Styl e' , ' text' ,
' Uni ts' ,' i nches' ,
' Posi t i on' , [3. 0,0. 5,0. 5,.2] ,
' stri ng' ,' x2( max) ' ) ;
HT_X2U ui cont r ol ( Hf i g_1,
' Styl e' , ' edi t' ,
' Uni ts' ,' i nches' ,
' Posi t i on' , [3. 6,0. 5,0. 7, .2] ,
' BackgroundCol or' ,' whi te' ,
, St ri ng' , ' ,
, Cal l bacl 屯 , ,' x2max st r 2num( get ( HT_X2U,'
, St ri ng' , ) ) ; , ) ;
Hl abel ui cont r ol ( Hf i g_1,
, Styl e' , , text' ,
' Uni ts' , ' i nches' ,
' posi t i on' , [0. 1,0. 1,2. 2, .21 ,
' BackgroundCol or' ,' y' ,
' Stri ng' ,' Af t er ent er i ng val ues pl ease pr ess
EX工T' ) ;
Hpush ui cont r ol ( Hf i g_1,' Styl e' ,' push' ,
' uni ts' , ' i nches' ,
' Posi t i on' , [2. 5,0. 1,0. 7, .21 ,
, Backgr oundCol or ' ,' red' ,
, St ri ng' , ' EXI T' ,
' Cal l back' ,' cl ose' ) ;
ui wai t
Intheabovecode出 estatemenlSshoul dbecωY tointerpret . Much of thecodec叩 be
edit凶 after copy andp邸 teoperation. The last command ui wai t instruclSMATLAB
tohol doff executi onuntil theuser closes 出 eGUI wi ndow. Fi gu陀 2.14 shows the
i mage of theGUI ぉ i tapp回 rsinMATLAB. 百 letext boxes arcinitiali z.edwi thno val ues
forcing 血 e user to set values 出 ereby causi ng al1 of the Cal l Back strings to be
evaluated. Inaddition, th巴 dataenteredby theuser isa s甘 ing. It has tobeconvertedto
a number. The ∞dehe印 is basic. It does not verify也 at theuser didenter a number.
百 is verification can be done by extendi ng 血 e Cal l Back string over mul ti pl e
statemenlS. Thi s is left ぉ anexerciseinprogrammi ng for theuser.
The script m-fileusedtoselect 佃 dcreatethepl0lSis as follows.
目 盛 m. m
者 Scr i pt f i l e t o go wi t h Cont our pl ot t i ng
宅 Dr. P. Venkat ar aman, Appl i ed Opt i mi zat i on Usi ng MATLAB
- ・ ・ ・ ・ 幽 画 圏 直 幽 画 趣 醤 岨 血 畠 圃 ・ ・ ・ ・
圃 圃 E二コl 1l i 圃 E二 コ 匝 圃 仁 二 コ
園 圃 仁 二 コ 圃 圃 仁 二 コ 固 圃 仁 二 コ
関 宮 島 睡 眠 融 担 醐 暢 腫 酷 謹 繭
置璽置 E
Flgure2.14 Imageof GUI.
も Ch. 2, Sec 2. 4

も scr i pt M f i 1e f or 20 Cont our P10t t i ng of f unct i ons
% i nvo1ved i n opt i mi zat i on
者 A11 f unct i ons ar e expect ed t o be avai 1ab1e as
者 MATLAB f unct i on m- f i 1es
昔 Funct i ons wi l 1 be se1ect ed t hr ough t he i nput se1ect i on
も box
告 F unct i on i nput par amet er s wi 11 be meshed mat r i ces
告 based on t he r ange of par amet er s avai 1ab1e f r omGU工

も The Number of p10t s and r ange f or t he p10t
も ar e obt ai ned f r omr unni ng t he
告 user i nt er f ace scr i pt GUI 2_4 pr i or
者 t o t he ca11 t o P1ot 20. m

GU工2 4 も cal l t he GUI
告 X工nf o, Yl nf o ar e i nt r oduced f or i nt r oduci ng def aul t
者 val ues
Xi nf o [ xl mi n x1max x1i nc1 ;
も t he va1ues x1mi n, x1max, x1i nc ar e avai 1ab1e i n
者 t he Command wor kspace due t o Ca11Back
i f i sempt y( Xi nf o)
Xi nf o [- 4 4 0. 051 告 def au1t
f pr i nt f ( ' ¥n' )
Yi nf o [ x2mi n x2max x2i nc1 ;
i f i sempt y( Yi nf o)
Yi nf o ・4 4 0. 051
f pr i nt f (' ¥n' )
xvar Xi nf o( 1) : Xi nf o( 3) : xi nf o( 2) ;
yvar Yi nf o( 1) : Yi nf o( 3) : Yi nf o( 2) ;
[X1, X21= meshgr i d( xvar ,yvar ) ;
% xl vect or
% x2 vect or
も mat r i x mesh
も set def au1t number of p10t s t o 1
i f i sempt y( Np)
Np 1;
c1王 ; 奄 c1ear f i gur e


f or Mp10t l : Np
t ext 1 [ ' The f unct i on whi ch i s bei ng p10t ヒed must
' ¥nbe a MATLAB'
' ¥nf unct i on M - Fi 1e. Gi ven a meshed mat r i x i nput '
' ¥ni t must r et ur n a Mat r i x'
' ¥nP1ease se1ect f unct i on name i n t he di a10g box .
and hi t r e t u r n ¥ n ' 1 ;
% t hi s pr i nt s t he t ext t o t he scr een i n t he command
も wi ndow and ser ves as a pr ompt
者 t he ¥n i s a new 1i ne command
f pr i nt f ( t exヒ1)
も usi ng t he ui get f i 1e di a10g box
[ f i 1e, pat hl ui get f i 1e( ' c: ¥*. m' ,' Fi 1es of t ype
MATLAB m- f i 1e' ,300,300) ;
も check i f f i 1e i s st r i ng
者 st r i p t he . mext ensi on f r omt he f i 1e so i t can be
者 ca11ed by t he pr ogr am
i f i sst r ( f i l e)
f unct name st r r ep( f i l e,' . m' , " ) ;
f pr i nt f ( ' ¥n¥n' )
t ext 2 [ , You have chosen CANCEL or t he f i 1e was
' ¥nnot accepヒab1e'
' ¥nThe pr ogr amneeds a Fi 1e t o Cont i nue'
' ¥nP1ease ca11 P1ot 20 agai n and choose a f i 1e OR '
' ¥npr ess t he up- ar r owbut t on t o scr o11 t hr ough
' ¥npr evi ous commands ¥n¥n'
, Bye 1' ] ;
er r or ( t ext 2) ;
c1ear t ext 1 t ext 2; 告 c1ear s t he var i ab1es t ext 1 .
も and t ext 2 f or r euse
c1ear Fun maxva1 mi nva1 st r con conva1ue oneva1ue
1abcont 1abcont U
F un f eva1( f unct name,Xl ,X2) ;
maxva1 max( max( Fun) ) ;
mi nva1 mi n( mi n( Fun) ) ;
f pr i nt f (' The cont our r anges f r omMI N: 昔 12. 3f MAX.
も 12. 3f ' ,mi nva1,maxva1) ;
f pr i nt f ( ' ¥n' ) ;
st r con i nput ( ' DO you want t o set cont our va1ues ?
[ no] : ' ,' S' ) ;
st r conU upper ( st r con) ;
i f st r cmp( st r conU,' YES' ) I st r cmp( st r co凶, ' Y' )
f pr i nt f ( ' I nput a vect or of cont our l evel s ,) ;
f pr i nt f (' ¥n' )
f pr i nt f ( ' bet ween 者10. 2f and 巷10. 2f ' ,mi nval ,maxval ) ;
f pr i nt f (' ¥n' )
conval ue i nput (' I nput cont our l evel as a Vect or : ' );
l abcont i nput ( ' Oo you want l abel l ed cont our s ?
[ no] : ' , , s' ) ;
el se
l abcont U upper ( l abcont ) ;
i f st r cmp( l abcont U,' YES' ) I st r cmp( l abcont U,' Y' )
[C,h] cont our ( xvar ,yvar ,Fun,conval ue) ;
cl abel ( C,h) ;
el se
cont our ( xvar ,yvar ,Fun,conval ue) ;
ncon i nput ( ' I nput number of cont our s [20] : ' ) ;
i f i sempt y( ncon)
ncon 20;
l abcont i nput ( ' Oo you want l abel l ed cont our s ?
[ no] : ' , , s' )
l abcont U upper ( l abcont ) ;
i f st r cmp( l abcont U,' YES' ) I st r cmp( l abcont U,' Y' )
[C,h] cont our ( xvar ,yvar ,Fun,ncon) ;
cl abel ( C,h) ;
el se
cont our ( xvar ,yvar ,Fun,ncon) ;
el sei f ncon 1
oneval ue i nput (' I nput t he si ngl e cont our l evel ' ) ;
l abcont i nput (' 00 you want l abel l ed cont our s ?
[ no] : ' , , s' )
l abcont U upper {l abcont ) ;
i f st r cmp( l abcont U, ' YES' ) I st r cmp( l abcont U,' Y' )
[C,h] = cont our ( xvar , yvar , Fun, [ oneval ue,oneval ue] ) ;
cl abel (C, h) ;
el se
el se
cont our ( xvar ,yvar ,Fun, [ oneval ue,oneval ue] ) ;
l abcont i nput ( , 00 you want l abel l ed cont our s ? [ . . .
no] :' ,' s' ) ;
l abcont U upper ( l abcont ) ;
i f st r cmp( l abcont U,' YES' ) I st r c即 ( l abcont U,, Y' )
el se
[C,h) cont our ( xvar , yvar , Fun,ncon) ;
cl abel ( C,h) ;
cont our ( xvar ,yvar ,F山 l ,ncon)
i f Np > 1
hol d on;
Hf gcf ;
f i gur e( Hf ) i
gr i d
hol d of f
In the above the pl otti ng commands have been used before. The newMATLAB
commands arei sempt y, cl f , i sst r, st r r ep, max, mi n, upper, st r cmp, and
f pr i nt . Use thehel pcommand 10l earnmore about these. Run thescript file叩 d
understandthes叫 uenceof acti ons as wel l 俗 曲 eprompts regardi ngthecontour. In
白 isexampl e thefuncti ons白 紙 creal ethepl ot do not need gl obal val ues for cal cul ati on
釘 佃 sfeπ' edf rom白 ecommand workspace. Inother words thefuncti ons 白 紙 aretobe
pl ottedcanbe cal cul atedi ndependentl y.
1. MATLAB The Langooge of Tecl l1l ical Compl ui ng: Usi l l g MATLAB, Versi on 5,
Ma山 Wo北 sInc. , 1998.
2. MATLAB The Language of Techni cal CO岬 ul i ng: Usi ng MATLAB Graphi cs, Versi on5,
MathWorks Inc. , 1996.
3. Bohachevsky, 1. 0. , 10hn80n, M. E. , andSlein, M. L. , General i zedSi mul atedAnneal i ng
for FuncL i onOpti mi zati on, Tecl mometri cム Vol. 28, No. 3,1986.
4. Arora, J. S. ,l ntroducl i on10 Opl i l l l umDesi gn, McGraw- Hi I I , NewYork , 1989.
5. Be侃附阜釘r れ ' , F.P. ,andJゐ0加 s蜘ω目 , E. R.可 . , Jr
6. Fox, R. W. , andMcDonal d, A. T. ,l ntroducl i on10 Fl ui dMechani cs, 4由 ed. ,Wi l ey, New
7. Arpaci , V. S. , Conducti ol l Heat Tral l Sfer, Addi son-Wesl ey, Readi ng, MA, 1966.
8. Hansel man, D. , andLi ttletield, B. , Masteri ngMatl ab5, A Comprehensi veTutori al and
Reference. 泊 施 MA'τ ' LABCurri cul umSeries, Prenti ce-HaJ l, Engl ewood Cliffs. NY.
9. Bui l di ngGUI s wi 山 MATLAB. onl i nereferenceinpdf format. Math Works Inc.. 1998.
( Many of thegraphi ca1enhancements arenowpossi bl ethroughpl otedi t features in
la包 r rel eases of MA1LAB. The fo¥l owi ng probl ems arej ust suggesti ons. The suggested
probl ems use addi ti onal graphi cal el ements. and a1sor叫 ui redel eti nghandl es. Pl eas巴
consul t MATI.. ABdocumentati on. )
2.1 Pl ot theba1lis即 位 aj ectory合 o msi mpl e two-di mensi ona1m田 hani cs.
2.2 Produc巴 anani mated di spl ayof t11巴館司 j ectory.
2.3 Createa two-di mensi onal programthat will di spl ay a randomfi ri ngl ocati on
and a randomtarget l ocati on. Al l ow the user to choose initi a1 vel oci ty
magni tude and angl e. Pl ot his tr吋ectoryand ca1cul atehis error.
2.4 Createa thre巴・ di mensi onal program出 at wi l l di spl ay a randomfiringl ocati on
and a randomtarget l ocati on. Al l ow出 e user to choose initi a1 vel oci ty
magni tude and di recti onangl es. Pl ot his traj ectoryand cal cul atehis error.
2.5 Dr awtheboundary l ayer profi l eof l ami nar l10w over a l1at plate. Dr awlines
i ndi cati ngvel oci typrofi l eat tenpoi nts intheboundary layer.
2.6 Create a program出 at will pl ot 白巴 vel oci ty profi l e for user-speci fi c i nputs.
AlI ow theuser toexpress his i nputs througha di a10g box.
The maj or part of this book deal s wi thmnthemnti ca1model s 血 nt nrechnrncteri zedby
nonl i near equati ons. Ii ke most of 由巳 exampl es used for iII ustrati on 白 us far. The
presence of a si ngl e nonl i near equati on in the model is suffi ci ent to i denti fy the
probl emf romthe cl ass of nonl i near programmi ng ( NLP) probl ems. Mathemati cal
programmi ng is ano出 er termthat is used todescri besuch model s and白 eir sol uti on
飽 chni ques. Most of engi ne巴ri ngdesi gnfalls inthiscategory. Havi ng representedsuch
probl ems graphi ca11y in the previ ous chapter. it app儲 rs 白 紙 the gradi ent and th巴
C町 va胞 re of 血 巴 白 ncti ons i n the mathemati cal model had a si gni fi cant i mpact on
i denti fyi ng 血 e sol uti on, even though no effort was made to poi nt out 白 is feature
sp即 ifically. In fact , graphi cal sol uti on was obtai ned by i nspecti on rather than
determi nedby mathemati cal rel ati ons or entities.
There isan equal l y i mpo民 組 tcI ass of probl ems whos巴 mathemati cal model is made
up excI usi vel yof functi ons that areonl y Ii near, such as Exampl e 1.3,鎚 model edin
Equ姐 ons 1. 21- 1. 24. The same exampl e wぉ modi f i edinEquati on51. 25-1. 2850伽 t
a graphi cal sol uti oncoul dbe di 5cusSed. It is app釘 ent 合 o m血巴 graphi ca1descri pti on
in Fi gure 1. 6 白 at 出 町 巴 is no curvature evi dent in 白 e fi gure. These probl ems are
旬 m巴d l i near programmi ng (LP) probl ems. They are natural in 出 e subj ect of
operati ons research, whi ch covers a vast vari ety of model s used for several ki nds of
deci si onmaki ng. Ex創 npl e1.3does represent a deci si on-maki ngprobl em. Typi cal l y,
L P canbe a course by itself (or several courses for 伽 tmatter). A si gni fi cant porti on
of such a co町 民 woul dbe to devel op mathemati ca1model s f romdi fferent areas of
appl i cati ons, as there is usua11y one numeri cal techni que白 at is commonl y used to
sol veL P probl ems. It is cal1ed thesi mpl ex met hod and is based on theal gori thmby
D佃 包 ig [1]. The met hod i nvol ves mostl y el ementary r owoperati ons typi cal1y
encountered in Gauss-el i mi nati on type m巴白 ods that are part of the numeri cal
techni ques used inl i near al gebra. I n出 is chapter, onl y a l i mi l ed, but useful di scussi on
i sp問 sented. First, it isi mportant tounderstandL P probl ems,出巴 model i ngi ssues, and
their sol uti on, as 白 ey are di fferent f rommost of 出 e other probl ems in this book.
S配 ond,m佃 Yof 由巳 current numeri cal techni ques for NL P probl ems obtai n their
sol uti on by l i neari zi ng the sol uti on at the current desi gn poi nt. These l i neari zed
叫 uati ons 回 n be sol vedby白 emet h吋 S of this chapter. 百 le陀 ader is dir田 tedto却 y
of thebooks on L P for a mor e detai l eddescri pti onof themodel s and thetechni ques
as thep回 sentati onin山 is book is si mpl e and brief. References 1- 3 areuseful for this
pu中 ose.
Exampl e 1.3wi l l be used todefi nethetermi nol ogy associ atedwi 白 LPprobl em. 百 le
modi fi edversi oDof theprobl emis used 10devel op出 eformat of theL P probl emso
由 at we canfol l owthroughwi th曲 egraphi cal sol uti on. The mathemati cal model has
t wo desi gn vari abl es xJ ・X2 thal represent 白 e number of Component Pl acement
Machi nes of typeA and B. respecti vel y. The obj ecti veis 10maxi mi ze thenumber of
boards tobe manufactured. Constrai nt gl repr民 間 ts 白 eacqui si ti ondoJ] ars avai l abl e.
Constrai nt g 2 represen15 白 e fl oor space constrai nt . Constrai nt g3 represen15 白 e
numbcr of operators avai l abl c.
Maxi mi ze j(x): 990 XI + 900 X2 + 5250 (3.1)
Subj ect to: gl(x): 0.4XI + 0. 6X2 S 8.5 (3.2)
g2( X) : 3 XJ - X2 S 25 (3.3)
g)(x): 3 XJ + 6 X2 S 70 (3.4)
xJ ~ O ; X 2 ~  
百 le probl em. defi ned in Eq回 ti ons (3. 1)ー (3.4) is a natural representati on of 山
mathemati ca1 model in engi neeri ng desi gn or 血 y other di sci pl i ne. By al l owi ng
叫 ual i ty (=) constJ泊 015組 dgr田 ter 山 anor eq凶 1(注) constrai n15theprobl emcoul d
accommodat e any mathemati cal model 白 at is characteri zedby l i near functi ons. Thi s
represental i on can be consi dered the i nequal i ty f ormof the mathemati cal model
al thoughit is not exp陀 ssedin由 estandardf ormat for theL P probl em.
3.1.1 S旬 ndar dFor mat
The standardformat of theLP probl emi ncl udes onl y equal i tyconstrai n15. It is sct up
as a mi ni mi zati on probl em(somc authors prcfer a maxi mi zati on probl em). ln
addi ti on. all of the vari abl es in也 e model 町 e expected to be semi posi ti ve ( 注 0) or
nonnegati ve [4]. Fi nal l y, thecons位置 int l i mi 15,出 equanti ty on theri ght-handsi deof
出 econstrai nt , arerequi redto be posi ti ve(>0). In山 eprobl emdefi nedinEquati ons
(3.1)-(3.4), the changes 出 at must be carri ed out i ncl ude converti ng the obj ecti ve
functi onto脱 opposi tetype, and transformi ngtheconstrai n15to恥 equal i ty type.
Thev紅 i abl esXJ and X2, representi ngthenumber of Component Pl acement Machi nes
of model A and B, respecti vel y, can be expected tobe semi posi ti veP- 0) ,白紙 is,ei ther
some machi nes wi l l be orderedor no machi nes wi l l be ordered. Not e白 紙 thedi sαete
mωr e of theprobl emis bei ngi gnor吋, as 3. 75machi nes of Type A is not somethi ng
that woul d be ordered. The sol uti onwill be rounded to a sui tabl ei nteger.τ 1tereare
extensi vedi scussi ons indi sc問 胞 andi nteger programmi ng litera側 隠 to suggest 白 紙
白 is may nol be 白 e opt i mumval ue. The j usti fi cati on for roundi ng off is 白 紙 it is
conveni ent 10do soat thisj uncture.
In the obj ecti ve functi onj(x) ,出巴 coeffi ci en15 990 and 900 are cal l ed the cost
coe.伍 cien15. Each coeffi ci ent 白 紙 is associ atedwi 白 adesi gnvari abl e陀:presen15自己
i ncrease(dec胞 ase) incost per uni t change intherel atedvari abl e. 百 lesi mpl est way to
紅 白 l sform山 o切ecti vefuncti onto山 erequi redformat is tomul ti pl y all of theterms
by -1. Therefore,出 enewobj ecti vefuncti onis
Mi ni mi ze j(x): - 990 XJ - 900 X2 - 5250 (3.5)
InEquati on (3.5) , 山 constant termon the right (- 5250) can be absoぬ ed i nto the
l efl -handsi deby defi ni nga newobj ecti vef uncti onL(x) wi thout affecti ng曲 eopt加 al
va1ues for thedesi gnvari abl es. lt can be observed出 at i ncreasi ngtheval ues of the
vari abl es will make theobj ecti ve mor e negati ve, whi ch is good as we aretryi ngto
makej(x) as l owas possi bl e. If a parti cul ar cost coeffi ci ent had a posi ti vesi gn, then
i ncreasi ngtheamount of thecorrespondi ngvari abl ewoul d l eadtoan i ncreaseof the
obj ecti vefuncti on, whi ch is nol desi rabl eif 白 efuncti onneeds tobe mi ni mi zed. Thi s
i deais expl oi tedin出 eSi mpl ex met hod presentedlater.
The泊。 qual i ty constrai nts have to be transformedto an叫 ual i ty co田 町 ai nt. The
si mpl est wayωachi eve thischang巴 i stoi n回 ducean addi ti onal semi posi ti vevari abl e
for each i nequa1ityconstrai nt. Thal wi l l create出 E ∞ 町 田 pondi ngequal i tycons甘剖 nt
ina strai ghtforwardway. Consi der 出 efirst cons凶 nt:
gJ(x): 0.4x
+ 0. 6 S 8.5 (3.2)
τ五is canbe transformedto
gJ(x): 0.4x
+ 0. 6 + x
= 8.5 (3.6)
The newvari abl e X3 is referred to as th巴 sl ack vari abl e. The defi ni ti on is qui l e
appropri atebecause it takes up thesl ack of theori gi nal cons住 ai nt terms when it is less
由 加 8.5.τ 1tesl ack wi1l be posi ti veas 出 efrrst t wo terms wi l l be less 也 佃 or 珂 ual to
8.5(for constrai nt sati sfacti on) f rom山 eori gi nal defi ni ti oninEquati on (3.2). Inother
words, it is thedi fferencebetwecn山 constrai nt limit (8.5) and恥 first t wo terms of
theconstrai nt . If tbeval ues of XI and X2 cause theconstrai nt tobe at i15l i mi t val ue.
出 enX3 will be zero. Thi s is thereasonit is deti nedtobe semi posi ti ve. By defi ni ti on.
therefore, X3is si mi l ar totheoriginaJ vari abl es intheLP probl em. Transformi ngthe
remai ni ngconstrai nts ina similar manner, thecompl ete equati ons instandardformat
Mi ni mi ze j(x): - 990 x, -900 X2- 5250 (3.5)
Subj 回 t to: 8'(X): 0.4x, + 0.6X2+ XJ = 8.5 (3.6)
82(X): 3 X, - X2 +X4 = 25 (3.7)
83(X): 3 x, + 6 X2+ Xs = 70 (3.8)
x, 0; ~ ~ 0; x
0; x
0; X
0 (3.9)
Equati ons (3.5)ー(3.9) exprl 蹴 theLP probl eminstandardformat. An i nconsi stency
canbeobservedwi 由 resp田 tto曲 eformat of theprobl ems previ ousl ydefi nedinthe
book. Namel y, 8(") was reservedωdefi ne i nequaJ itycons位置 ints whi l eh(,,) defi ned
珂 uaJ ity∞m釘 aints. Here, 8(X) is aJ sobei ngusedfor equaJ ity∞ns住 aints. Thi s change
is acconunodatedin白 is chapter only. Insubsequent chapters theprior meani ng for
thesymbol s is restored. Actual l y, symbol s arenot requi red創 出 econstrai nt under
disc凶 si on伺 na1sobe児 島 rred10by equati onnumber. The formu1ati oninEquati ons
(3.5)-(3.9) can be exp問 ssed succi nct1y by resorti ng to a matri x notati on. ηl i s
conversi on is qui te strai ghtforward. Before embarki ng on 山 is transformati on, the
representati onof theobj ecti vefuncti onneeds tobechang' 巴 dagain. The constant term
is brought totheri ghl -handsideso白 紙 由 巳 obj 田 tivefuncti on伺 nbeexpress叫 wi 白
terms onl yi nvol vi ng出 edesi gnvari abl es on theri ght. Thi s is shown asL(x) bel ow.
Once agai nfor conveni ence thesame symbol j(x) is usedinpl aceof thisnewfuncti on
in subsequent di scussi ons. These extra symbol s have been thrown in to keep the
deri vati ondistinct. Thi s is a smal l pri cefor theability 10devel op血 ernathemati caJ
model ina natuml way and 陀 construct it toconf ormtothestandardrep陀 sentati onin
most references. Therefore:
Mi ni mi ze L(x): j(x) + 5250 = - 990 x, -900 (3.5)
Th巴fol l owi ngdefinitions arcusedtorewri t巴tI leprobl emusi ngmal ri ces. Let thecost
coefficients berepr, ωented by c,・C2,• . • , c". The coefficients in出 econstraints are
representedby山 esym加 1ai j (usi ngdoubl e subscripts). 百 lefirst subscri pt identi 自民
自巴 constrai nt number whi1e the secondconstrai nt is the same as the vari abl e it is
mul ti pl yi ng. It is aJ sousefu1toidentify 出巴日 rst subscri pt wi ththerownumber and
thesecondsubscri pt wi 白 thecol umn number to∞rrespondwi 由 opemti ons 伽 t use
I i near al gebm. For exampl e, the∞ 田 町 aint 8,(x) wil1be rep陀 sented邸
g,(x): a" x, + a' 2x
+ a' 3x
= 8.5
where all = 0.4. a12 = 0.6. al3 = 1. Because thereare five desi gn vari abl es in 山 e
probl emdefinition, addi ti onal terms wi 白 血 巴 fol l owi ngcoeffi ci ents, a' 4 (=0) and
al 5(=0). arei ncl udedin山 eexpressi onfor g,(x) wi thout changi ngits meani ng:
g,(x): a
X, + a' 2x
+ a' 3x
+ a'4x4+ a,sxs = 8.5
百 leconstrai nt g2(X) canbe si mi 1arl yexp隠 ssedas
g2(X): G z , x, + a
u ろ
+ a
+匂 sXs = 25
The coe田 cients intheabove equati oncanbe回 silyestabl i shedby compari sonwi th
Equati on(3.7). Letting白 eri ght-handsidevaJ ues of theconstrai nts berepresentedby
, ・ ・ ・ , b
• 出 e st血 血 rdformat inEquati ons (3.5)ー(3.9) c佃 be位 叩 slatedto
Mi ni mi ze j(x): cTx (3.10)
(3.11) Subj 即 日 0: g(x): A x=b
Si de∞n柑 ai nts:" ~ 0 (3.12)
x represents thecol umn vector of desi gnvari abl es, i ncl udi ng山 esl ackvari abl es, l x"
X2・・... x, , ] T σm戸 田 ents the佐 ansposi ti onsymbol ). c陀 presents thecol umn vector
of cost coefficients. [c" C2. ・. • ,C,J T. b represents 曲 ecol umn vector of theconstrai nt
I i mits. [b" b
• • • • , bm]T. A represents them x n matri xof constrai nt coefficients. In
山 is book all vectors 眠 col umnvectors unl ess ot I l erwi seidentified.











- ﹄
















Compari ng Equati ons (3.10) and (3.11) wi thEquati ons (3.5) ー (3.8) 由 evectors and
matri xare
c = [-990 - 900 0 0 O]T
b = [8.5 25 70]T
Azh?i :!│
The f oUowi ng addi ti onaJ defi ni ti ons have beenusedinthestandardformat. n is 出 E
number of desi gnvari abl es 佃 dmi s 白 enumber of constraints. Intlteexampl e, n = 5
andm = 3. Nol e t I lat thesl ackvari abl es arei ncl udedinthecount for n.
Negat l ve Val ues of Desi gn Varl abl es: The LP standard mathemati ca1 model
al l ows onl yfor nonnegati vedesi gnvariables. Thi s does not limit 白 eappl i cati onof LP
inany way. Thereis a standardproceduretohandl enegati veval ues for variables. If a
desi gnvari abl e, sayx.. is cxpectedtobeunreslrI ctedinsi gn,出 enit is defi nedas 出e
di fferenc巴 of two nonnegati vevari abl cs
11 ..1
x. =x-; - x
I - ^J ^J
where and areacceptabl eLP variables. If thelatter is hi gher thantheformer, xJ
canhave negati vevalues. Equati on(3.13) is usedtorepl acex, inthemath抑 制 cal
model . Note that 白 is procedurc alsoi ncreases thc number of desi gnvari abl ωi nthe
model .
Type of Constrai nts: The less 白 an or equal to (::::) constrai nt was handl ed
naturallyindevel opi ngtheLP standardmathemati cal model.百 lestandardnumeri cal
techni quefor obtai ni ngthesol uti onrequi res that 出 esl ackvari abl es be posi “vetostart
and b also be posi ti ve throughout . If there is a constrai nt 白 紙 is requi red to be
mai ntai nedabove a speci fi edval ue, for exampl e
g,(x): a" x, + a
xz+ a' 3x
b, (3.14)
thenit appe紅 snatural toi ntroducea negati vesl ackvari abl e- X4 tochange Equ姐 on
(3.14) toanequal i tyconstrai nt:
g,(x): a
X, + a
+ a' 3x
- x
主 b
Whileit is possi bl etoch皿 ge血 esi gnon X4 by mul ti pl yi ngwi th- 1 白 roughout,白 is
causes白 eri ght-handconstrai nt limit tobe negati ve, whi ch shoul dbeavoi ded. Insuch
cases, another vari abl e, xs, is i ncl udedintheexp胞 ssion. Inthis case, two addi ti onal
vari abl es arenowpart of themodel :
g,(x): a" x, + a' 2x2+ a
- x
+ X
= b
τ'he vari abl eXs is called剖 artificial vari abl e. Arti ficial vari abl es 創 芭 alsoi ntroduced
for 回 chequal i ty constrai nt 白 紙 is naturally present inthemodel. The useof these
vari abl es wil1beillustratedlater throughan巴xampl e.
3.1.2 Model l ng I ssues
The LP programis characteri zedby 白 emathemati cal model defi n巴d inEquati ons
(3.10)ー(3.12). Due tothelinear natureof a11 of thefuncti ons   , thedi scussi on
of the sol uti on needonl y consi der 恥 coefficients of the functi ons-the variabl 邸
themsel ves can be i gnored. These coefficients, whi ch are present as vectors or a
ma回 x,canbemani pul atedtoyieldsol uti ons usi ngconcepts fromlinear algebra.
A l ook at thegraphi cal sol uti ontotheLP probl emsuggests that thesol uti ontothe
probl em, if it exists, must lieon theconstrai nt boundary becausethecurvatureof the
o吋ectivefuncti oniszeroeverywhere (it is linear). Furthermore, thesol uti onto恥 LP
probl emis pri mari l yi nfl uencedby Equati on(3.11). Bei ng a I i near equati on. topics
fromlinear al gebra are necessary to underst加 d some of 出 e i mpl i cati ons of 白 e
mathemati cal model .
If n =m,白 at is, thenumber of unknowns is thesame as thenumber of variables.
therecanbeno scopefor opti mi zati onas Equati on(3.11) will determi ne恥 solution.
provi dedone exists'. Inthe fol1owi ng. many of 白巴 concepts arecxpl ai nedthrough
si mpl egraphi cal illustration, or by consi derati onof si mpl eexampl es. They mostl y
deal wi thexampl es of two variables (x" X2). For exampl e, consi d巴r twocons釘 aints
gl: xl +Xz=2
g2: - x
+Xz = 1
Fi gure3.1 illustrates thegraphi cal sol uti ontotheprobl em. The solutionis at x, = 0.5,
andX2 = 1. 5. Thi s is i ndependent of theobj ecti vefuncti on(whatever it may be). ln
matri xform. A andb are







﹁ 一


百 leexi stenc巴 of a sol uti ondepends on therows of A (or col umns of A- we will deal
wi throws onl yforconveni ence). Iftherows of A arel i nearl yi ndependent, thenthere
is a uni que sol uti onto白 esystemof equati ons.
ThDef i neanother probl emas fol l ows (onl y changi ngb 食 omtheabove exampl e) ・
e newprobl emis
g, : x, +ろ =2
g2: 2x
The newfuncti ong2represents twi ce出 eval ueof gl ・Thi s means g2canbeobtai ned
加 m8J' Therefore, 82is saidtodepend on 81・ Techni cal l y,出 et wo 巴quati ons 釘 e
l i near, か dependent一 白 eac加 叫 dependenceof thefuncti ons is not a concemasmuch
as thefact 白 創 出 eyar芭 not linearlyi ndependent . Both 81and82establishthesame line
graphically. What is thesol uti ontotheprobl em? There areinflnitesol uti ons, as l ong
as the pair of (Xl ・X2) val ues satisfies 81・In Fi gure 3. 1 , any poi nt on the line
representi ng81 is a solution. Not onl y (0.5, 1. 5), but (2,0) , (0,2) arealsosolutions.
From卸 opti mi zati onperspecti ve,白 is is qui tegood for unl i ketheprevi ous caseof
u瓜 quesol uti on, theobj ecti vefuncti oncanbe usedtofindthebest solution. 1t woul d
bebetter, however, if thechoi ces were finiterather thaninfinite.







4 2




)('" 1

- 3
J o
2 - 2
Fl gure3.1 Uni quesol uti on-Unear i ndependence.
Whi l e severaJ cxampl es can be constructed to illustrate linear dcpendence, it
appears reasonabl etoassume thatfora凶 φ11回 ercise仇 opti mi zati on,thenumber of
constrai nts cannot equal thenumber of ulI knowns ( m手 n). Intheprevi ous illus岡 山 n,
n=2釦 dm = 1 (therewas onl y one effecti vecons位 aint). It is useful torecogni zethat
出 is corresponds tothecasen > m.
If m > n, therearemore equati ons thanthenumber of variables. 百 lis i mpl i es 白 紙
thesystemof equati ons representedby Equati on(3.11) is 叩 i nconsi stent set or has a
redundant set of cquati ons. Consi der thefol l owi ngillustrationfor m = 3 and n = 2
whi ch uses thesame 8¥and g2as inthc first illustrationwhi l eaddi nga new83:
g¥: X. +X2=2
g2: - x
+X2 = 1
g3: X¥ + 2 x2= 1
Fi gurc 3.2illustrates 白 紙 theset of equati ons is i nconsi stent si ncea sol uti ondoωnot
exi st. If one were toexist , thenthethreelines must pass through出 esolution. Si nce
出 eyar芭 all s釘泊 ght lines,出 erecanbe onl y one uni que i ntcrsecti ngpoi nt InFi gurc
3.2onl y two of th巴 threelines inter百 ect at different points.



Case n a 2; m a 3: Nosolutlon; In∞nslsten! 5e!
  ~ ・ 1

- 3
4 -2 -1 o
2 3
Flgur由 3.2 No solulion, Inconslstent set 01equations.
Rcdefi neg3as
g3:XI +2ろ =3. 5
Thi s new83is 山 edashedlineinFi gure3.3. Now a uni qu巴 sol uti onto出 eprobl em
at (0. 5 , 1. 5) is establ i shed. 百 lis is 山 othesol uti onestabl i shedby consi derati onof 8¥
叩 d g2 alone. This i mpl i es 白 紙 g3 is redundant. If 81 is mul ti pl i cd by 1. 5, 82 is
mul ti pl i edby 0.5and bothadded. thc result is g3defi nedabove. That is,
g3= 1. 5 gl + 0.5g2
ぉ canbeobtai nedby linearlycombi ni ng g¥and g2. 白 紙 is. addi ngconstant mul ti pl es
of 仰 向 nctions. Thi s is another exampl e of linear dependence. Thi s l i near
dependence and redundal l CYis alsoiII ustratedby theset
g¥: X¥ + "2= 2
g2: -xけ ら =1
g3: 2x
. . ・2x
= 4 (same as gl )
For n ==m, and wi thgl: xl + x2==2, and g2: ーX1+X2 = 1,出 ematri x A is
Casen a 2; m. = 3: UnlqU8solullon, Unoar dependonc9 -







a and its detenni nant is
det(A) = IAI = (1)*(1)一(1)*(ー 1) =2
- ・ "3. 5
Fr omtheorems inIi near al gebra[5] , if det(A) is not zero, inwhi ch case thematri x A
termed nonsi ngul ar, a uni que sol uti onexi sts totheset of equati ons. This was 町 uc for
白 eexampl e iII us回 tedabove, as 恥 onl y sol uti onw俗 l ocatedat (0.5, 1. 5), shown
graphi cal l yinFi gure 3. 1.
Consi der theexampl e wi thgl: XI + X2= 2, 叩 dg2: 2x
+ 2x
=4. In山 is case,
.r 1

det(A) = IAI = (1)*(2) ー( 1) ペ 2) ==0
- 3
- 3 - 2 O
2 3
lf det(A) is zero, that is, matri x A is si ngul ar,恥 reareei ther no sol Ul i ons or infinite
sollllions. For 山 is exampl e, 出 ere were infinite sol uti ons. Dctcrmi nants of
hi gher心 rder squ町 'e matri ces are eval uated by setti ng up l ower-order determi nants
until 出 eyar e 陀 duced to a 2 x 2 determi nant, whi ch is eval ua民 d as above. Any
textbook on engi neeri ngmathemati cs or l i near a1gebra shoul dillustrate白 istechni que.
It is not 回 producedhere.
If n "" m, di scussi onof theexi stencc of sol uti ons requi 隠 s addi ti onal concepts Ii ke
rank of a matri x and augmenl ed matri x. Si nce 由巴 case n > m is of interest in
opti mi zati on, onl y that case is empl oyed in subs叫 uent ilI ustrati on. A useful
di scussi on wi l l need at 1east threevari abl es and t wo equati ons. Three vari abl es wi l l
deny theuse of graphi cs todevel op thefolI owi ng concepts. Usi ng theset
Fl gure3.3 Unl quesolutlon, IInear dependence.
This suggests 出 創 出 e concept of redundancy can be associ ated wi th l i near
dependence. The di scussi onof l i near dependence and i ndependence was establ i shed
凶 泊 g equati ons above. The same di scussi on can take pl acc by reasoni ng on the
coeffi ci cnts themsel vcs. In this case thecoeffi ci ent ma凶 x A shoul d l ead us to 血 e
same concI usi onrcgardi ng山 el i near i ndependence of a set of Ii near equati ons. 百 e
concept of a del ermi nanl is n田 cssarytodevel op thecriteriafor l i near dependence.
gt: x
= 3
g2: - Xけ ら +0.5x
= 1. 5
Determl nant: The determi nant is associ atedwi tha squarc matri x. For a general 2 x
2 matri x A, where
thematri ces 紅 白

















- A









│ a a
1いいいい=吋吋叫吋 d白似削巴 d 叫t(仏 ω山A削ル山門)=同=吋 a
a句21 句 2
The newmatri x A. is cal l ed出 eaugmented mat ri x- t he col umns of b areadded toA.
Accordi ng 10山 ∞rems of l i near al gebra(presentedherewi thout proof):
出 edetermi nant is expres'i ed and eval uatedas
• If theaugmented matri x ( A・) and thematri x of coeffi ci ents (A) have thesame
rank r <n,山 en出 er巴 M 巴 many sol uti ons.
. If曲 eaugmented ma凶 x( A・)加 d出 ematri xof coefficients (A) do not have the
same rank, a sol uti ondoes not exist.
・If theaugmented matri x(A・) andthematri xof coefficients (A) have thesame
rank r = 1 1, where m = n, thenthereis a uni qu巴 solution.
Rank 01 a Matri x: Whi l e thereareformal definitions for defi ni ngtherank of 出 e
ma凶 x,a useful way todetermi ne出巴 ranki sωl ook鉱 山 edeten凶 n剖t . τ'herankof a
matri x A is theorder of 出 elargest nonsi n肝 lar square submatri x of A, that is,出 e
largest submatri x wi tha determi nant other 白 anzero.
Intheexampl e above, thelargest squaresubmatri x is a 2 x 2 matri x(sincem = 2
andm < n). Taki ng the subma住 ix whi ch i ncl udes the first two col umns of A,出 e
determi nant was previ ous establ i shedto have a val ue of 2, thereforenonsi ngul ar.
Thus, therank of A is 2 (r = 2). 百 lesame col umns appear inA市 maki ngits rank also
2. 官 lerefore,infinitely m血 y sol uti ons exist (r < n). One way ωde民 rmi ne 白 E
sol uti ons is to assi gn(n - r) vari abl es arb加 的 val ues and use themtodetermi ne
val ues for theremai ni ngr variables. The val uen - r is alsooftenid巴ntifiedas 出巴
deg reeof f reedom(DOF) for thesystemof equati ons. lntheparticular exampl e above,
山 e DOF can be identified wi th a val ue of 1 (i.e. , 3 - 2). For i nstance, x 3 can be
assi gn巴 da val ueof 1 inwhi ch caseXI = 0.5叩 dX2 = 1. 5. On th巴 other hancJ. if X3 = 2,
thenXI = 0.25and X2 = 0.75. The above cases illustratethat for a DOF of 1, once x 3 is
selected. Xl 阻 dX2 canbe obtai nedthroughsome further processi ng.
Another exampl e for iI1ustrating曲 erank is 出 ei nconsi stent systemof equati ons
in釘 oducede紅 lier where n = 2叩 dm = 3, reproducedfor conveni enceω
gl: x
g2: - x
+ろ =1
g3: x
+ 2x
= 1
The matri ces for this systemare
A = H l }b=[ I } A ' =H i l ]
The rank of A cannot be greater 白 血 2,sincen = 2 and is less th加 m= 3. Fromprior
cal cul ati ons 出 edetermi nant of thefirst two rows of A is 2. Thi s is gr凶 ter thanzero
and th巴 reforetherank of A is 2. For A. thedetermi nant of the3 x 3 matri xhas tobe
exami ned. The determi nant of a general n x n squarematri xA Canbecal cul atedin
terms of thecザ'actorsof thematri x. The cofactor i nvol ves血 ede飽 nni nant of a mi nor
matri x- whi ch isan(n- 1) x (n- 1) matri xob凶 nedby del eti nganappropri atesingle
rowand a singlecol umn f romtheoriginal n X n ma住 ix A, mul ti pl i edby 1 or - 1,
dependi ngon曲 eto凶 of therowand col umn values. References 5 and6 c胡 provi de
mor巴 i nformati on. Here,出巴 determi nant for the3 X 3 matri xisdefi nedfor iIl ustration
in出 ebook. 百 le (+1)加 d(ー1) bel owar芭 obtai ned by usi ng( ー 1);ザ, wher巴 i andj
represent 出 erowand col umn val ues of thecoefficient mul ti pl yi ngtheval ue
al1 a門 al "l l ー -
A = 10;; lln iln卜( +l ) a111273│+(ー l ) a1212J 2│+( +l ) a13仕ご│
1 ~ 2 a
1 品 目 I ~ . ~ , I 1-- ~ I   ~ : l l
百 lerefore,









- A

= (1)(0) + (-1)(-2) + (1)(ー 3) =- 1
A. is nonsi ngul ar andits rank is 3 whi I e therank of A is 2. The systemof equati ons
will have no sol uti ons fromtheabove th巴 orcmandcanbeseengraphi cal l yinFi gure
3.2. For the sake of compl eteness, usi ng the previ ousl y defi ned dependent set of
equati ons:
gl: x
g2: - x
= 1
g3: 2x
+ 2xa=4
whi ch yields thefol l owi ngma凶 ces:
A =[-l l }b=[ ! } A ' =H 1 !]
The rank of A is2. Note白 紙 usi ngthefirst andlast rowof A for thecal cul ati onof the
determi nant yields a si ngul ar 2 x 2 subma凶 x. The determi nant of A・is















且 ''z
、 -








whi ch makes A. singular. However, therank of A・is at least 2 due to出巴 presenceof
曲 目 阻 l esubma佐 ixwhi ch determi nedtherank of A. Fr omtheabove theorem,血 er叩 k
of A isequal totherartkof A. , whi ch is2 and thisisless出 制 m= 3. It canbe concl uded
伽 t 出 ereareinfillitesol 凶 ons for 山 systemof equati ons.
A useful observati onof matri x A・回 t hep陀 cedi ngexampl e is 白 紙 t woof therows
are a constant mul ti pl e of each other. 百世 s makes 出 e ma回 ces si ngul ar. The same
hol ds for t wo col umns 出 at aremul ti pl es of each other.
Un/ t Vectors: Real -worl d opti mi zati on probl ems frequentl y i nvol ve a l arge
number of desi gnvari abl es (nvari abl es). Inreal LP probl ems. 白 enumber of vari abl es
can be staggeri ng. For a successful sol uti on to 出 e op也 凶 zati on probl em. it is
i mpl i ci tl y assumed that the set of desi gnvari abl es is l i nearl y i ndependent . That is.
there is no di rect dependence among any t wo vari abl es of the sct. In出 is case. the
abstract p   c e spawned by thedesi gnvari abl es is cal1ed白 en-di mefl si ofl al Eucl i dean
space. The word abstract isuscd herc tosi gni fythat it isa construct inthei magi nati on,
as onl y threevari abl es can be accommodat ed in由巳 fami l i ar 3D physi cal space. 百 le
desi gnvector is a poi nt in白 is space. The Eucl i deanspa田 is a di rect extrapol ati onof
the physi cal , geome凶 cal 3D space 伽 t surrounds us. Thi s space is termed 恥
Cartesi anspace. For an i11ustrati on, consi der thepoi nt (2,3,2) inCartcsi anspace or in
therectangul ar coordi nate system. InFi gure 3. 4 , thepoi nt is marked P. The凶 adof
numbers wi thi n parentheses denotes the val ues al ong the x" X2・and X3 畝 es,
陀 specti vel y.
The Cartesi anspace is identifi 吋 by a set of mutual l y pe叩 endi cul 釘 lines or axes
markedx"x2' andx3・Thepoi nt wherc theyi ntersect is theori gi n(0) of thecoordi nate
sysl em. The poi nt P can al sobe associ atedWitll thevector OP. drawn f romtlleori gi n
o to血 epoi nt P. Very often a poi nt and a vect or- t wo very di fferent enti ti es-are
used synonymousl y. Fr omel ementary vector addi ti onpri nci pl es, thevector OP can
becons回 ctedby consecuti vel y j oi ni ngthreevectors al ongthethr田 coordi nate出 es.
Inthe fi gure this is shown by l ayi ngout a vector al ongthex, axi s, addi ng a vector
al ong 出 e X2 axi s to it, and finally addi ng thc third vector al ong thc X3 axis. The
addi ti onal smal l arrows inthefi gure are白 eLmi t vectors. 百 l ey have a magni tude of
one and a di recti onal ongeach of the蹴 es. Theseuni t vectors arel abel ede" e2, and
e3 inthe fi gure. Mathemati cal l y,出巴 poi nt P (2,3,2) and thevector addi ti oncan be
rcpresentedω(col umn vectors)
Fl gure3.4 Rectangul ar coordl natesystem.
d e






















The advantage of uni t vectors is白 紙 曲 eycan cstabl i sh加 ypoi nt intheCar飽 si anspace
through an approp巾 te triad of numbers si g凶 fyi ng coordi nate val ues or 10回 tion.
Acωal l y, any poi nt in 由 eCart ωi an space can be detenni ned by any set of thl 国
l i nearl yi ndependent vectors. Thesevectors aretermed thebasi s vectors. Inthis case.
the si mpl e connecti on between the poi nt and the el ementary addi ti on, illustrated
above, will not be avai l abl 巴. Fu巾 ermore,伽 oughthemethods of I i near al gebra, these
three vcctors can be reduccd to the uni t vcctors through el ementary row/ col umn
opcrati onsー al so cal l ed Gauss- J ordan el i mi nati on or reducti on. In n・di mensi onal
Eucl i deanspace spanned by then desi gnvari abl es,白 epoi nt p"(p.. P2, . . . ,p") , will
be representedby a col l ecti onof n val ues encl osedinpar官 n白 eses. The correspondi ng
目 uni t vectors ared巴fi nedas
官 levector f rom由巳 ori gi n(0) toP" is 陀 P問 sentedas
OP = p,e, + P2e2+ . . . + p"e
These newdefi ni ti ons will be suffi ci ent tounderstandthetechni ques used tosol veLP
probl ems.
百 le graphi cal di scussi on is agai n l i mi ted 10 t wo desi gn vari abl es in 由巳 ori gi nal
probl emdescri pti on. It is usedheretodevel op certai ngeometri c i deas rel ated10LP.
I nLP, th巴 i ntroducti onof sl ack vari abl es wi l l easi l yexceed this limit of t wo vari abl es
for graphi cal ill 凶 住 ati on. so thegraphi cal sol uti on is illus回 ted wi th 民 spect to 出 e
ori gi nal vari abl es in the desi gn probl em. MATLAB is once agai n used to 合 aw出 e
graphi cal sol uti on. The probl emis not reduced 10thestandardformat for thegraphi cal
sol u“on. It is i mportant todo 50 for a numeri cal sol uti on. Si nce Ihepri mary graphi cal
el ement is 出 e strai ght li 1Le, a functi on for drawi ng 1i nes is devel oped through th巴
functi on m-fi l e drawLi ne. mbel ow. 百 is functi on is al so used to provi de more
exposぽ 巴 toMATLAB programmi ng.
dr awLl ne. m: 古 山 is a functi on m・ file for drawi ng str泊 ght lines. 百 le 1i ne 10 be
drawn is representedas
以 +by=c
Si ncex, y areusedωsymbol s for vari abl es, xl represents thel ower rangeon x, whi l e
X2 represents theupper rangc. Li mi ti ngIi nes/constrai nts can bc drawn by settingeither
a or b tozero. Col or (red) is uscdtodrawa p町 'allel lineat 10%i ncrease/decreasc in
theori gi nal val ueof c (dependi ngon thedi recti onof theori gi nal i nequal i ty) i nstead
of hashmarks. Li mi t constrai nts areinmagenta and th巴 obj ecti vefuncti onis inbl ue
dashed line. The newMA 11. AB command here is 出 e line command. You are
enco町 agedtos田 k hel pon thecommand. The code:
dr awLi ne. m
者 Dr awi ng l i near const r ai nt s f or LP pr ogr ammi ng pr obl ems
者 Dr . P. Venkat ar aman
者 Opt i mi zat i on Usi ng Mat l ab
告 Chapt er 3 ・ l i near Pr ogr ammi ng

者 Li nes ar e r epr esent ed as: ax + by c ( c >= 0)
者 x1, x2 i ndi cat e t he r ange of x f or t he l i ne
者 t yp i ndi cat es t ype of l i ne bei ng dr awn 1 ( <=)
も 9 ( >=)
告 n ( none)

苦 The f unct i on wi l l dr aw l i ne( s) i n t he f i gur e wi ndow
者 t he gr een sol i d l i ne r epr esent s t he act ual val ue
% of t he const r ai nt
者 t he r ed dashed l i ne i s 10 者 l ar ger or smal l er
者 ( i n l i eu of hash mar ks)
者 t he l i mi t const r ai nt s ar e i dent i f i ed i n magent a col or
苦 t he obj ect i ve f unct i on i s i n bl ue dashed l i nes

f unct i on dr awLi ne( x1,x2,a,b,c,t yp)
告 r ecogni ze t he t ypes and set col or
i f ( t yp ' n' )
st r 1 ' b' ;
st r 2 ' b'
cmul t 1;
el se
st r 1 '" ' g'
st r 2 '" ' r' ;
者 val ues f or dr awi ng hash mar ks
者 dependi ng on t he di r ect i on of i nequal i t y
i f ( t yp -=' n' )
i f ( t yp == ' l ' )
cmul t +1;
el se cmul t - 1;
告 set up a f act or f or dr awi ng t he hash const r ai nt
i f ( abs( c) >= 10)
cf ac 0. 025;
el sei f ( abs( c) > 5) & ( abs( c) < 10)
cf ac 0. 05;
el se
cf ac 0. 1;
i f (c 0)
cdum cmul t *O. l ;
el se
cdロm (1 + cmul t * cf ac) *c;
苦 i f b 0 t hen det er mi ne end poi nt s of l i ne x l i ne
i f (b -=0)
y1 (c - a*x1) / b;
y1n ( cdum- a* x1) / b;
y2 (c - a* x2) / b;
y2n ( cdum・ a*x2) / b;
el se
者 i dent i f y l i mi t const r ai nヒs by magent a col or
st r 1 ' m'
st r 2 ' m' ;
y1 x1; 者 set y1 same l engt h as i nput x1
y2 x2; も set y2 same l engt h as i nput x2
x1 c/ a; 苦 adj ust x1 t o act ual val ue
x2 c/ a; 苦 adj ust x2 t o act ual val ue
y1n ' " 0; 者 set y 0;
y2n 0; 者 set y 0
i f (a 0)
st r 1 ' m' ; 告 set col or f or l i mi t l i ne
st r 2 ' m' ; も set col or f or l i mi t l i ne
者 dr aw axi s wi t h sol i d bl ack col or
hh l i ne( [x1,x2] , [0,0] ) ;
set ( hh,' Li neWi dt h' ,1,' Col or ' ,' k' ) ;
hv 1 i ne ( [0, 0] , [xl , x2] ) ;
set ( hv,' Li newi dt h' ,1,' Col or ' ,' k' );
者 st ar t dr awi ng t he l i nes
hl l i ne ( [ xl x2] , [ yl , y2] ) ;
i f (t乎 p ' n' )
set ( hl ,' Li neWi dt h' ,2,' Li neSt yl e' "ーー' , ' Col or ' ,st r l ) ;
el se
set ( hl ,' Li neWi dt h' ,1,' Li neSt yl e' ,'・', ' Col or ' ,st r l ) ;
i f ( b - = O) &{a - = 0)
t ext ( x2,y2,num2st r ( c) ) ;
i f ( b ==0) I ( a ==0) I (ヒ yp ' n' )
gr i d
r et ur n, end
gr i d;
h2 l i ne ( [ xl x2] , [ yl n, y2n] ) ;
set ( h2,' Li neWi dt h' ,0. S,' Li neSt yl e' ,' : ' ,' Col or ' ,st r 2) ;
gr i d
hol d on
3.2.1 Exampl e 3.1
The probl emf rompage 94 is reproduced0 0田 morefor cooveoi eoce.
Maxi mi ze j{X): 990 XJ + 900 X2+ 5250 (3.1)
Subj 回 tto: g. (X): 0.4x. + 0. 6x2 S 8.5
g2(X): 3xJ - X2S 25
めは) : 3x. +6x2S70
x. 0; .¥i 0
Th巴 graphi ca1sol uti oois shown inFi gure 3.5. The sol uti on, rcadf romthc fi gu飽 (you
C姐 Z∞m白 earcaof sol uti oo) , i sxj = 10, andxi = 7. It is 出 ci ntersecti onof theacti ve
constrai nts (3. 3)叩 d (3.4).百 leactual val ues arexj = 10.48加 dxi = 6.42. Si nce an
i ntegra1 number of machi nes have to be ordered, the sol uti on is adj ust edωa
nei ghbori ngi nteger va1u巴 由 at satisfies l hecontrai nts.
8Or '
3綿 』
- 40

. .
句 ー
-,・- -守 . .
一 . .

・ .
‘ ・ . 旬 、

. .
・ . . .
・ . . .
一一一一一二三. J - 5250
5 10 15 20 25 30
Numbor 01machlnas01IypeA
Fl gure3.5 Graphl cal sOlulion, Exampl e 3.1.
3. 2. 2 Characterl stl cs of t he Sol utl on
τbeglωmetry evi d巴nt inthegraphi cal sol uti onof Exampl e 3.1isusedtoexpl ai nsome
of the concepts associ ated wi th LP and ilS numeri ca1 techni que. Fi gure 3.6 is 白 E
graphi caJ rcp問 senl ati onof 曲巴 constrai nts i nvol vedinExampl e 3.1 (same as Fi gure
3.5wi thout theobj ecti vc 釦 ncti on). Inorder to隠 late出 egcomet r yωLPconceplS, thc
stan也 rdforrnat of LP is n配 ωS釘 y. The standardforrnat establ i shedbeforeis
Mi ni mi ze j{X): - 99Ox
- 900x
- 5250 (3.5)
Subj ect to: g. (X): 0.4x. + 0. 6x
+ x) = 8.5 (3.6)
g2(X): 3x. - X2+ X4= 25 (3.7)
g3(X): 3xI + 6x
+ Xs = 70 (3.8)
In Fi gure 3.6 the constrai nts x. 0 and X2 0 are addcd to the three functi onal
constrai nt Ii nes (3.6)-(3.8). The ha.'!hedareais thc feasi bl eregi on,出 at is, th巴 desi gn
! 20

i 10

- 30
0 5 10 15
Numbor 01Machl n0801TypoA
20 25
Fl gure3.6 Feasi bl eregion, Exampl e 3.1.
space inwhi ch all of theconstrai nts aresatisfied. The circles exaggerate thepoi nts of
i ntersecti onof fi veconstrai nts takent wo at a ti me. Many of themarenumbered Pl ,
P2, . . . , P6. AI I of thesepoi nts of inter富 田 tioncan be associ atedwi tha certai nproperty
concemi ng theval ues of thevari abl es. Note therearefi vedcsi gnvari abl es (n= 5) and
threc functi onal constrai nts ( m= 3). For e瓦 創 npl c:
PI : (x
= 0, x2= 0, X) = 8. 5 , x
= 25, X
= 70)
P2: (x
= 0, = 11. 67, X) = 1. 5, x
= 36. 67, X
= 0)
P5: (x
= 10. 7, x2= 7. 05, X) = 0, x
= 0, X
= -4. 4)
P6: (x
= 0, x
= 14. 17, x
= 0, x
= 39. 17, Xs =ー 15.42)
The val ues of 出 evari abl es areobtai nedas由 ei nte路 氏 ti onofthe constrai nts takent wo
at a ti me. I n出 eabove Ii st , for each poi nt, exactl y t wo of thevari abl es arezero. The
number 2 corresponds totheval ue of n - m. n represents thenumber of vari abl es and
m the number of constrai nts. Poi nts P5 and P6 are i nfeasi bl e because one of the
vari abl es has a negati veval ue.
Basl c Sol utl on: A basi c sol uti onis one obtai nedby setti ngcxactl y n - m vari abl es
to zero. I n Fi gure 3.6, aI l of the poi nts i denti fi ed by the ci rcl es rcprcscnt basi c
sol uti ons. The poi nts choscn above are all basi c vari abl es. In general , for n desi gn
vari abl es and m cons佐官 ints,出巴 numher of basi c sol ul i ons is gi vcnby thecombi nati on
This woul d yi el d10poi nts for our ex儲 npl e,whi ch is shown inFi gure3.6.
Basl c Varl abl es: The set of vari abl es inthebasi c sol uti onthat have nonzero val ues
arecal l edbasi c variables. Correspondi ngl y, theset of vari abl es inthebasi c sol uti on
白 紙 havetheval ue of zeroarecal l ednonbasi c vari abl es. For thepoi nt Pl , XJ 血 dX2
arenonbasi c vari abl es whi l e X3・X4・andxs ar泡 basi c vari abl es.
Basi c Feasl bl e Sol utl on: This is a basi c sol uti onthat is al sofeasible.τ 'heseare
出 epoi nts Pl , P2, P3, and P4 inFi gurc 3.6. InLP, thesol uti ontotheprobl em, if it is
uni que, must be a basi c feasi bl esol uti on. The basi c sol uti oncan aIso be consi dered
geome凶caI lyas a corner poi nt or an extremepoi nt of thefeasi bl eregi on.
Convex Pol yhedron: This is a bounded regi onof thefeasi bl ed巴si gnspace- 白 e
rcgi ondefi nedby出巴 quadri l ateral compri si ng thepoi nts PI , P2, P3, and P4 inFi gure
3.6. The termcOl l vex set represents a col l ecti on of poi nts or vectors havi ng the
fol l owi ngproperty: For any l wo poi nts inthesct (or wi thi na rcgi on) , if all ofthe poi nts
on thel i ncconnecti ng血巴 twopoi nts al soIi e in出 csame regi on, theregi onis a convex
set. I magi n巴 削 y linedrl1wn inl hc regi ontl 巴 finetl by 山 equadri l atendwhose comer
poi nts arePl , P2, P3, and P4. Fr omFi gure 3. 6it is c1ear thelinewilI still be wi 白 in
theregi onestabl i shedby臨 qua合i J ater叫 - maki ng it a convex set.
Opt l mumSol utl on: Thi s is a feasi bl e sol uti on 出 at mi ni mi zes the obj ecti ve
f uncti on- poi nt P4ー in Fi gure 3.6. In LP, the opt i mumsol uti on must be a basi c
feasi bl esol uti on.
Basl s: τ'he basi s 陀 P隠 sents 出 ccol umns of thecoeffi ci ent matri x A 白 紙 coπespond
to the basi c vari abl es. They f ormthc basi s of the m- di mensi onaI space. They are
termed thebasi s vectors.



Canonl cal For m: τ'he basi s vector宮 間 duc吋 to uni t vectors throughrow/ col umn
operati ons (or Gauss-J ordan eIi mi nati on). The basi c feature of 出 e numeri cal
techni que for L P (Si mpl ex method) is a隠 peti ti veproced町 e starti ngf roman initial
basi c feasi bl esol uti on, and determi ni ng出 ebest nei ghbori ngbasi c feasi bl esol uti on
that i mproves theobj ecti ve. The procedure is carri edon until theopt i mumsol 凶 oni s
reach巴d,or if it is determi ned白 紙 nosol uti onis possi bl e. Thc canoni cal f ormis used
for rapi d1y i denti fyi ngthesol uti on. Each i terati onin出巴 procedurecan be descri bed
mathemati caI ly as fol l ows.
The starti ngconstrai nts 創 ' c organi zedas
[A]II 附[ 丸田 1= [b]附
After theGauss-J ordan el i mi nati on, (3.20) is assembl ed as
凹 m糊[ X]m+[R] 州 n-m)[X](s弔問) =[b]
The set of m desi gnvari abl es inthefirst term副 官 thebasi c vari abJcs. The sct of (n-
m) dcsi gnvari abJ es in出 csccond tcrmare肱 nonbasi c val ucs. In出 cSi mpJ ex met hod,
thc nonbasi c vari abl cs aresummari l y set tozcro.
3.2.3 Di 偽 陪 nt Sol 凶 onTyp関
Thcrea回 at most four di fferent resuJts 白 at can be expcctedfor thesoJ uti onof theL P
probJ em: (1) a uni que sol uti on, (2) i nfi ni teJymany soJ uti ons, (3) unbounded sol uti on,
and (4) thepossi bi l i ty山 at thereis no soJ uti on.
(1) Uni que sol uti on: 百 le exampl e used for di scussi on has a Usi que sol uti on
(Fi 割 問 3.5). 百 le condi ti on necess紅 y for 血 is to occur is 山 鉱 山 e obj ecti ve
functi on and theconstrai nts have di ssi mi l ar sJ opes, and thefeasi bJe regi onis
boundcdl cJ osed. Geometri cal l y, thiscan be vi sual l yexpJ ai nedas themovement
of 山 elinereprescnti ngthc obj ecti vefuncti on, pぽ ullcJ 10itseJf, ina favorabJ e
di recti on, until it remai ns j ust incontact wi thone of thefeasi bJecomers (bωi c
feasi bJ e soJ uti on) of the feasi bl e regi on. The fe悩 ibJe regi on is i denti fi ed
ui sti nctl y in Fi g町 宮 3.6. A si mpl 巴 巴 瓦 巴 rcisc uf i magi ni ng diffe印 刷 obj ecti v巳
functi onJines shoul dconvi nce thevi ewer that several di fferent obj ecti ves 伺 n
be defi nedtol ocatetheuni que sol uti onat P 1, P2, P3, or P4. Thi s 20 geome町
and constructi oncan be used tounderstandtheextensi onton di mensi ons. He児
theI i nes wilJ be representedby lI yperpl anes.
(2) lnfinile sol uti on:・ In order for 出 is to occur,出 e obj ecti ve functi on must be
paral l el to one of thecons位置 ints. For exampl e, in Fi gure 3.6, let theori gi nal
probl embe redefi nedso白 紙 theobj ecti vefuncti onis paral l el totheconstrai nt
h ・Anypoi nt on theconstrai nt, and l yi ngbetween thelinesegment dcfi nedby
出 epoi nts p 2 and P4, is an opti mal sol uti ontotheprobl em. They wi l l yi eJ d出 e
sarneval ue of 白 州 問 ve.
(3) Unbounded sollllion: I n出 is case, the feasi bl e regi on is not bounded. In
Fi gure 3.6, if theconstrai nts gl and g2 were not part of theprobJ emformuJ ati on,
出 en the feasi bJ eregi onis not bound on thetop. Referri ngto Fi gurc 3.5. 出 E
obj 民 tivefuncti oncan be shi f tedωunl i lTUtedhi gher val ues. Inpracti ce, there
wi l l be an upper bound on出 巴 rangeof thedesi gnvari abJ es (not p訂 tof stand紅 d
format) that wilJ be used tocJ ose山 is regi on. inwhi ch case sol uti ons of type (1)
or (2) can be recovered. The prcsence of anunbounded sol uti ona1sosuggcsts
伽 t the formul ati on of 批 probl emmay be l acki ng. Addi ti onal meani ngf ul
constrai nt(s) c叩 beaccommodat ed todefi nethesoJuti on.
(4) No sol ul i on:・ Fi gure 3.6is used to expl ai n血 is possi biIi ty. Consi der 白 紙 白 E
di recti onof i nequal i ty in81 is changed to thc opposi tetype(注). The feasi bl c
regi onwi 出 rcspect totheconstmi nts gl and 83is totheright of XI = J 5.0. Thi s
poi nt is not feasi bl ewi threspect to出 econstrai nt 82・Therefore,thereisno poi nt
白 紙 is feasi bl e. Thereis no sol uti onto山 eprobl em.
In the above di scussi on. 出 e two-vari abl e si tuati ons provi ded an obvi ous
cJ assi fi cati onof thesol uti ons. Inpracti ce. LP model s arel argewi thover hundreds of
vari abJcs. Model i ng and transcri pti on errors may gi ve rise to many of the above
si tuati ons. The genemti on of sol uti on is based on numeri cal tcchni qucs of l i near
al gebra, whi ch is oftensensi ti veto山 equal i ty of thematri x of coeffi ci ents. Fi Jteri ng
out errant datais usual l y a si gni fi cant exerci seinthesearchof opti mal sol uti ons.
The standard numeri cal procedure is based on 山 e al gori thmdue to Oantzi g 笛
menti oned earlier. It is refe町 叫 10as 出 eSi mpl ex method. The procedure is rel ated
to 白 巴 sol uti on of a systemof Ii near equati ons. The actual appl i cati on of 白 巴
procedure can be associ atedwi th theGauss- J ordan met hod f roml i near al gebra.
where thecoeffi ci 巴nt rows aretransf ormed through e1ementary mul ti pl i cati onand
addi ti on. Most mai nf rame computcr i nstal l ati ons usual l y carry software that wi l l
hel psol ve L P probl ems. MATLAs al so provi des procedures 10sol veL P probl ems
in its Opti mi zati on TooJ box. In this sccti on, the Si mpl 巴x met hod is appl i ed to
si mpl e probl 巳 m昌 pri mari l y tuundcrstunu thc pl ' Ogr ummi ng and geomctri c features.
This wi l l be used when we di scuss di recI techni ques for nonl i near probl ems. I nthe
next secti on, the Si mpl ex met hod is i ntroduced in detai J wi th expl anati ons. In
subsequent secti ons, MATLAB. or spreadsheet programs I i ke Excel can al sobe used
toi mpl ement theSi mpJ ex method.
3.3.1 Feat ur es of t he Sl mpl ex Met hod
Inthis secti on. themachi ne seJ ecti onex創 npJ ei ntroducedearJier is set up for appl yi ng
theSi mpl ex mc白 od. It is i nstructi vetonotc themet hod is iterative. Gi ven a starti ng
poi nt , it wi l l march f orward throughi mprovi ng desi gns un臥 it has f ound thesoh瓜 on
or cannot proceed further. For compl eteness theori gi nal probl emis rewri ttenhere:
Maxi mi zc j(X): 990x
+ 90Ox
+ 5250 (3.1)
Subj ect to: 81(X): O.4xl + 0. 6X2:s; 8.5 (3.2)


g2(X): 3xI - X2:s; 25
83(X): 3xI + 6x
:S; 70
O ; x 2 ~  
The probl emwas aJ sotransformed10thestandardformat as fol l ows.
Exampl e3. 1
Mi ni mi ze J(X): - 99Ox
- 90Ox
- 5250
Subj ect to: g,(X): 0.4xl + 0. 6x
+ x3= 8.5
g2(X): 3x, - x2 +X4 = 25 (3.7)
g3(X): 3xI + 6x2 + Xs = 70 (3.8)

0; x
O ; x
注 0; x
さ 0; X
注 o (3.9)
X3, X4, and Xs ar芭 曲 esl ack vari abl es.
The Si mpl ex met hod is used on the probl embei ng expressed in the standard
f ormat. Thef ol l owi ng i nformati onis useal l i norgani zi ngthecal cul ati onas wel l as
陀 cogni zi ng the moti vati on for subsequent iterations. 百 1ese refcr to a probl emfor
whi ch a uni que sol uti on exists. While m阻 y of the i tems bcl owwcrc in加 duced
earl i er, they arereferencedhere for compl eteness.
• Thc number of vari abl es inthc probl emisn. 羽 山 i ncl udes thesl ack and surpl us
vari abl es.
・ 百 1enumbcr of constrai nts is m( m< n).
・ 百 1eprobl emis a1ways tomi ni mi ze theobj ecti vc functi onf
• The number of basi c vari abl es is m (same as 出 巴 number of constrai nts).
・Thenumber of nonbasi c vari abl cs is n - m.
・ The set of uni t vectors expected is e" e   , ・ ・ ・ , e

• The col umn of theri ght-handsi de(b) is posi ti veand greater 白 anor equal to
・ T heca1cul ati ons areorgani zedina tabl e, whi ch is cal l eda tabl eau, or Si mpl cx
tabl eau. It wi l l be referredto鎚 thetabl einthis book.
・Onl ytheva1ues of thc c佃 仔 i ci ents ar官 necess町 y for thecal cul ati ons. The tabl e
中crefore contai ns onl y coeffi ci cnt va1ues, the matri x [A] in previ ous
di scussi ons. These紅 巳 thecoeffi ci ents intheconstrai nt equati ons.
・ 百 1eobj ecti vefuncti onis出 el astrowi n出 etabl e. 百 1econs住温 int coeffi ci ents are
wri tten自民t.
• Row operati ons consi st of nddi ng (subtracti ng) a defi ni temul ti pl eof thepi vot
r owtoothEr rows of thetabl e. The pi votrowi denti fi es ther owinwhi ch theuni t
vcctor (ina.ccrtai ncol umn) will havc theval ueof 1.
• Row operati ons areneedcd toobtai nthecanoni cal fonn. Practi cal l ythis回 pl i cs
白 紙 youshoul d bEabl e tospot thEuni t vectors i nthetabl e. I f this happens,
then the current i terati on is compl 巴 te and consi derョti ons for the next onc
must begi n.
Wi t h this inmi nd, theSi mpl ex met hod is appl i edtothemachi ne sel ecti onprobl em.
Thc exampl e wi l l use and cJ arifythei tems intheabove Ii st .
3. 3. 2 Appl l cat l on of Sl mpl ex Met hod
Si mpl ex Tabl e 3. 1: InTabl e 3.1 (Si mpl ex Tnbl e 1),出 c first r owi ndi catcs the
vari abl enames. The last r owis 出 eobj ecti vefuncti on. Spreadsheets arean effi ci ent
way to pro田 ss the i nfonnati on. 百 it is bei ng cmpl oyed,白 en 由巳 symbol f can be
removed f rom出 elast col umn and last r owtoal l ownumeri cal ca1cul ati ons (ぉ shown).
Thel nstcol uD1ni s出 巴 ri ght品 nndsi d巴 val ues. The rest of thc cntri es紅 白 thecoeffi ci ents
of the cons住 ai nt equati ons.
The current i tcrati onis over if thetabl edi spl ays thecanoni cal fonn. Inpracti ce白 e
canoni cal f onn compri sωspotti ng them uni t vcctors inthetabl e, as wel l 箇 maki ng
sure theen凶 es under theb col umn, except for the rows representi ng白 eobj ecti ve
functi on(s), are nonnegati ve P-Q). A gl ance at Si mpl ex Tabl e 3.1 i ndi cates 伽 t 出 E
canoni cal f onn is present.
Thc uni t vectors in出 c tabl eal soi dcnti fythosem vari abl cs 出 at wi l l bel ong tothe
b錨 is. Those vari abl es will have a nonzero val ue for 出 is iteration. The remai ni ng,
n- m, nonbasi c vari abl es aresct tozero. Thi s sol uti onisdi rectl yi ntcrpreted合 omTabl e
3. 1. The sol uti onthereforeis
x, = 0. 0, xz= 0. 0, ろ =8.5, x
= 25, X
= 70,叩 df =- 5250
Has theOpti mal Sol uti onBeen Ob飽 i ned? The opti mnl sol uti onrequi res that f
be reducedぉ much as possi bl e. For thecurrent sol uti on 白巴 val ueof XI is O. Fr om
Equati on(3.5), if x, were i ncreasedf romzero,出 entheobj ecti vefuncti onf decreases
further inval ue. Si mi l ar reasoni ngcnn be used for X2・Therefore,thesol uti oncanbe
further in司proved. Sol uti onhns not converged. The second/ next i terati onwi l l 問 qui re
nssembl i ng another tabl e, Tabl e 3.2.
The onl y way for x, and/or X2to have a posi ti ve val ueis if 自己 y b田 amea basi c
vari abl e. Si nce thenumber of bnsi c vari abl es is prescri bedfor theprobl em, some of
出 e current bnsi c vari abl es must become nonbasi c. For each iteration,出 e Si mpl ex
met hod a1l ows onl y one pai r of vari abl cs toconduct 出 is basi s-nonbasi s exchange.
Each i terati ontakes pl acethrougha newtable. To affect theexchang巴 it is necessary
Tabl e 3.1 Si mpl ex Tabl e 3.1
2 XJ X4 X ~ b
0.4 0.6
。 。
3 -1
。 。
3 6
。 。
司 990 - 9∞
。 。 。
Si mpl ex Tabl e 3. 2 ( Tabl e 3.2): The pi vot r owis obtai nedf romori gi n叫 r ow20f
Tabl e 3. 1. The fol l owi ngis 白巴 ori gi nal r ow
The val ue of the first el ement in the newr owwi1l be 1. The r owis modi fi ed by
di vi di ngthroughby 3. The newsecond r owinTabl e 3.2wi l l be
InTabl e 3.2,血 efirst el ement of thefirst r ow( Tabl e3.1) must be reduced toO. Thi s
isal ways achi evedby addi ng/subtracti ngfromtherowbei ngmodi fi edan appropri ate
mul ti pl eofthe pi vot row. The modi fi cati onto白 efirst r owis
r ow1 - [0.4" (pi vot r ow2)} toyi el d
o 0. 73331 - 0. 1333 0 5. 1667
Thef our出 r owwi1l be obtai nedby
r ow4 + [(990" (pi vot r ow2) I
Tabl e 3.2is obtai nedafter thesecal cul ati ons.
The negati veval ue of thecoe:悶 ci ent inTabl e 3.2inr ow4 i ndi cates 白 紙 weneed
at least one mor e i terati onl tabl e. The EBV is X2・Byexami ni ng白巴 posi ti veratios of
曲 eva1ues in白巴 bcol umn tothecoeffi ci ent in白巴 X2col umn, theL BV is id回 tifiedas
Xs・Thepi vot r owis r ow3. Tabl e 3.3wi1l be constructedby transformi ngtherows of
Tabl e3. 2.
Si mpl ex Tabl e 3. 3(Tabl e 3.3)
pi vot r ow3 = (row3)刀
newr ow1 =r ow 1- {0. 73333 * (pi votrow3)}
newr ow2 = r ow2 + {0. 3333* (pi vot r ow3)}
newr ow4 =r ow 4 + {1230場( pi vot r ow3)}
Tabl e 3.3is obtai nedbased on theabove operati ons.
Si mpl ex Tabl e 3.3 Tabl e3. 3
。 。
O 0.3333

一0. 3333
The thirdr owwi l l be obtai nedby
r ow3 - {3"(pi vot r ow3)}
-0. ¥047
-0. 0285
0. 2857
- 0.1428
X )



todetermi ne whi ch is thevari abl e白 紙 wi l l enter thebasi s (XI or X2) , Enteri ngBasi c
Vari abl e (EBV). Al so to be establ i shedis whi ch of 白 evari abl 回 (X3' X4' or xs) wi l l
l eave恥 basi s,Leavi ng Basi c Vari abl e( LB V). Interms of thenumeri cal cal cul ati ons,
inthenewtabl e, theuni t vector under theL BV col umn wilI be transferredtotheEBV
col umn throughel ementary row/ col umn operati ons.
Enteri ng 8asi c Vari abl e: The EBV is chosen by its abilitytoprovi de for thel argest
decrl団 関 inth巴 obj ecti vefuncti onin也 ecurrent i terati onl tabl e. 百 lis is detercni nedby
thel argest negati ve coeぽi ci ent inther owassoci atedwi ththeobj ecti vefuncti on. Fr om
Tabl 巴 3.1,this val ue is - 990 and co町 邸 pondsto由 evari abl c XI ・EBVis X!. A tiecan
be brokcn arbi trariJ y.
Leavi ng 8asi c Vari abl e: The L BV is al so chosen by its abi l i tyω 泊 l prove 血 e
o町田 tive. The L BV is onl y determi ned after the EBV has been chosen. 官 邸 is
i mportant because onl y thecol umn under 出 eEBVi sexaI 凶 nedtoestabIi sh血巴 LBV.
The mi ni muml' atio of the val ues of 出 e ri ght-hand si de (b col umn val ues) to the
corrcspondi ng coeffi ci ents in the EBV col umn, provi ded 出 c coeffi ci cnt val uc is
posi ti vc, d, 巴 ci dcs 山 eLBV. Thi s rni ni mumval uc i dcnti fi es ther owwhi ch is aIso the
pi vot row. Inthis r ow, thecol umn, whi ch contai ns theuni ty val ueof 白巴 someuni t
vector (or a current basi c vari abl e) , is thc col umn of 白 c I eavi ngbasi c vari abl e. 百 lis
is t heLBV.
InTabl e3. 1曲 eEBV has al ready been i denti fi edas Xl' The rati os under this col umn
are 21. 25, 8. 3333, and 23.3333. The least val ue is 8. 3333. 百l I s corr邸 ponds to 由 e
S巴condr owof Tabl e 3.1. The uni t vector under theX4 col umn is in出 is r ow(row2).
Thcrefore, X4 is 由 e L BV加 d 出 e second r owis the pi vot r owfor Tabl e 3.2. Thi s
i mpIi es 出 創 出 巴 se氾 ondr owwill be uti l i zed to chang巴 出 巴 current XI col umn (EB V
col umn) iilto a uni t vector 白 al is currentl y under X4・Thi s mechani cal procedure is
based on thefol l owi ngreasoni ng. 百 leval ueof EBV must be such that it wi l l not cause
constrai nt vi ol ati onwi ththecurrent val ues of 血 eother basi c feasi bl evari abl cs. The
appl i cati onof theGauss- J ordan process shoul d shift thecol umn under theL BV to出 e
EBV l eavi ngthecol umn of theother uni t vectors (that arepart of thebasi s) unal t巴red.
Tabl e 3.2is thenconstructedusi ngthc pi vot r ow2.
Si mpl ex Tabl e 3.2 Tabl e3.2
5. 1667
-0. 1333

0. 7333
- 0.3333
- 1230
A( : ,6) / A( : ,1)
o 0 0 - 17/ 1980
o 0 0 - 51198
o 0 0 - 7/ 99
0 0 0 ・ 16/ 3
' somet hi ng not r i ght - The above di vi si on shoul d have' ;
' been an el emenヒ by el ement one' i
ヒhe cr eat e t o r ow pi voヒ t he i dent i f i es ' The r ow
t abl e' ;
A( : ,6) . / A( : ,l )
ans =
21. 2500
8. 3333
23. 3333
- 5. 3333
' Second r ow i s t he pi vot r ow and x4
' Const r uct i ng Tabl e 2' ;
' not e t he scal i ng f acヒor f or t he mat r i x' i
1. 0e+003 *
0. 0004 0. 0006
0. 0030 - 0. 0010
0. 0030 0. 0060
- 0. 9900 - 0. 9000
' The el ement at A( 2,1)
A( 2,: ) = A( 2, : ) / A( 2,1)
1. 0e+003
0. 0004
0. 0010
0. 0030
- 0. 9900
A( : ,6) . / A(: ,l)
85/ 4
25/ 3
70/ 3
- 16/ 3
f or mat shor t
i s LBV' ;
0. 0085
0. 0250
0. 0700
5. 2800
0. 0010
0. 0010 0
o 0. 0010
o 0
o 0
must be 1' ;
0. 0085
0. 00自 3
0. 0700
5. 2800

0. 0010
0. 0003
0. 0010
0. 0006
- 0. 0003
0. 0060
- 0. 9000
1 l

Tabl e 3.3 does not have 佃 y negati ve val ue 泊 白 e obj ecti ve functi on row. A
sol uti onhas been achi eved. The basi c vari abl es have theunit vector intheir col umn.
The nonbasi c vari abl es areset to zero. Fr omTabl e 3.3 the op白 numval ues canb巴
observed邸 :
x ; = 10.4762 , xi = 6.4285. X; = 0.4524,I = -21437. 14
Note, theval ueofj " +21437. 14=Of romwhi ch th巴 sol uti onfortheobj ecti vefuncti on
(f.) canbe obtai ned.
Sol utl on Usl ng MATLAB
The tables intheprevi ous computati oncanbe generatedqui teeffortlessly usi ng a
spre叫 sheet . 百 lis woul d be accompl i shed by setting up 血 .e mathemati cal
mani pul ati oninone cell and copyi ngand p脳 血 gto出 eother cells. Si ncethis book
al so邸 sistsinMATLAB progr創 nmi ng,theillus住 ati onherewill al l ow佃 opportuni tyto
practi ce the use of MATLAB for matri x mani pul ati ons. The code presented here is
四 pturedusi ngMATLAB' S di ary command. The command requi res thename of a白 川
filewhi ch records all communi cati on wi l hMATLAB as wel l ωi ts responseduri ng叩
i nteracti ve sessi on (see hel p di ary). The text file is commented liberally (for your
understandi ng) and usi ngthesemi col onat 由 eend of thestr.加 gs prevents 由 ei recho.
h 出 is filethecomments arequo飽 ds回 ng. The code represents 由巴 detailsof anactual
sessi on. These statements are typed a食 er 白 e MATLAB prompt 也 the Command
wi ndow. Thisisthereforeani nteracti vesessi on. Val ues to白 eright of ans = istheecho
of result of 出 e ex田 uti on of 出 e MATLAB command. Note: comments need not be
Exampl e 3_l. txt
f or mat compact
f or mat r at i onal
' Set up t he coef f i ci ent mat r i x and b as mat r i x A' ;
- 1 0 1 0 25; 3 6 0 0 1 70; - 990 1 0 0 8. 5; 3
0. 6
o 0
A=[ O. 4
- 900 0
17/ 2
canoni cal f or m' ;
o 0
1 0
o 1
o 0
not e t he
col umn' ;
2/ 5 3/ 5 1
3 ・ 1 0
3 6 0
・ 990 - 900 0
' Thi s i s Tabl e 1 -
' EBV i s x1 - f i r st
' To f i nd L BV di vi de l ast col umn by coef f . i n EBV' i
' col umn' ;
' Take t he mi ni mumof t he posi t i ve val ues' ;
' The el ement at A( l ,l ) must be a 0 ' ;
A( l ,: ) = A( l ,: ) ー 0. 4*A( 2,: )
1. 0e+003 合

0. 0007 0. 0010 - 0. 0001 o 0. 0052
0. 0010 ー0. 0003 o 0. 0003

0. 0083
0. 0030 0. 0060

o 0. 0010 0. 0700
ー0. 9900 - 0. 9000 o o

5. 2800
' El ement at A( 3,1) must be a zer o' ;
A( 3,: ) = A( 3,: ) - A( 3,1) *A( 2,: )
1. Oe+003 *

0. 0007 0. 0010 ー 0. 0001

0. 0052
0. 0010 - 0. 0003

0. 0003 o 0. 0083
o 0. 0070

- 0. 0010 0. 0010 0. 0450
- 0. 9900 - 0. 9000 o o o 5. 2800
' El ement at A( 4,1) must be 0' ;
A( 4,: ) A( 4,: ) - A( 4,1) *A( 2,: )
1. 0e+004 *

0. 0001 0. 0001 ー 0. 0000 o 0. 0005
0. 0001 - 0. 0000

0. 0000 O 0. 0008
o 0. 0007 o - 0. 0001 0. 0001 0. 0045

ー 0. 1230

0. 0330 o 1. 3530
' Tabl e 2 compl et e - and canoni cal f or mi s pr esent ' ;
' Sol ut i on i s not conver ged because of A( 4,2) ' ;
, EBV i s x2' ;
' Cal cul at i on of LBV' ;
A( : ,6) . / A(: ,2)
7. 0455
・25. 0000
6. 4286
- 11. 0000
' Pi vot r ow i s t hi r d r ow and LBV i s x5'
Pi vot r ow i s t hi r d r ow and LBV i s x5
' Oops f or got t he semi col on' ;
' Cal cu1at i on of LBV' ;
' Const r uct i on of Tabl e 3. 3 - no echo of cal cul at i ons' ;
' A( 3,2) must have val ue 1' ;
A( 3,: ) = A( 3,: ) / A( 3,2) ;
' A( 1,2) must have a val ue of 0' ;
A( l ,: ) A( l ,: ) - A( 1,2) *A( 3,: ) ;
' A( 2,2) must have a val ue of 0' ;
A( 2,: ) = A( 2,: ) - A( 2,2) *A( 3,: ) ;
' A( 4,2) must have a va1ue of 0' ;
A( 4,: ) A( 4,: ) - A( 4,2) *A( 3,: ) ;
A '"
1. 0e+004 *
o 0
0. 0001 0
o 0. 0001
o 0
f or mat r at i onal


























- 11/ 105 19/ 42
1/ 21 220/ 21
1/ 7 45/ 7
1230/ 7 150060/ 7
' No f ur t her i t er at i ons necessar y' ;
di ar y of f
The last f ewcommands sugg巴 sl Ihc possi bi l i ty of devel opi nga for l oopfor therow
mani pul ati ons. It is suggestedas a probl emat theend of 出 echapter.
3. 3. 4 Sol utl on Usi ng MATLA8' S Opt i mi za舗 onTool box
Thi s secti on is useful for thosc that have access 10the Opti mi zati onTool box . f rom
MA'τ1. AB(Ver 5.2).
Start MATI.. AB. IntheCommand wi ndowtypehel plp. Thi s will provi de i nformati on
on theuseof thel i near programmi ng routi ne10sol vetheprobl em. Here. thestandard
useis empl oyed. It i nvol ves speci fyi ngva1ues for thecosl coeffi ci ents (ιvector). the
coeffi ci entl constrai nl matri x ( A- ma凶 x). and the ri ght-hand si de vector (b・vector).
τbe fol l owi ngsequence of steps illus住 ates 出 euse of theprogram.
Usi ng theLi near Programmi ng Routi ne
>> f ・ 990; ・ 900] ;
>>A [ 0. 4 0. 6 3 - 1; 3 6)
>>b [ 8. 5 25 70) '
>> x '" 1p( f ,A,b)
The sol uti onas posl edintheCommand wi ndow:
10. 4762
6. 4286
>> sol ' " f ' *x
8 0 1
・ 1. 6157e+004
To this sol uti on must be added 白巴 constant - 5280 whi ch was omi tted inprobl em
defi ni ti onfor MATLAB. There ar巴 many di fferent ways touse山 巴 l i near programmi ng
functi oninMATLAB.
h 出 is secti on addi ti onal exampl es 釘 e p偲 sented. These exampl es illustrate 山 e
extensi on/modi fi cati on of the Si mpl ex met hod to handl e greater 由 回 or equal
constrai nts, negati veval ues for thedesi gnvari abl es, equal i l y constrai nts, and soon.
In alI 由 e cases, the probl emis transformed appropri atel y and 白 e same Si mpl ex
mel hod is thenappl i ed.
3. 4. 1 Exampl e 3. 2-Transport at i on Pr obl em
The Fresh MiI k c∞perati ve supp1 i es mi Ik in gal l onj ugs f romits t wo warehouses
l ocatedinBuffal o ( New York) and WilI i amsport (Pennsyl vani a). 1t hぉ acapaci ty of
2000 gal l ons per day at Buffal o and 1600 gaI l ons per day at Wi11i amsport. It del i vers
800 gal l ons/ dayωRochesωr ( NewYork). Syracuse ( NewYork) r,叫 ui res 1440
gal l onsl day, and曲 eremai nder (1360 ga1l ons) aretruckedtoNew Yor k Ci ty. 百 lecost
to shi p l he mi l k to each of l he desti nati ons is di fferent and is gi ven inTabl e 3.4.
Estab1 i sh出 eshi ppi ngs位置 tegyfor mi ni mum∞st.
Pr obl emFormul at/ on: Let x be出 e number of gaI l ons shi ppedf romBuffal o to
Rochester. Let y be出 enumber of gal l ons shi pped針。 mBuffal o toSyracuse.
The warehousi ng cons回 int at Buf faI o is 2∞o gal l ons/day. Therefore,
x + y $ 2000
Amount shi pped企 o mWi l l i amsport toR, ∞hes町 =8oo- x
A mount shi pped企 o mWi l l i amsport toSyracuse = 1440 - y
Amount shi ppedf romWi l l i amsport toNew Yor k Ci ty is
1600 - (800 - x) ー (1判 O- y) 注 o
Tabl e 3.4 Shi ppi ngCost ( Cen匂 perGal l on)
WiIl i創 nsport
New York Ci ty
x   y ~ 6 4 O (3.22)
The si deconstrai nts on x and y areth巴 respecti vewarehouse I i mi ts. The shi ppi ngcosl
Cost =4. 2ホ x+ 4.5* y + 6.0* (2000 - x - y) +
4. 7・( 8oo- x) +4. れ ( 1 判 0 - y) + 5.1 ・ (x+ y - 640)
Cost =ー 1. 4' " x- y + 19120
Assemb1i ng theprobl em(Exampl e 3.2)
M泊 b凶 ze f(x,y): ー 1. 4x- y+19120
Subj ect ω81( X, y): x + y :52000
82(X, y): x + y注 640
0$x:58∞; 0:5y:51440
Fi gure 3.7iII ustrates thegraphi cal sol uti ontoExampl e 3.2. The linerepresenti ngthe
constant obj ecti vefuncti onval ues l ermdoes not i ncl udetheconstant term(19120).
The sol uti onis at thei ntersecl i onof constr温 int gl and theupper 1 i mi t on出 eval ueof
xas 出 eobj ecti veis todecreasef as much as possi bl e. Fr omthefi gurethesol uti onis
x=800; y=1200,f = 168ooor$168. oo (3.28)
Two- Phase S/ mpl ex Met hod: The mai n di fference bel ween Exampl es 3.1 and
3.2is 出 e注 cons町 ai nt intheIatter. Si nce thestandardL P probl emonl y requi res the
desi gnvari abl es be semi posi ti ve位 0) ,出 eri ght-handconstrai nts on x and y have to
be accommodat ed through addi ti onaI i nequaI ity constrai nts. Appl yi ng the regul ar
Si mpl ex met hod wi thout addi ti onaI processi ngwoul d cr回 tea probl eminrecogni zi ng
也 e c釦 oni 回 1 form. Exampl e 3.2 expressed in 出 e standard f ormat of l i near
pr ogr ammmg IS
Mi ni mi ze ft. x,y): ー 1. 4x- y + 19120
Subj ect to: gl (X, y): x + y + SI = 20∞
g2(X, y): x + y - S2 = 640
g3(X, y): x + S) = 800
84(x, y): Y + S4 = 1440


崎 町
• ﹄﹃



句 ‘


﹃ h



• ﹄﹃


- ・‘ 』旬。
0 200 300 400 500 600
x:g副 10間 110mB凶 f副 ol oR∞h田 101
100 700
Fl gure3.7 Graphl cal 80lullon, Exampl a 3.2.
x 0; y 0; sl' s2' s3' s4  0 (3.33)
Heresl ・S3' s4arethesl ackvari abl es and S2is 白巴 surpl .凶 (si mi l ar toa sl ack- used for
  constrai nts) vari abl e. In Exampl e 3. 1 , when the f1rst Si mpl ex tabl e was set up
(Si mpl ex Tabl e 3.1), the sl ack vari abl es were hel pful in i denti fyi ng the canoni cal
form. Here inExampl e 3.2, it do凶 not wor kout 伽 t way because thecoeffl ci ent of S2
is -1. Mul ti pl yi ngEquati on(3.30) by - 1 mi grates thenegati vesi gntotheri ght-hand
side. whi ch is al so disaJ l owed as f :紅 as 白 e recogni ti on of 白 e canoni cal f ormis
concerned. Thi s means addi ti onal probl em-speci fi c preprocessi ngmust takepl aceto
i denti fy theinitial canoni cal form. Si nce theSi mpl ex me出 odis used tohandl e l arge
probl ems wi thseveral vari abl es, it is mor e conveni ent toappl y theSi mpl ex procedu陀
ina consi stent way. Thi s is accompl i shed by appl yi ngthesame Si mpl ex techni que in
a two-part sequence.
百 le f1rst p副 is recogt甘zedas Phase 1. Here a newki nd of vari abl e. an artificial
vari abl e, is defi ned for each surpl us vari abl e in the same equati on. Al so, a new
obj ecti vefuncti oncal l edan artificial cost functioll or 加 artificial obj ecti vefuncti on
is i ntroduced. The artiflcial obj ecti veis defi ned邸 asumof aJ l theartificial vari abl es
intheprobl em. InPhase 1,白 eartiflcial obj ecti ve is 同 ducedtozero usi ngthestandard
Si mpl cx procedure. When this is accompl i shed, Phase 1 is compl ete. I f 山 eartifici aJ
obj ecti vedepends onl y on theartifici aJ vari abl es. 佃 dif its vaJ ue is zero,出 is i mpl i es
that 白巴 artifici aJ vari abl es are lI onbasi c vari abl es. This al so suggests that these
vari abl es werc basi c vari abl es at thestart of theprocedurc. When Phase 1 iscompl eted,
曲 enboththeartifici aJ obj 巴cti vefuncti onand theartiflcial vari abl es aredi scarded台。 m
出巴 tabl eand Phase 1/ begi ns.
Phase II is the standard Si mpl ex techni que appl i ed to the tabl e f romthe end of
Phase 1 negl ecti ngall 山 artificial el ements. The tabl e shoul d be incanoni cal form.
Addi ti onal tabl es areobtai nedas necessary until thesol uti onis reached.
I nExampl e 3.2set up earl i er, therewi l l be one artifici aJ vari abl ea 1 and an artifici aJ
cost functi onA" Intheexampl e onl y Equati on (3.30) wi l l be affectedas
g2(X,y): x + y- s2 + a
=640 (3. 34)
The artifici aJ cost functi onwi l l be
Af =a
Tabl e 3.5 rep陀 sents 出巴 f1fSt tabl e in Phase I. The f1rst four rows rep隠 sent 白 E
constrai nts. The f1fthr ow問 p民 sents血 eori gi nal obj 田 町 efuncti on. The last r owis由 e
artifici aJ obj ecti ve functi on. This is 出巴 r owuSed to 合 iv巴 出 巴 i terati ons in Phase 1.
Tabl e 3.5 is not in canoni caJ form. but repl aci ng the si xth r owby the resul t of
subtracti ng血 esecond r owf romthesix出 r owwi l l provi de a canoni cal form. Tabl e
3.6illustrates theresul t of such a r owmani pul ati on.
Sl mpl exMe的 od,PhBse 1: InTabl e 3.6therearet wo choi ces (x, y) avai l abl efor
曲 eEBV(ente自 Igbasi c vari abl e) as both of t hemhave a coeffi ci ent of - 1 in血 elast
row. Whi l e the choi ce can be arbi trary, x is a good choi ce because it has a l arger
negati ve coeffl ci ent in the ori gi naJ obj ecti ve functi on. 百 le L BV (l eavi ng basi c
vari abl e) is i denti fi edthrough themi ni mumposi ti vevaJ ue of theratioof theval ues in
the b col umn to出 e val ues under 出巴 xcol umn. 官 l esc rati os are2000/ 1,64011, and
80011. The sel ecti on i denti fi es 出 esecond r owas 自己 pi vot r owand a 1 総 白 eLBV.
Usi ng the second r owω 血 e pi vot r ow, the uni t vector [0 1 0 0 0 O]T hωt o be
Tabl e 3.5 Exampl e 3.2: Initial Tabl e, Phase 1
x y SI S2 01 S3 S4 b
o o o

。 。 。
。 。 。 。
o 800
。 。 。 。 。
ー1. 4

。 。 。
f - 19120

。 。 。 。
Tabl e3.6 Exampl e 3.2: Sl mpl ex Tabl e1, Phnse, (Canonl 凶 I Form)
x $1 $2 b al $) $4 y
-1. 4


o f - 19120
o A
- 640


町創 lsfe町 edf romtheal col umn tothex col umn. Constructi onof Tabl e 3.7st訂 ts wi th
thesecondrowinTabl e 3.6, whi ch is al sothesecondrowof Tabl e 3.7becauseofthe
coe仔 icient of 1 inthesecondrowunder thex col umn (noscal i ngis necessary). This
is the pi vot row. To obtai n a 0 in the first rowin thex col umn, the pi vot rowis
subtrョctedf rom出 巴 白 rst rowinTabl e 3.6. The newfirst rowis
-1 0 0 1360
To obtai na 0 in thex col umn inthethirdrow. thepi vot rowis subtractedfromthe
thirdrowof Tabl e 3.6. The fourthrowhas a 0 inpl aceand hencc iscopi ed合' OmTabl e
3.6. Thc fifthrowisobtai ncdby addi nga 1. 4 mul ti pl eof thepi vot rowtothefi t1出 row
inTabl e 3.6. The last rowis theaddi ti onof thepi vot rowtothelast rowf romTabl e
3.6. Tabl e 3.7is compi l edas fol l ows:
Tabl e3.7 Exampl e 3.2: Sl mpl e宜 Tabl e2,Ph脚 1(Canonl col Form)
x $2 b $) $1 al $4 y

o f - 18224
o A




-1. 4


Fr omTabl e 3.7. 山 eval ueof Af is O. and al is not a basi c vari abl e・Thi s identifies
theend of Phase 1. The a 1 col umn and白巴 last rowaredi scardedand Phase 11is started
wi thTabl e 3.8.


Tabl e3.8 Exumpl e 3.2: 創 mpl exTabl e1, Phase s (Canoni ωI Form)
b y $2 $) $ ,



制 0
0. 4


-1. 4 o f - 18224
Si mpl ex Met hod, Phase 11: The col umn wi th the gr田 test negati ve coeffi ci ent
(一1. 4) iss2・百 lis is 出 eEBV. The LBV is s3・百 le出 irdrowis thereforethepi vot row.
The unit vector [00 1 0 O]Tmust be町 ansferredf romtheS3 col umn to由巳 S2 col umn
throughrowmani pul ati ons usi ngthepi vot row. 百首 sexerci seresults inTabl e 3.9.
The negati vecoeffi ci ent under they col umn suggests we arenot fmi shedyet. For
thenext iteration, EBV isy. L BV is SI ' The first rowis 出 epi vot row. The unit vector
[1 0 0 0 0 ]Tmust beconstructedunder they col umn usi ngrowmani pul ati onusi ng
thepi vot row. This results inTabl e 3.10.
Thereareno negati vecoeffi ci ents inthelast row. The sol uti onhas beenrellched.
Fromi nspecti onof Tabl e 3.10, thesol uti onis
x = 800, Y = 1200. S2 = 1360. S4 = 240,f =16800 (3.36)
In this ex創 npl e,the same Si mpl ex method was repeatedl y appl i ed al though the
probl emwas sol vedusi ng抑 ophases. The first phase was a preprocessi ngphぉ eω
move the surpl us vari abl es away f rom出巴 initial set of basi c variables. 百 lis was
achi eved by i ntroduci ng addi ti onnl vari abl es for each su叩 lus vari abl e and an
addi ti onal cost functi onthat drove出 eiterations of thefrr宮tphase.
To summari ze, for   and = constrai nts, artificial vari abl es and釦 artificial cost
functi on釘 e in官。 du回 d into the probl emfor conveni ence. 百 le Si mpl ex method is
appl i edintwo phases. 百 lefirst phaseis termi natedwhen theartificial vari abl es clln
beel i mi natedf rom山巴 probl em.
Tabl e3.9 Exampl e 3.2: Sl mpl ex Tabl e2, Phas量 n( Canonl cal Form)
x $) $4 b y $ ,


1. 4
o 12∞
o 8∞
o 160
o f - 180ω


3 ,A
直組長 3
Tabl e 3.10 Exampl e 3.2: Sl mpl ex Tabl e 3 (Fioal), Phasc D
x y SI S2 S) S4 b
。 。 。
。 。 。

。 。

。 。 。
。 。 。

f - 16800
3. 4. 2 Exampl e 3ふ - Equal l t yConst悶 i nts and Unrestri cted Varl abl es
Exampl e 3.3 wil1 inc1ude an equal i ty constrai nt and iJl ustrates the techni que for
handl i ngvari abl es 白 紙 areunres町 i ctedinsi gn(canhave negati veval ues). Vari abl es
like profi t , temperature, and net i ncome can be negati ve qui te ofl刷 1. The 叫 ual i ty
cons回 int is handl ed by in凶 duci ng印 刷 ificial vari abl eand appl yi ng出 et wo- phase
Si mpl ex t l田 加 i qu巴 illus回 ted in Exampl e 3.2. When this correspondi 略 的 自 cial
vari abl eis zero( 巴 ndof Phase 1) ,由巳 constrai nt has been met. The negati veval uefor
a vari abl eis si mul atedthrough出 ediffl巴 rencebetween t wo posi ti vevari abl es. Once
agai n, thesechang巴 spreservetheSi mpl ex met hod inits ori gi nal form. The fol l owi ng
is an i magi nary probl emωi l l ustrateaddi ti onal features sosome L i berty is takeninthe
formul ati on.
The Probl em: Today, a full-tirnestudent on campus is al ways dri ventomaxi mi ze
grades. A strongi nfl uencefor a favorabl eout come is 血 eamount of i nves回 開 t made
inhi tti ng出巴 books versus 出 eti me spent pl ayi ngpi nbal l . Ina typi cal day, at least one
hour is defi ni tel y extended to the pursui t of l earni ng or pressi ng the si des of the
coi n-operated machi ne. Not more than fi ve hours is avai l abl 巴 for such di spersi on.
Over 出 eye紅 s,a fati guethreshol dof four units. basedon a combi nati on of 白巴 t wo
activities, has been establ i shed as a nor mfor acceptabl e perf orm四 ce. Thi s
combi nati on is thesumof thehours spent at thepi nbal l machi ne and twi c巴 thehours
spent on homewor k, an acknowl edgment that hi tti ng血 ebooks is stressful. A negati ve
hour of pi nbal l pl ayi ng wi1l go to i ncrease the ti me spent studyi ng, whi ch will
contri bute posi ti vel y to the grades. The overal l grade is determi ned as a l i near
combi nati on of twi ce the ti me spent academi cal l y and subtracti ng 白 e ti me spent
pl ayi ng. The probl em出 erefore is to di stri bute the ti me to maxi mi ze the grades
obtai ned.
Pr obl emFormul ati on: There aret wo ori gi nal desi gnvari abl es intheprobl em. XI
is thenumber of hours spent studyi ng, and X2 is theti me spent enj oyi ngthegame of
pi nbal l when th巴 sameti me can be advantageousl y spent mai ntai ni nghi ghgrades.τ 'he
obj 配 tivefuncti oncan be expressedas
Maxi mi ze j{x
, X
): 2x
1ー ら
The constrai nts can be recogni zedf romthestatement of 出 eprobl emdi rectl yas
g2: 2x. + x
= 4
g3: x
0, x
is unrestri ctedinsi gn
Fi gure 3.8, usi ngthedr awLi ne. mm-fi l e, isa graphi cal descri pti onof theprobl emand
thesol uti on. 明 1esol uti onis atxl = 3 and X2 = - 2 in自 己 fi gure,where conveni entl y白 e
obj ecti vefuncti onis al sopassi ngthrough.
St andar d Format : As thestandardf ormat expe出 onl ynonnegati ve vari abl es. the
vari abl ex2 is repl acedby a pai r of vari abl es intheformul ati on:

Zト ¥ ; ¥ ドーく!
4 5 6
Fl gure3.8 Graphl cal solution, Exampl e 3.3.
- X
The standardfoml at after converti ngto a mi ni nuzati onprobl em, i ntroduci ngsl ack,
surpl us,佃 dartificial vari abl es, and i ncl udi ngEquati on(3.37) results in
Phase " Tabl e 2: Tabl e 3.13 is 1he secondtable under Phase 1 . The vaJ ueof the
artificial functi onis2, soPhase 1 is not yet over. The EBV is X22・TheLBVi sal ・The
secondrowis 1he pi vot row. Thi s rowwiII be usedfor rowmani pul ati ons ωyi el d the
next table.
Mi n回 rize f - 2x1 +x21 - X22
Tabl e3.13 Exampl e 3.3: Tabl c 2, Phasc I
Su険jectto: gl : X1 +X21- X22 +81 =5
X22 $) U) 8
2 a2 b X) X2)
gl: 2x
+X2 1 - X
辺 +a t =4
。 。 。

g3: X1 + X21- X22 - 82+ a2 = 1
。 。
2 -2 2
。 。
The artifici aJ functi ontobedri ventozerois

3 - 3
。 。
- 2 2 / +2
。 。 。
-2 3 Ar- 2
f = a l   ~
Si mpl ex Tabl e 0: Tabl e3.11伺 p伽悶1hes回 dardfOm18t ina table. It is caII edTabl e
o toaII ow for preprocessi ngwi 1hrespectωthe artifici a1variables sothat a canoni ca1fOl I D
四 nbeob鴎 rved. Addi ng血 esecond卸 d血 i rdrow阻 dsubtrac出 g血 eresul t 合 o m由 elast
rowand repl aci ngthelast rowwi 1hthis computati onwil1yi el dthenext table.
Phase " Tabl e 3: Tabl e 3.14is thenewtableafter therowoperati ons. The val ueof
由巳 artifici aJ func丘onis 0 as bo1hal and a2 釘 enonbasi c vari abl es wi tha vaJ ueof 0
for 白 is iteration. 百l I ssig凶 fies 白 eend of Phase 1. Phase n will start wi ththelast row
( artifici a1釦 ncti on) and thetwo col umns that represent thearti 白 cial vari abl es removed
f romthecurrent table.
Tabl c 3.11 Exampl c 3.3: Tabl e0, Phasc I
T自 bl c3.14 Ex圃mpl c3.3: Tabl e3, PhaseIσ10111)
8) a) S2 a2 b
2 oz
X) X2) X22
X) X2) X22 $) 。1
。 。 。
l o 4
。 。


2 I

。 。
2 一2 2 o

-1 1


- 2 I
。 。 。 。


3 4 4 / +8


。 。

。 。 。 。 。
1 Ar
Phase 1, Tabl e 1: Tabl e 3.12provi des 白 estart of 由 eSi mpl ex method. InPhase 1
th巴 moti vati onis todri vetheartificial functi onto0, whi ch has a val ueof 5. The EBV
iSXl ・TheL BV is a2' The thirdrowis thepi vot row. Row mani pul ati ons wi ththepi vot
rowl eadtoTabl e 3.13.
Tllblc3.12 Exampl c 3.3: Tabl e1, Ph由scl
X, X2' X22
$ , a) $2 a2 b





- 1
。 。 。 。
。 。 。
Ar- 5


Phase 11, Tabl e 1: Tabl e 3.15represents 曲 etabl 巴 tostart thei temti ons for Phase 11.
However, thereareno negati vecoeffi ci ents in曲巴 last row(rowcorrespondi ngtothe
obj ecti vefunction). Scanni ngthe帥 lefu氏 her,1he canoni cal f ormcanbe observed.
百 lerefore,thesol uti onhas beenobtai ned. 百 lesol uti onis
Tabl c 3.15 Exampl c 3.3: Tabl e1, Phasc 11
x, X幻 局 z
$ ,
。 。 。
。 。
2 2
o o

。 。 。 。
4 / +8
= 3; x
=0; x
= 2; sl =4; s2 =0; / =- 8
The val ueof X2 is
= ( X
- x
) = ( 0- 2) =- 2
whi chwas identifiedearlier graphi cal l y.
百 leabove exampl e illustratedtheSi mpl ex method for handl i ngnegati vevari abl es
andequal i tyconstraints. Not巴 自 己 appl i cati onof theSi mpl ex techni queitself di dnot
change. An interesting feature of 白 eprobJ emwas that Phase II was i mmedi at巴ly
obtai nedaftel' Phase 1 wi thout any further iteration, whi ch suggests 白 紙 thesearchfor
the opti mumis taki ngpl ace in Phas巴 1even though 出 e focus is on the artificial
3.4.3 Exampl e 3. 4 -A Four-Varl abl e Pr obl em
Exampl e 3.4presents a probl emwi thfour variables. The procedureisidentical tothe
one inExampl e 3.1, except 出 at 血 ereis no graphi cal sol uti ontotheprobl em.
The Pr obl em: The RI T student-run mi croel ectroni c fabrication facility is taki ng
orders for four i ndi genousl y devel oped ASI C chi ps 出 at can be us吋 in (1) touch
sensors, (2) LCD, (3) pressure sensors, and (4) controllers. There are several
constrai nts on theproducti onbasedon space, equi pment availability, student hours,
and the fact 山 at 出 epri mary mi ssi on of 白 e facility is student training. First,出 e
handl i ng ti me cons回 int , outsi de of processi ng, for all chi ps is 600 hours. Touch
sensors requi re4 ho町 s,L CD 9 hours, press町 'esensors 7 hours, and con住 ollers 10
hours. Second, theti meavai l abl eon thel i thographi c machi nes is about 420 hours.
Touch sensors and LCD r叫 uire 1 hour, pressuresensors 3 hours, and controllers 8
hours. Packagi ng consi derati ons pl ace the maxi mumat 800 vol ume units. Touch
sensors requi re30 voJ urneunits, LCD 40 vol ume units, press町 ' c sensors 20 units, and
con住 oller百 10 units because of 山 eir compact size. All 山 e cons住 aints above are
indi 回 tedper week of operati onof thefacility. 百 enet revenue is$6, $10, $9,叩 d$20
for thetouchsensor, LCD, p問 ssuresensor, and control l er, respectively. 百 lefacility
is interestedinmaxi mi zi ng revenueper week andwoul d ¥ i ketodetermi netheright
mi xof 由巳 four devi ces.
Pr obl emFormul at i on: The formul ati on is strai ghtforward based on the
statements above. Let x ( rep問 sent thenumber of touchsensor chi ps per w田 k,X2白 E
number of LCD, x3 thenumber of pressuresensors, and X4 白 enumber of con佐 ollers.
明 記 obj ecti vefuncti onis
Maxi mi ze: f. 6x. + l Ox
+ 9x
+ 20x
The handl i ngti meconstrai nt canbeexpressedω
gl: 4x
+ 9x
+ 7x
+ IOx
S 600
The avai l abi1i tyof thel i thographi c machi ne canbe devel opedas
g2: x
+ x
+ 3x
+ 8x
s; 420
百 Jepackagi ngconstrai nt is
g3: 30x
+ 40.ら +20x
+ IOx
S 800
Al l desi gnvari abl es ar官 邸 p田 tedtobeg偲 ater 白 肌 zero. As formul atedabove,出 e
probl emsuggests a degreeof i ncompl eteness. τnerearefour desi gnvari abl es and onl y
threeconstraints. 明 lenumber of vari abl es inthebasis canonl y be山 間 e. Hence, at
least one of 出 e vari abl es must have a val ue of zero. Sev巴ral useful addi ti onal
constraints canstiUbe i ncl udedtodefi nea validopti mi zati onprobl emwi tha nonzero
solution. 前 lis is nowa model i ng issue. For probl ems wi tha l i mi tednumber of desi gn
vari abl es, payi ng attention 10 the probl emdevel opment al l ows anti ci pati on of the
sol uti on as wel l 俗 the opportuni ty to 町 oubl eshoot deci si ons f rompractical
consi derati ons. It is left to出 estudent toexpl o陀 this probl emfurther.
St andar d For mat : The obj ecti v巴 functi on requi res a mi ni mumformul ati on.
Mi ni mi ze f. - 6x
1 O ~ - 9x
- 20x
The symbol / has been retai ned for conveni ence even though the dII 宅 cti on of
opti mi zati onhas changed. 百 lecons回 ints areset upusi ngslackvariables SI ・S2,ands
g.: 4x
+ 9x
+ 7x
+ 10x
+ S. = 600
g2: x. + + 3x
+ 8x
+ s2 = 420
g3: 30x. + 40x
+ 20x
+ l Ox
+ S3 = 800
Al l vari abl es are~ O. 百 leformul ati onabove suggests thestandardSi mpl ex method.
S何 'P1ex Tabl e 1: Tabl e3. 16i s出 ei ni 削 tablefor Exampl e 3.4. The canoni caI form
is observabl efromthetableand theinitial basi c feasiblesol uti oncanbe determi ned.
Tabl e3.16 Exampl c 3.4: Si mpl ex Tabl e1
4 81 J"2 SJ
4 9 7 10
。 。
3 8
。 。
30 40 20 10
。 。
-6 -10 -9 -20
。 。 。
Tabl e 3.17 Exampl e 3.4: Si mpl ex Tabl e2
x, X2 X3 X4 s, 82 8J b
2.75 7.75 3.25

l -1. 25

0.125 0.125 0.375


28.75 38.75 16.25
。 。
ー1. 25 275
-3.5 -7.5 -1. 5
。 。

f + 1050
SI ・S2. and $3arethebasi c variables. The EBV is X4 and白 eL BV is $2whi ch is 白 E
mi n泊 l umof {60. 52. 5, 80}. The pi vot r owis the second rowused for the row
mani pul ati ons tol eadtoTabl e 3.17.
Si mpl ex Tabl e 2: Usi ng thepi vot r owi denti fi edin山 elast table,出 eunil vector [0
o 1 O]T under the $2 col umn needs to be transferred to 出 e X4 col umn through
el ementary r owoperati ons. Tabl e 3.17shows 白 ecanoni cal f ormafter compl eti onof
all theoperati ons. The basi c vari abl es areX4. $" 組 dS
3・TheEBV is X2 and白 eL BV
is S3・百 lepi vot r owis the 白 irdrow. The obj ecti ve of the r owmani pul ati ons is to
transfer theunit vector [00 1 O]Tf romthe$3col umn to白 exzcol umn.
Si mpl ex Tabl e 3: Tabl e 3. 18denotes thereducedtablewi 出 thecanoni caJ f ormafter
therequi redr owoperati ons arecompl eted. The basi s vari abl 回 釘 eX2. x4,and SI ・There
m芭 nonegati vecoeffi ci ents inthelast r owsuggesti ngthesol uti onhas been obtai ned.
Fr om白 etable白巴 sol uti onis
= 0, x
= 7. 096, x3 = 0, x
= 51. 61,/ =1103. 22
The sol uti onabove is not sati sfactorybecause theactual deci si onwiIl i nvol vei nteger
val ues for thedesi gnvari abl es. At this ti me i nteger programmi ng is not an opti on. The
adj ustedsol uti onis
= O,x
= 7,x
= O,X4 = 52,/ =1110
Whi l e白 is choi cesatisfies出 econstrai nts g3. g2however needs a Ii ttleelasticity(limit
is 423) tobe satisfied. Not巴 白 at sJ is a basi c vari abl esoconstrai nt gJ shoul dnot be a
probl em.
Tabl e 3.18 Exampl e 3.4: Si mpl cx Tnbl e 3
x, X2 XJ X4
$ ,
$) b
。。 。
-0.2 20
0.032258 0 0.322581 o 0.1290322 -0. 00323 51. 6129
0.741935 1 0.419355
。 。
- 0.032258 0. 025806 7. 096774
2. 064516 0 1. 645161
。 。
2. 25806451 0. 193548 f + 1103. 22
Thi s is a standardLP probl em. Thi s secti onuses MATLAB inani nteracti vesessi onto
soI ve 白 e probl em. 百 le fol l owi ng is a di aryfile for the MATLAB sessi on for 出 is
ex創 npl e. Note the wami ng f romMATイLAB. as weI l 凶 白 e subsequent use of the LP
programusi ng bounds on 曲 e desi gn vari abl es: 百 le bol d italicized statements are
commands usedtoobtai nthesol uti on. The MATLAB prompt is edi tedout .
hel plp
LP Li near pr ogr ammi ng.
X=LP( f ,A,b) sol ves t he l i near pr ogr ammi ng pr obl em:
mi n f ' x subj ect t o: Ax <= b
X=LP( f ,A,b,VL B,VUB) def i nes a set of l ower and upper
bounds on t he desi gn var i abl es , X, s o t hat t he sol ut i on
i s al ways i n t he r ange VLB <= X <= VUB.
X=LP( f ,A,b,VLB,VUB,XO) set s t he i ni t i al st ar t i ng poi nt
t o XO.
X=LP( f ,A,b,VLB,VUB,XO,N) i ndi cat es t hat t he f i r st N
const r ai nt s def i ned by A and b ar e equal i t y const r ai nt s.
X=LP( f ,A,b,VLB,VUB,XO,N,DI SPLAY) cont r ol s t he l evel of
war ni ng messages di spl ayed. War ni ng messages can be
t ur ned of f wi t h DI SPLAY - 1.
[x,LAMBDAJ = LP( f ,A,b) r et ur ns t he seヒ of Lagr angi an
mul t i pl i er s ,
LAMBDA, at t he sol ut i on.
[X,LAMBDA,HOW] LP( f ,A,b) al so r et ur ns a st r i ng how
t hat i ndi cat es er r or condi t i ons at t he f i nal i t er at i on.
LP pr oduces war ni ng messages when t he sol ut i on i s ei t her
unbounded or i nf easi bl e.
c = [- 6 - 10 - 9 - 20] ;
A = [4 9 7 10; 1 1 3 8; 30 40 20 10] ;
b = [ 600 420 800] ;
x = [ c, A, b]
x = l p( c, A , b)
War ni ng: The sol ut i on i s unbounded and at i nf i ni t y;
t he const r ai nt s ar e not r est r i cヒi ve enough.
1. 0e+015 *
- 0. 0000
3. 7067
・ 8. 8391
2. 8513
vl b = [ 0 0
vub = [ 100,
x = l p( c, A,
0. 0000
7. 0968
51. 6129
0] ;
100, 100] ;
b, vl b, vub)
Thi s is thc sol uti onobtai nedby worki ng throughtheSi mpl ex method.
This secti on l ooks bri efl y at addi ti onaJ i deas associ ated wi 山 theL P probl em. First,
出 ereis a di scussi onof thedual probl emassoci atedwi than L P probl em. Dual i ty is
i mpo此 anti n恥 di scussi onof sensi ti vi tyanal ysi s- var削 ioninopti mi zati onsol uti on
due to vari ati on in tfte origil 聞 1parameters of tlte probl em. Fol l owi ng 出 is is a
di scussi onof several pi eces of i nform凶 onthat can be ex回 ctedf rom出 efinaJ tabl e
of theSi mpl ex method.
3.5.1 Prl mal and Dual Pr obl em
Associ ated wi 出 every L P probl em, whi ch wi l l be referred t oωa pri mal probl em,
there is a correspondi ng di 即 1probl em. I n m佃 y ways, 出 e duaJ pl ' Obl emis a
町 叩 sposi ti onof thepri maJ probJ em. 百l I sis meant toi mpJ y that if thepri mal probJ em
is a mi ni mi zati onprobJ em白 eduaJ probl emwi l l be a maxi mi zati onone. lf thepri maJ
probl emhas 11 vari abl es, the dual probl emwi l l have n constrai nts. lf the pri mal
probl emhas m constrai nts, theduaJ probl emwi l l have m vari abl es. For thedi scussi on
of dual i ty,出 e standardpri mCl l probl emis general l y defi ned as a maxi mi zati on
probl emwi th in叫 ualities (though 白 e standard probl emin the L P programmi ng
di scussi on was a mi ni mi zati on probl em[s田 E quati ons (3.10)-(3.12)] 加 d 由 e
∞ns回 Ints were叫 ualities). The s凶 伽 rdpri maJ probl eminthis secti onis defi nedas [3]
Maxi mi ze z: CI XI + CI X2 + . . . + c,,x,, (3.40)
Subj ect 10: al l xl + al2x2 + . . . + al ,.xn b, (3.41)
a21xI +     + . . . + az"xn b2
a" , IXI + anaX2 +・・・ +an",xn
与さ O,j =1,2. . . . . n (3.42)
: ;

Thi s is al soreferred10踊 anormal maxi mumprobl em.
百 ledual of theabove probl emis defi nedas
Mi ni mi ze w: bl YI + blY2+ . . . + b"J ' rn (3.43)
Subj ect 10: al l YI + a21Y2+. . . + arnl Ym cJ (3.44)
al 2Y1 + a22Y2 + . . . + am2Ym c2
al ,J' 1 ... a2"Y2+ . . . + a,tuJ'", さ C"
, i =1, 2,. . . ,m (3.45)
Rel ati ons (3.43) ー(3.45) descri bea normal mi ni mumprobl em. There is al soan i nverse
rel ati onshi p between 曲巴 defi ni ti on of the t wo probl ems above as far as 山巴
i denti fi cati on of the pri mal and dual probl em. If 出巴 latter is consi dered a pri mal
probl em,出 enthef ormer is 出 ecorrespondi ng dual probl em.
百 le fol l owi ng observati ons c回 be made wi 由 民 spect 10 the pai r of probl ems
establ i shedabove:
1. The number of dual vari abl es is 出巴 sameω 出 enumber of pri mal constrai nts.
2. The number of dual constrai nts is thc same as thenumber of pri mal vari abl es.
3. The coeffi ci ent matri x A of the pri mal probl emis transposed to provi de 山 E
coeffi ci ent matri x of thedual probl em.
4. Thei n叫 ualities 釘' Creversedindi recti on.
5. The maxi mi zati on probJ emof the pri maJ probl embecomes a mi ni mi zati on
probJ eminth巴 dual probl em.
6. The cost coeffi ci ents of thc pri mal probl embecome theri ght-handsi devaJ ues
of theduaJ pl ' Obl em. The ri ght-handsi devaJ ues of thepri mal become thecost
coeffi ci ents of thedual probl em.
7. The pri mal and duaJ vari abl es both sa出 fythenonnegati vi ty condi ti on.
Exampl e 3. 5百 le fol l owi ng exampl e, adapted and modi f i ed f romRef erence 3, is
used toiII ustratethepri mal /dual versi ons of the same probl em. The probl emis sol ved
usi ngtechni ques devel oped inthis chapter.
而 e Probl em: The l ocal col l ege' s School of A meri can Craf tsman has deci ded 10
parti ci pateseri ousl yina 10caJ chari tyevent by manuf acturi ng speci al commemorat i ve
desks, tabl es, and chai rs. Each 1ype of fumi ture requi 陀 s l umber as weIl as t wo types
of ski l l edl abor: f mi shi ng加 dc細' Pentry. Tabl e 3. 19document s theresources needed.
The 10caJ l umber cooperati vehas donated 48 board feet of l umb巴r. Facul ty and staff
have vol unt田 町 d 20 fi ni shi nghours and 8 carpentry hours. The desk wi l l be sol dfor
Tnbl c 3.19 Exampl e 3.5: Rωources
Resources Desk Tabl 巴 Chai r
Lumber(bo釘 dfeet} 8 6
Fi ni shi nghours (hours) 4 2 1. 5
Carpentry(hours) 2 1. 5 0.5
$60,出 etabl efor $30, and出巴 chai r for $20. The school does not expect to seJ1mor e
出 印 日 vetables. The school woul d liketomaxi mi z巴 rev巴nuecoJ1ected.
8tandard For maト Pri mal Probl em: Def i ni ng Xl as the number of desks
produced, X2 as the number of tabl es manuf actured, and X3 as the number of chai rs
manuf actured, thepri mal probl emcan be expressedinstandardf ormat 悩
Maxi mi ze z: 6 0 x, + 30X2 + 20X3
Su句 ect to: g, : 8x, + 6x
+ X3 :::; 48
g2: 4 x, + 2x
+ 1. 5X3 :::; 20
g3: 2x, + 1. 5x2 + 0. 5X3 ::; 8
1!4: X2: : ; 5
Xl>X2 ,x 3  
(3. 46)
(3. 47c)
The sol uti ontothepri maJ probl emis avai1abl einTabl es 3. 20-3. 22. Tabl e 3. 20 is th巴
initi aJ table. The EBV is x, and theL BV is $3・Thepi vot r owis the出 i rdrow. The
subsequent r owoperati ons determi ne Tabl e 3.21. I nthis tabl e,出 eEBVi sx3' t heLBV
issz,zmd thepi vot r owis theSEcond Eow. Tabl e 322i s theanal tabl eestabl i shi ng血 E
opti mal val ues for the vari abl es. xt ,白 e opti maJ number of desks, equal s 2. ~ , 血 e
Tabl e 3.20 Exnmpl e 3.5: Pri mal Probl cm, Si mpl ex Tabl e 1
x, X2 X) $ , $2 S) $4 b
B 6
。 。
o 48
4 2 1. 5
。 。
o 20
2 1. 5 0.5
。 。 。
。 。 。 。 。
- 60 - 30 - 20
。 。 。 。
f +O
l l i l 1
「 主
d d d f
S 堅
l 準1
Tabl c 3.21 Exampl c 3.5: Prbnal Probl cm, Si mpl cx Tabl c 2
X, X2 X3
$ , $2 $3 $4 b
。 。

4 o 16


- 2

0.75 0.25
。 。

。 。 。 。 。
o 15 - 5
。 。

f +240
opti mal number of chai rs tobe made, is 8. No tabl es wi l l be made. 百 letotaJ revenue
i s$280. 00.
81 . ヨ ndardFormat ー Dual Probl em: Whi letheduaJ probl emcan b巴 巴 xpressed佃 d
setup m田 h阻 ica1l y,it is 函 館 cul t toassoci ate出 edesi gnvari abl es ina di rect m佃 ner
aswasdone泊 thepri maJ probl em. Usi ng y" Y 2, Y3, and Y4 as thedesi gnvari abl es, the
probl emcan be formul ated as in Equati ons (3.48) and (3.49) bel owusi ng the
defi ni ti onat 由 ebegi nni ng of this secti on. Some associ ati ons arepossi bl einlight of
transposi ti onof 出 ecost coeffi ci ents and theconstrai nts. For exampl e, thevari abl ey,
can be associ ated wi th the l umber constrai nt because in the dual probl emits cost
coeffi ci ent is the l umber constrai nt of 出 epri mal probl em. Ina si mi l ar manner, Y2 is
associ atedwi ththefi ni shi nghours, Y3 wi th出巴 carpentryhours, and Y4 wi l hthespeci aJ
tabl econstrai nt . 明記 obj ecti vefuncti onof thedual probl emis expressed inEquati on
(3.48). Invi ewof 白 e f ormof the expressi onof the obj ecti vefuncti on w,血 E毘 is a
possi bi l i ty that 出 e desi 伊 V紅 i abl 民 in the duaJ probl emcan have 四 econorni c
i mpl i cati onas fol1ows:
y, uni t pri cefor a board foot of l umber
Y2 uni t pri cefor a fi ni shi nghour
Y3 uni t pri cefor an hour of carp巴 ntry
y4 白 i scannotbe白 euni t pri ceas 也 edesk has a pri cedef med i nthepri maJ
pr obl em- maybe a speci aJ pri cefor thedesk resource
Tabl e 3.22 Exampl e 3.5: Pri mal Probl em, Si mpl ex Tabl e 3
X, X2 X3
$ , $2 .1' 3 $4 b

- 2

2 - 8 o 24
o - 2

2 4

1. 25
。 。
-0.5 1. 5

o o o
。 。
1 5

5 o o 10 10 o f +280
The dual canbeexpressedas
Mi ni mi zew: 48YI + 20Y2+ 8Y3+ 5Y4 (3.48)
Subj ect to: h
: 8YI + 4Y2+ 2Y3+ OY4注 60 (3.49a)
: 6YI + 2Y2+ 1.5Y3+ lY4  30 (3.49b)
: lYI + 1. 5Y2+ 0.5Y3+ OY4  20 (3.49c)
YI>Y2, Y3  0 (3.50)
l ntroduci ngsurpl us vari abl es (SI>S2, S3) and artificial variabl 巴 s(al ・ 。 2' a3) 出 edual
probl emis expressedas a standardLP probl em:
Mi ni rni zew: 48YI + 20Y2+ 8Y3+ 5Y4 (3.48)
Subj ectto: h
: 8Yl + 4Y2+2Y3- SI + al = 60 (3.51a)
h2: 6)'1+ 2Y2+ 1. 5Y3+ lY4- S2+ a2= 30 (3.5Ib)
: lYl + 1. 5Y2+ 0.5Y3+ OY4- S3+ a3= 20 (3.51c)
[n Phase 1 of thetwo-phase approach,山 eartificial vari abl es areremoved from白 E
basis usi nganar甘白 cial cost functi onA defi nedas
Mi ni mi zeA: a
+ a
+ a
The Phase [ computati ons aredi spl ayedinTabl es 3.23-3.26. Tabl e 3.23is identified
as th巴 si mpl exTabl e 1, and it inc1udes some preproccssi ngby whi ch a" a2, and a3釘 e
made 山 ebasis variables. As a remi nder, the last row問 presents 山 earti ficial ∞st
functi on. Fromi nspecti onof Tabl e 3.23, YI is theEBV. The LBV is a2and由巳 pi vot
rowisthesecondrow. Fol l owi ngthestandardrowoperati ons, Tabl e 3.24is obtai ned.
[nthis table, theEBV isY2(也 elargest negati vecoeffi ci ent).ηl eLBV i sa3(白 E 出 ird
Tabl e3.23 Example3.5: Dual Probl em, Phose1, Tabl e1
Y1 Y2 Y3 Y4 51 52 S)
Q2 a) b
8 4 2
。 。 。 。 。
6 2 1.5
。 。 。 。
1. 5 0.5
。 。 。 。 。
48 20 8 5

。 。 。 。
-15 ー7.5 ー4
。 。
o A - 110
Tabl e3.24 Exampl e 3.5: Dual Probl em, Pbase1, Tabl e2
Y1 Y2 Y) Y4 51 52 S) a1 a2 a) b

1. 3333 0 -1. 3331 1. 3333

-1. 3333

0.3333 0.25 0.1666 o - 0.1666 o


o 1.1666 0.25 -0.1667

0.1666 -1 o - 0.1666 15

4 4 -3


o -8 o w- 2柑
o -2.5 - 0.25 1. 5 一1. 5


A- 35
rowcontai ns therni ni mumof ([20/ 1. 333] , [510.3333] , [15/ 1. 6667]}). 百 lepi vot rowis
row3. Tabl e3.25rep隠 sentsth巴 next table. 百 l eEBVi ss
Noteinthiscぉ εtherearetwo
candi dates for EBV (S2・S3)' S2is chosenarbitrarily. 百 leLBV is al' The pi vot rowis 出 E
街百 trow. After the[珂 uiredrowmani pul ati ons, Tabl e3.26isgenerated. Scanni ngthelast
row, th釘 eareonl ypositivecoefficients. Phase1 isnowcompl ete. The artificial variables
and白 巴 紅 tificial cost functi oncanberemoved fromtheLP probl em.
El i rni nati ng白 eartificial vari abl es and theartificial cost functi onf romtheprobl em,
Phase II is startedwi thTabl e 3.27. Not巴 that Tabl e 3.27contai ns no i nformati onthat
is not al ready avai l abl e in Tabl e 3.26. It onl y provi des an unc1uttered table for
conti nui ng th巴 appl i cati on of the Si mpl ex l echni que, and therefore is not really
necessary. InTabl e 3.27, theEBV isS3・TheLBV isYl after a toss between Yl and sl'
The pi vot rowis thesecondrow. Thi s rowis usedtoobtai nTabl e 3.28. Inthis table,
note that the val ue of the basi c vari abl e SI is O. Thi s is cal l ed a degenerate basi c
vari abl e. The EBV for thenext tableisY3・TheLBVi ss
Thepi vot rowis 白 es田 ond
row(note: the pi vot rowcan be 曲 erowwi th the degenerate basi c sol uti on if the
col umn of theEBV has a posi ti vecoeffi ci ent in曲 at row). Tabl e 3.29is 曲 efinal table
and contai ns theopti mal solution. The final sol uti onof thedual probl emis
Y; =0,活=l O , y; = 10,見 =0,s; =5, w・=280
To summari ze, thesol uti onof thepri mal probl emis
x; = 2, xi = 0, x; = 8, s;p= 24,勾 p=5,z*=280
・f hesubscri pt “p" is added toassoci atethesl ackvari abl ewi th白 epri mal probl em.
The sol uti onof thedual probl emis
Y; =0,点=10, Y ; = 10, Y4= 0, si = 5, W・=280
Some For mal Observat i ons
These observati ons arejustifiedmorc formal l y inmany books on linear programmi ng,
some of whi ch have been inc1uded inthereferences at theend of this chapter. Here,
theformal results areel aboral edusi ngthesol uti on 10theexampl e di scussedinthis

Tabl e 3. 25 Exampl e 3.5: Dual Probl em, Phase 1, Tabl e 3
Y2 Y3 Y4
$ ,
$2 S3 a, a
2 a3 b

o - 0. 2857 -1. 1429
1.1429 1.1429
-1. 1429 -l . ¥429
2. 8571

0. 1786 0. 2143

- 0. 2143 02857

0. 2143
- 0. 2857
0. 7143 。
0. 2143 - 0. 1429

0. 1429 -0. 8571

- 0. 1429
0. 8571
12. 857

o - 4.8571 ー2. 4286


ー7. 4286 - 3. 4286
w - 291. 43
。 。
0. 2857
-1. 1429 -1. 1429

2. 1429 2.1429
A - 2. 857
Tabl e 3.26 Exampl e 3.5: Dual Probl em , Phase 1, Tabl e 4
Y2 Y3 Y4
$ ,
$2 S3
  a3 b

o -0. 25 - 0.875 0. 875 2. 4999

0. 125 -6E-O岳 - 0. 1875 6E- 06 0.5 0. 1875 - 6Eー 凶 - 0.5 1. 25

0.25 4E- 05 0. 125 - 4E- 05 - 0. 125 4E- 05 12.5
。 。
-3. 0001 4. 9998 6. 4998 0. 0002 - 3. 9998 - 6. 4998 -0. ∞02 3. 9998 w- 3I O
。 。 。 。
。 。

Tab1e 3.27 Exomp1e 3.5: Duo1 Prob1em, Phasc D, Tab1c 1
YI Y2 Y3 Y4 SI S2 S3 b
。 。
-0. 25 -0. 875 2.5



0.5 1. 25




-3 5 6.5

w- 31O
secti on. No proofs 創 'egi ven. A¥l of theobservati ons may not be rel evant toExampl e
3.5di scussedabove, but areincI uded herefor compl eteness.
1. The pri mal and dual probl ems have the sam巴 val ue for theopti mal obj ecti ve
functi on, 7." = w" = 280. This is true, however, onl y i ftheprobl ems have opti mal
sol uti ons.
2. (f x is any feasi bl esol uti ontothepri mal probl cm. and y is 叩 y feasi bl esol ul i on
tothc dual probl cm. 出 叩 w( y) ~ 7. (x). Thi s featureprovi des an esti matc of the
bounds of thedua1 opti mal val ue if a fcasi b1e pri mal sol uti on is known. Thi s
rcsul t al sohol ds for thereversecasewhen a feasi bl 巴 dual is known.
3. I f出 e pri ma1 probl emis unbounded. the dual probl emis i nfeasi bl e. The
unbounded probl emi mpl i es 白 紙 theobj ecti vefuncti onva1ue can be pushed to
infinite limits. 百 lis happens if the feasi bl e domai n is not c1osed. Of coursc,
practi cal consi derati ons wi l l l i mi t thc obj ecti vefuncti onval uetocorrcspondi ng
Ii mi ts on the desi gn vari abl es. The i nverse rel ati onshi phol ds too. If the dua1
probl emis unbounded, thepri mal is i nfeasi bl e.
4. If the ith pri mal constrai nt is an equa1ity constrai nt, the ith dual vari abl e is
unresl ri ctedinsi gn. The rcversehol ds too. If thepri mal vari abl eis unrcstri cted
insi gn. 出 enthc dual constrai nt is an equal i ty.
5. Obtai ni ng pri mal sol uti onf romdual sol uti on:
(i) If theithdual constrai nt is a strict i nequal i ty, then出 ei山 pri mal vari abl eis
nonbasi c (for opt i mumsol uti ons onl y). Fr omthedual sol uti on, thesecond
Tobl e 3.28 Exampl e 3.5: Dual Probl em, Phose 11, Tnbl e 2
YI Y2 Y.l Y4 SI S2 5J b
- 2

-0.5 -0.5
。 。



2 0.5

-0. 25
。 。

- 2 5 5
。 。
I V- 300
Tab1e 3.29 Examp1e 3.5: Dua1Prob1em, Phose 11, Tob1e 3
YI Y2 Y.l Y4 目.fl S2 SJ b
。 。
-1. 25 2 5
。 。
-1. 5

4 10

o 0.5 o - 2 10
。 。
5 2

8 .f - 280
constrai nt (3. 51b) is sati sfi edas an i nequal i ty. Theref o陀, thesecond pri mal
vari abl ex 2 is nonbasi c 剖 d出 is is evi dent inTabl e 3.22.
(ii) If the ith dua1 vari abl e is basi c, then the ith pri mal constrai nt is a strict
equal i ty. Fr omTabl e 3. 29, Y2 and Y3 are basi c vari abl es, therefore (3.47b)
叩 d(3.47c) must be equal i ti es. Thi s is i ndecdtruc.
Thesc rel ati ons can al sobc cstabl i shcdby val ucs of sl ack vari abl es.
6. Rccoveri ng pri mal sol uti on f romfinal dual table: When lhe pri mal and dual
probl erns are in the standard f orm, the val ueof thc ith pri mnl vari abl eequal s
thereduced coeffi ci ent (that is, thecoeffi ci ent inthelast r owof thefinal tabl c)
of the sl ackl surpl us vari abl e associ ated wi th the ith dual constrai nt. InTabl e
3. 29, the val ues of the reduced cost coeffi ci ent correspondi ng to the surpl us
vari abl es SI ・S2' and S3 are 2, 0, and 8, respecti vel y. Th回 e are preci se¥y the
opti mal val ues of thepri mal vari abl es.
7. lf 出 e ithdual vari abl e is nonbasi c, the val ue of its reduced cost coeffi ci ent is
the val ue of 由巳 sl ackl surpl us vari abl eof thecorrespondi ngpri ma1constrai nt .
InTabl e 3. 29, YI and Y4arenonbasi c vari abl es wi 山 reducedcost coeffi ci ents of
24 and 5, respωti vel y. These arc theval ues of SI (Sl p) and S4 (S4p) inTabl e 3.22.
8. Obtai ni ng duaI sol uti on f rompri mal sol uti on: When the pri mal and dual
probl erns areinthestandardf orm, theval ue of theithdual vari abl eequal s the
reduccd coeffi ci ent ( 白紙 is,thecoeffi ci ent inthelast r owof the final table) of
the sl ackl surpl us vari abl e associ ated wi th 出 e ith pri mal constrai nt. ln Tabl e
3. 22, the val ues of the reduced cost coeffi ci ent correspondi ng to the surpl us
vari abl es SI , S2' S3, and S4 副 官 0,10, ¥0, and 0, respecti vel y. These arepreci sel y
theopti mal val ues of thedual vari abl es inTabl e 3.29.
9. If theithpri mal vari abl eis nonbasi c, theval ueof its reduced cost coeffi ci 巴 nt is
theval ueof thesl ackl surpl us vari abl eof thecorrespondi ngdua1constrai nt . In
Tabl e 3. 22, X2 is a nonbasi c vari abl ewi threduccd cost coeffi ci ents of 5. Thi s is
the val ue of S2 inTabl e 3.29. Si nce XI and X3 arebasi c, wi ththercduced cost
coeffi ci ent of O,出 ecor児 spondi ngsl ackl surpl us SI and S3 vari abl es will b巴 zero,
as observed inT油 le3.29.
The above list referred to the pri mal and dual probl ems instandard form. A si mi l ar
list can bc obtai nedfor nonstandardform. There will be appropri atemodi fi cati ons for
negati veval ues of vari abl es ぉ wel l as equal i tyconstraints. The listedreferences can
be consul tedfor further i nformal i on.
3. 5. 2 Sensl tl vi ty Anal ysl s
The sol uti on10theL P probl emis dependenl on theval ues for 出 ecoeffi ci ents, c, b,
and A (alsotermedparameters) , i nvol vedintheprobl em. Inmany practi cal si tuati ons
these par創 nel ers are onl y known approxi matel y and may change f romits current
val ue for any number of reasons, parti cul arl y afler a sol ul i onhas beenestabl i shed.
I nsteadof recomputi ng a newsol uti onit is possi bl etoobtai na freshone basedon the
exi sti ngone and its final Si mpl ex table. Thi s i ssueis si gni fi cant inpractical probl ems
wi ththousands of vari abl es and constrai nts. The adaptati onof a n巴 wsol uti onwi thout
re-sol vi ng the probl emis cal l ed sensitivi.η anal ysi s. In L P probl ems, sensitivity
anal ysi s refers todetermi ni ngtherangeof parameters for whi ch theopti mal sol uti on
still has thesame vari abl es inth巴 basi s,even thoughval ues al Ihesol ul i onmay change.
Exampl e 3.1is usedfor 出 is di scussi on. Fi gures are出 edfor i1I ustrati onrather than
a formal proof. Fi gures are not useful for more thantwo variables. General l y, mosl
compul er codes that sol ve Ii near programmi ng probl ems also performsensitivily
anal ysi s. Revi si ti ng Exampl e 3.1, it w悩 necessary 10 identify the number of
component pl ncemcnt machi nes of l ypeA and B. The obj ecti ve is to maxi mi ze山 e
number of boar由 tobe manufactured. Constrai nt g I represents theacqui si ti ondol l ars
avai l abl e. Cons紅 'aint g2represents thefloor spaceconstrai nt. Constrai nl g3represents
the number of operators avai l abl e. The probl emstatement and the results are
reproducedbel ow(note白 紙 X3' X4and Xs arethcsl ackvari abl es)
The sol uti onis
Mi ni mi ze j(X): - 99Ox
- 9∞1X2- 5250
subj ecI to: gl (X): O.4xl - + ・ 0. 6X2+ X3= 8.5
g2(X): 3xI - X2+ X4= 25 (3.7)
g3(X): 3xI + 6x
+ X5 = 70 (3.8)
O   x
ミ o x
0, x
0, X
0 (3.9)
x ; = 10.4762, X; = 6. 4285,x ; = 0.4524,] = - 21437.14
Changi ng Cost Coeffi cl ent Val ues (c): First consi der the e仔 ect of changi ng
thecost coeffi ci enl s of thedcsi gnvariablcs. If CIOthecoeffi ci cnt of desi gnvari abl eXI ・
is changed f rom- 990, will theset of basi s vari abl es rcmai n thesame?
g s
舌 ー 1白

Sensltlvlty10Cost Coefflclent
電 ? ・ ・ ・ ・ ・ ・ ・ ー ー
司・. - 一 一 ー ・ ・ ・ ・ ・ ・ ・ ー ー ー -
一 一 ー 司 ・ - - ー - - -
‘ ・・- 句、 司・・・ち片・・・‘ ー 16150
・ . 句
-- 司、ー 16150
c1= ー 200- optl mumchanged10A
5 10 15
Machl nes of Type A
円 guro3.9 Sensitivityanalysis, changi ngcosl c佃 町 icient .
Changi ng thc cost coeffi ci ent changes the sl ope of the line representi ng 出 e
obj ecti ve functi on血 at may l eadtoa change in批 sol uti on. For exampl e, Equati on
(3.5) can be wri ttenぉ
= ー {990/ 900) X I ー ([5250+ j]/ 900)
If CI were to be made - 1900,出 en the sl ope of 白 e line will be ー(1900/ 900). The
sensitivity due to CI can be experi enced by runni ng Sensl tl vl ty_cost. m.
change wi l h respect 10 the ori gi nal obj ecti ve functi on is shown in ani mati on. The
ori gi nal sol uti onismarked on thefi gureby a circle. The ori gi nal sol ul i onisunchanged
when t hema伊 i tudeof thesl opeis rai sed, for i nstanceCI = -1900. On theother hand,
if CI were 10be made - 200, Ihedottedlinei ndi cates thenewcost lineand inthis case
thesol uti onwi1l change topoi nt A. This is shown inFi gure 3.9白 紙 is 山 E問 sult of
runni ngthem- fiI e. The floor constrai nt is no l onger bi ndi ng. It cunbeestabl i shedby
si mpl e ca¥cul ati ons 白 紙 infinitely many sol uti ons are possi bl e if CI has a val ue of
- 450. InIhis casc thecost functi onis parallel tothebi ndi ngcons凶 int 83' Fromthis
di scussi onit is apparent 白 紙 10keep thel ocati onof theori gi nal sol uti onunchanged,
CI must be greal er than -450. Such anul ysi s is possi bl e for all Ihe other cosl
coeffi ci enl s i nvol ved in 山 is probl em. In parti cul ar, the anal ysi s shoul d indicate for
lFiles 10bcdownl ondedfromtl 1e websilcnrcindicnledbyboldface阻 同 時 目 ftype
what range of cost coeffi ci ents the sol uti on woul d still remai n where it is. This is
referredtoas determi ni ngsensitivitytocost coefficients.
Change Inthe Resour ce Ll ml ts (b Vector): When theval u巴Son thcri ght-hand
side changcs, the constrai nt I i nesl pl an岱/ boun血 ries are moved paml l cl to themsel ves.
前 lis changes thefeasibleregion. Th巴 refore,theopti mal vaI ues may aIso ch叩 gedue to
thechanges in出 efl回 sibl 巴 隠 :gion. Once agai nthechanges canbe illus回 tedby runni ng
血 巴 S ens耐 vi町一巾 s. m. Fi gure3.10iII us回 tes thechange intheri ght-handsidevaI ueof
出 efrrst ∞nstr泊 nt. b.. 針。 mits vaI ueof 8.5to6.5. This chang巴 caus郎 副 tobecome an
activecons佐 aint . Inthis case, theopti maI sol uti onhas moved fromA toB. Note that in
theoriginaI probl emg 1 was not active. In血 enewformul ati onit is. However, insensitivity
anal ysi s,出 eprobl cmis todi scover thcrangeof b. so白 紙 thesol uti onstill has 由 esame
vari abl es in出 ebasis. lf b
we陀 toremai nabove 8. 0475, thenthesol uti onwoul d still be
at A卸 dgl will not be anactivecons町ai nt
Change Inthe Coeffl cl ent Matrl x A: A change in the coeffi ci ent matri x is
si mi l ar in effect to that due to the change in the cost coeffi ci ent , whi l e directly


o -5

号 ・ 10

b1: 6. 5- opti mumchanged- acti veconstrai nt
5 ro 15
Machl nes 01Type A
F1gure3.10 Sensillvityanalysis, changl ngright・handside.
Sensitivitytoconstrai nt coel fi ci ents (a11)
l   i l ~ O
- ‘ ‘ ・
- ・ .
- ‘ ....
. . ・ .
ーニご": - - - -
一 『 司 司 邑
ト ~ ・

ち 」
a1' :0.6- optl mumchanged
。 s 10 15
Machi nes of Type A
Fl gure3.11 Sensillvilyanalysis, changi ngcosl coefflclenl .
i mpacti ng由巳 probl cmby changi ngthefeasi bl eregi on. These changes alsodepend on
whether the vari abl e in the col umn is a bnsi c vari nbl c 01' a nonbasi c variublc.
Sensl tl vl ty_coef f . millustrates the result of changi ng the coeffi ci ent all' An
i ncrease inthe vaI ue moves the opti maI sol uti on to B, whi l e a decrease l eaves the
sol uti onunchanged. Inbothsi tuati ons, XI and x2 arestill part of 出 ebasis. Inthefirst
case, g. becomes a bi ndi ng constrai nt. Thi s is illustrated in Fi gure 3.11. Thc
coeffi ci ent is in the flf冨t col umn, whi ch corresponds to a basi c variable. Changi ng
coeffi ci ent val ues in the first col umn in other constrai nts wi1¥ yi el d si mi l ar
i nformati on. The reader is encouragedtotryother changes inthecoeffi ci ent usi ng出 c
m-fi l es avai l abl einthecode secti onof this chapter.
1. Dantzi g, G. B. , Li near Programmi ng and Extensi o叫 Pri nceton University Press,
Pri nceton. NJ . 1963.
2. Luenberger, D. G. , Li near and Nonl i near Programmi lJ g・Addi son-Wesl ey. Readi ng.
MA. 1984.
3. Wi nston, W. L., l ntroducti on to Mathemati cal Programmi ng. Appl i cati ons and
AI gori thms, Duxbury P陀 ss,Bcl mont, CA, 1995.
4. Arora, J. S. ,l ntroducti onωOpti mal Desi gl l . McGr aw- Hi I I , NewYork. 1989.
5. Wi l1i ams, G.. Li near AI gebra wi thAppl i cati ons. Wm. C. Brown Publishers, Dubuque.
I A, 1991.
6. Nob1c. B. . 飢 dDani el , J. W. , Appl i edLi near AI gebra. 針 enti α-Hal l ,Eng1ewood Cliffs.
NJ . 1977.
(For all two-vari abl e probl ems, provi de a graphi cal defi ni ti onl sol uti on of thc
probl em. )
3.1 Sol ve thefol l owi ngI i near programmi ng probl em:
Mi n ft.. xl ・X:z}: XI +X2
Sub: 3xI - X2 3
XI +X2S 4
3. 2 Sol vc thefol l owi ngl i near programmi ng probl em:
Max f(x )o xz): XI +巧
Sub: 3xI 3
XI +2x
S 5
XI +X2S 4
XI 2! 0; X2 2! 0
3.3 Sol veth巴 f ol l owi ngl i ncarprogrammi ngprobl cm:
Mi n f(x " X2): XI - X2
Sub: 3xI - X2 3
XI +2x2S 5
XI +X 2 S 4
XI 2! 0; X22! O
3.4 Sol ve thefol l owi ngI i near programmi ng probl em:
M田 f(x " X2): XI + X2
Sub: 3xI - X2 2! 3
XI +2x
S 5
XI +X2S 4
XI 2! 0; X22! 0
3.5 Sol ve出 efol l owi ngI i near programmi ng probl cm:
' ‘ A':_
1' 1.. 、 ・ u ・..
Sub: 3xI - X2 S 3
XI 2! 0; X2u町 四 回 c旬 dinsi gn
3.6 The l ocal bookstore must determi ne how many of each of thefour newbooks
on photoni cs it must order tosati sfythencwi nterest generatedinthedi sci pl i ne.
Book 1 costs $75, wiII provi de a profi t of $13, and 問 qui res 2 i nches of shel f
spacc. Book 2 costs $85, wi1l provi de a profi t of $10, and requi res 3 i nches of
shel f space. Book 3 co蜘 $65,wi l l provi de a profi t of $8, and requi r, ω1 i nchof
shel f space. Book 4 costs $100, wi l l provi de a profi t of $15 and requi res 4 i nches
of shel f space. Fi nd thenumber of each typethat must be orderedtom蹴 回 ize
profi t . Total shel f space is 100 i nches. Total amount avai l abl e for ordcri ng is
$5000. It has been deci ded toorder at 1east a to凶 of ¥0of Book 2 and Book 4.
3.7 The l ocal communi t y col l ege is p1anni ng to gr ow出巴 bi otechno10gy offeri ng
through newfcderal and state grants. An ambi ti ous programis bci ngp1anncd
for rccrui ti ngat least 2∞stude附 f rominand out of state. They aretorecrui t
at least 40 out-of-statc students. They wi11attempt to陀 氾 ruit at least 30 students
who are in the top20%of their graduati ng hi ghschool c1ass. Current fi gu問 S
i ndi cate白 紙 about 8 % of theappl i cants f rominstate, and 6 % of theappl i c叩 ts
f romout of statebel ong to白 ispooL They al sopl antorecrui t at least 40 students
who have APcourses inbi o10gy. The datasuggest that 10%and 15%ofi n-state
and out-of-stal e appl i cants, respecti ve1y, bcl ong to 出 is pool . They anti ci pal c
that theaddi ti onal cost per student is $800 for cach i n-statc sl udent and $1200
for each out-of-statc student Fi nd thcir actual enrol l ment necded tomi ni mi ze
cost and their actual cost.
3.8 Fi gure3. 12rep陀 sents an opti mi zati onprobl emtodel ermi ne themax . i mumtOla1
number of 出巴 smaI1er rectang1es 山 at wil¥ fit wi thi n thc l arger one. The
di mensi ons are i ndi cated on 白 e fig山 " C. τne number of the 1arger r芭 ctang1e
shoul d be at 1east 5 mor c than the sma¥1er oncs. Thc rectangl es cannot be
rol ated. No fracti onal r巴cl angl es areal1owed.
(Opti onal : conf i rmaIl sol uti ons usi ngtheOpti mi zal i onTool box . f romMATI . AB.)
t 仁コ
2. 5口 70
Rguro 3.12 Probl 日 m3. 8.
Opti mi zati on probl ems whose mathemati cal model s are characteri zed by nonl i near
equati ons are cal l edNonl i near Programmi ng 例 LP) probl ems. In Chapt巴r 3 it wぉ
noted血 at these probl ems al so fell i nto the category of mathemati cal programmi ng
probl ems. Engi neeri ng desi gn probl ems are mostl y nonl i near. In Chap旬 r 2 several
probl ems were exami ned graphi cal l y nnd it was evi dentthat curvatureand thegradi ent
of the functi ons i nvol ved had a si gni fi cant i nfl uence on the sol uti on. In subsequent
chapters we wi l l conti nue to center 由 e di scussi on of opti mal i ty condi ti ons and
numeri cal tcchni ques around two-vari abl e probl ems because 出 e i deas can al so be
expressedgraphi cal l y. Extensi on to mor e thant wo vari abl es is qui testrai ghtforward
and is most si mpl e when thep問 sentati onis made throughtheuse of vector a1gebra.
MATI . AB wi l l be uti l i zedfor all graphi c needs.
Tradi ti onal l y, there is a bottom-up presentati on of materi al for nonl i near
opti mi zati on. Unconstrai nedprobl ems are di scussed first fol l ow巴d by constrai ned
probl ems. For constrai ned probl ems 山 e equal i ty constrai nedprobl emis di scussed
first. A s回 ilar progressi on is observed wi th 問 gard to the number of vari abl es. A
si ngl e-vari abl e probl emis i ntroduced, f ol l owed by t wo vari abl es whi ch is 由 en
extended to a genernl probl emi nvol vi ng" vari nbl es. Thi s order al l ows i ncremental
i ntroducti onof newconcepts, but pri mari l y al l ows thecreati ve use of exi sti ngrul es
toestabl i shsol uti ons totheextended probl erns.
An anal yti cal foundati onisessenti al tounderstandand establ i shthecondi ti ons 山 at
the opti mal sol uti on will have to satisfy. 百 lis is not for the sake of mathemati cal
curi osi ty, but is 叩 essenti nl component of the numeri cal techni que: notabl y the
stoppi ng criterin. The necessary mnthemnti cal defi ni ti ons and i l l ustrati ons are
i ntroducedinthis chapter. Ref erences ar巴 nvai l nbl efor refreshi ngthecal cul us and the
numeri cal techni ques essenti al 10thedevel opment of NLP [1,2). The books fami l i ar
to the render shoul d do admi rabl y. This chnpter al so i ntroduces the symbol i c
compuωtioll (computer al gebra) resource avai l nbl e in MATLAB, namel y, Symbol i c
Mat h Tool box [3).
InNL P it is not essenti al 山 at all thefuncti ons i nvol vedbe nonl i ncar. It is suffi ci ent if
j ust onc of t hemis nonl i near. Ther芭 aremany exampl es inengi neeri nginwhi ch onl y
白 eobj ecti ve functi onis nonl i near whi ¥e theconstrai nts arelinear. lf inthis case the
obj ecti vefuncti onis a quadrati c functi on, theseprobl ems aretermed lillear quadral i c
probl ems (LQP). Opti mi zati on probl ems for 恥 most p制 rely on experi ence to
i denti fythemathemati cal model compri si ng of thedesi gnvari abl es, obj ecti ve, and the
constrai nl s. A knowl edge of engi neeri ng, or the appropri ate di sci pl i ne, is al so
essenti al tocstabl i sha matl l emati cal model . Pri mari l y, this i nvol ves determi ni ngthe
functi onal rel ati onshi ps among thc desi gn vari abl es. Thc remai ni ng task then is 10
establ i shthesol uti on.
How does one establ i shthesol uti ontothenonl i near opti mi zati onprobl em?
In mathemati cs (nfter nll at this stagc Lhere is a mathemati cal model for the
probl em) thesolutioll is obtai nedby sati sfyi ngthenecessaηand SI{伊 cient condi ti ons
rel ated to the cl ass of   The necessary condi ti ons ar官 those rel ati ons that a
C佃 di datefor theopt i mumsol uti onmllst sati.φ. I f i t does,出 en,and this is i mportant,
it may be an opti mal sol uti on. To qual i fy a desi gnvector XP (X represents thedesi gn
vector) as an opt i mum, it must snti sfy addi ti onnl rel ati ons cnl l ed the sujJi cient
condi ti ons. Therefore, 佃 opt i mumsol uti onmust sati sfybo出 necessaryand suffi ci ent
condi ti ons. Thi s chapter establ i shes these condi ti ons for the opti mi zati onprobl em.
Exampl e 4.1 is establ i shednext and is used inseveral ways inthe remai nder of 出 E
chapter 10 devcl op the condi ti ons menti oned above. Once 1hωe condi l i ons are
avai l abl e, thenumeri cal techni ques inopti mi zati onwiUi ncorporatet hemtoestabl i sh
thesol uti on.
4. 1. 1 Pr obl emFor mul at l on - Exampl e 4. 1
The probl emis 問 stri ctedto t wo vari abl es todrawgraphi cal support for some of 白 E
di scussi ons. There aret wo constrai nts, whi ch, duri ngthedevel opment of this chapter,
may swi tchbetwccn equal i ty and i nequal i tyconstrai nts toi¥l ustrate問 l atedfeatures.
Probl em: Fi nd thc rectangl eof thc l argest ar回 (in血巴 posi ti vequadrant) 白 atcanbe
町 anscri bedw地 ina gi venel l i pseand sati sfya prescri bedl i near constrai nt.
Fr om出 c probl emspeci fi cati on, the el l i pse will provi de an i ncqual i ty constrai nt
whi l e山 e l i near rel ati on among thc vari abl cs will provi de加 cqual i ty constrai nt for
tbis cxampl e.





0.5 1. 5 2 2.5
Fl gure 4.1 Consl rai nl s of Exampl e 4.1.
Mat hemat l cal Model : Fi gure 4.1 capturcs the es総 帥 of thep帥 l cm. Code filcs
Ex4L1. m
and creat e_el l l pse. mare filcs n田 cssary for MATI . AB to create the
fi gure.τ 'herearet wo vari abl es X I and X2・前 l erearestandardmathemati cal expressi ons
for 山 eel l i pse, and thestrai ght l i neis no probl cmafter Chapter 3.
St andar d Format : The standard f ormat of the NL P is reproduced here for
Mi ni rni ze f(xtt X2' • • • , X
) (4. 1)
Subj ect ω hk(x. , h ・・・. ,x,,) = 0, k = 1,2ド . ,L (4.2)
め(XI ,X2・・. . , Xn) $; 0, j = 1, 2, . • . , m (4.3)
xl $; Xj $; xl' ; =1,2,... ,n (4.4)
Invector notati on
IFi I田 10bedownl oadedfromIhewebsilc町 cindicllledbyboldfnce悶 悶 悶 riflype.
臨 玄 オ
Mi ni rni ze f {X), 医 ln (4.5)
(4.6) Subj ect to: [h(X)], = 0
[g(X)]m$; 0 (4.7)
$; X $; Xup (4.8)
For the speci fi c probl embei ng di scussed, and ref eni ng to Fi gure 4.1, the desi gn
vari abl es are the cool' di nate val ues xl 叩 d X2 山 al al l owthc computati on of the
rect如忠l I ar area. The opti rni zati onprobl emis
Mi ni rni ze , -XI X2 (4.9)
(4.10) Subj ectto: hl (x. , xz}: 20xI + 15X2 - 30 = 0
81( X. , X2): (xT/ 4) + 一   (4.11)
O$; xl $; 3; O$; x2$; 3 (4.12)
The si deconstrai nts in(4.12) can al so be postul atedas one-si dedand can be wri tten

4. 1.2 Dl scussi on of Const rai nt s
Usi ng the rel ati ons (4.9) ー(4.12) several addi ti onal c1asscs of probl ems can be
descri bedby i ncl udi ngonl y a subset of thc re1ati ons. They areexami ned bel ow.
Unconst ral ned Probl em: Thcre are no functi onal constrai nts al though the si de
constrai nts arc necessary tokeep thesol uti onfinite. For this cxampl e,
Mi ni rni ze (4.9) j{X
, -x

In this probl em, if 出巴 desi gn vari abl cs are unbounded at 山 c upper l i mi t , then 血 e
sol uti onwoul d be at thel argest posi ti veval u巴 of XI and X2・A two-si dedlimit for the
dcsi gn vari abl es is usual l y a good i dea. The desi gner does have the回 目 ponsi bi l i ty of
defi ni ngan acceptabl edesi gnspace.
Equal i ty Const ral ned Pr obl em1: The functi onal constrai nts inthisprobl em紅 巳
onl y equal i ti es. Wi t h reference10Exampl e 4.1 出 c fol l owi ngprobl emcanbe scl up
(after changi ng曲 c i nequal i tytoan cqual i ty):
Mi ni mi ze ft.. x. , X2): -x. x2
Subj ect to: 11)(x.. X2): 20 x. + 15X2- 30 = 0
(x. ・X2):   + ( 4) ー 1=0
I ntui ti vel y, such a probl emmay not b巴 opti mi zed si nce the t wo constrai nts by
themsel ves shoul d establ i sh the val ues for the I wo desi gn vari abl es. The argumenl s
used inL P for accepl abl eprobl emdefi ni l i onareal soval i dh巴 悶 . There is al ways the
possi bi l i tyof mul ti pl esol uti ons- whi ch is a strongfeal ureof nonl i n巴 ar probl ems. In
such an event theset of vari abl es that yi el d出 el ow巴sl val uefor theobj ecti vewi l l be
theopti mal sol uti on. NOl e白 紙 sucha sol uti onis obtai nedby scanni ng theacceptabl e
sol uti ons rather 白 anthrough appl i cati onof any ri gorous condi ti ons.
Equal l ty Const ral ned Pr obl em2: I ftheprobl emwe偲 10incI ude onl y one of l he
constrai nts, for exampl e,
Mi ni mi ze ft.. xl ・X2): -XI X2
Subj ecl l o: h
(x"X2):   + ー
0::;; XI ::;; 3; 0::;; X2::;; 3
Thi s is a val i dopti mi zati onprobl em. A si mi l ar probl emcan be defi nedwi l hthefirst
constrai nt by itself.
I nequal l ty Const rai ned Probl em: Inthis casetheconstrai nts arc all i nequal i ti es.
A vari ati on on Exarnpl e 4.1 woul d be (the equal i ty cons町 aint is transf ormed to an
equal i ty constrai nt)
Mi ni mi ze ftx.. x2l: -XI Xz
Subj cct to: g. (x" X2): 20 XI + 15X2- 30::;; 0
g2(X"XZ): (xt/4) + ( 4) ー 1::;;0
O::;;XI ::;;3; 0::;;x2::;;3
Li ke its counte中 町 t in l i near programmi ng,出 is is a val i d opl i mi zati on probl em.
Equal l y val i d woul d be a probl emthat incI uded j ust one of l he constrai nts or any
number of i n叫 ual i ty constrai nts.
It is essenl i al to understand both the nature and the number of constrai nts as
wel l as how they affect the probl em. I n general , equal i ty constrai nts are easy 10
handl emathematI Cal l y,di ffi cul ttosati sfy numeri cal l y. andmorerestri cti veonthe
search for the sol uti on. I nequal i ty constrai nts are di ffi cul t to resol ve
mathemati cal l y and are mor e fl exi bl e wi th respect to the search for the opti mal
sol uti onas they defi ne a l arger feasi bl eregi on. A wel l -posed probl em that
someconstrai ntsareacti ve (equal i ty) at theopti mal sol uti onas otherwi se it woul d
be an unconstrai ned sol uti on.
Li ke L P, some mathemati cal defi ni ti ons are n田 凶 sary before the n町 民 sary and
suffi ci ent condi ti ons for theNL P can be establ i shed. Defi ni ti ons areneeded for both
白 e anal yti cal di scussi on as wel l as numeri cal techni ques. MATLAB provi des a
Symbol i c Mat h Tool box whi ch permi ts symbol i c computati on i ntegral ed in 出 c
numeri cal envi ronment of MATLAB. This al l ows I he user 10 expl o陀 probl ems in
cal cul us, l i near al gebra, sol uti ons of systemof equati ons, and other areas. In fact,
usi ngsymbol i c computati on students can easi l yrecover 山 epr邸 宅 qui si tei nformati on
for thecourse. A short hands- on exerci setosymbol i c computati on is provi ded.
1 τ
pri mary defi ni ti ons we need arederi vati ves, partial deri vati ves, matri ces, deri vati ves
of matri ces, and sol uti ons of nonl i near equati ons.
4.2.1 Symbol l c Comput at l on Usl ng MATLA目
One of thebesl ways to gel f ami1iar wi thsymbol i c computati on is totakethequi ck
onl i ne i ntroducti on 10 the Symbol i c Mat h Tool box avai l abl e in the MATLAB Demos
di al ogbox [4]. The fol l owi ngsequence l ocates thelutorial:
>> demos
Symbol i c Mat h - ・ > I nt r oduct i on
The computal i onal engi ne execul i ng 由巳 symbol i c operati ons in MAT1. AB is thc
kemel of Mapl e markel ed and suppo巾 d by Waterl oo Mapl e, Inc. lf 出 E民 ader is
al ready fami l i ar wi th Mapl e,出 en MAT1. AB provi des a hook through whi ch Mapl e
commands can be ex田 uted in MATLAB. The symbol i c computati on in MATLAB is
perf ormed usi nga symbol i c obj ect or sym. Thi s is another datatypeI i ke thenumber
and stri ngdatatypes used in阻 rlier exerci ses. The Symbol i c Mat h Tool box uses sym
obj ects to陀 present symbol i c vari abl es, expressi ons, and matri ces.
Intheexerci sethat fol l ows, a functi onof one vari abl e, and t wo functi ons of I wo
vari abl es (constrai nts f romExampl e 4.1) at1巴 used for iII ustral i on. Drawi ng on the
au出 or' s c1assroomex戸 巾 nce血 is prel i mi nary di scussi on is interms of vari abl es X
and y for i mproved comprehensi on. I nlater secti ons, subscri pts on x areused todefi ne
mul ti pl evari abl es so that 出 etransi ti ontothegeneral probl emcan be facilitatedusi ng
vector descri pti on. The functi ons inthcsc exeai ses are
f t x) =12+( x- I ) 2( x- 2) ( x- 3)
g. (x,y): 20x+ 15y - 30
g2(X,y): x2/4+ l - 1
ITOOrclcasc of 100 newSwdent Edi l l ol l includcs Ihc symbolic loolbOl¥ ulongwilhIhe basic MATLAD
sortw町 t

, Y


. f


者 9 col umn vect or based on gl ,g2 9 '" [ gl ; g2)
[ 20*x+15*y- 30]
1/ 4*x+y- 1]
も 9 can be t he const r ai nt vect or i n opt i mi zat i on
者 pr obl ems
も t he par t i al der i vat i ves of 9 wi t h r espect t o desi gn
苦 var i abl es i s cal l ed t he J acobi an mat r i x
も t he pr oper t i es of t hi s mat r i x ar e i mpor t ant f or
者 numer i cal t echni ques
xy .. [x y] ; も r owvect or of var i abl es
J '" j acobi an( g,xy) も cal cul at i ng t he J acobi an
者 a pl ot of f f or - 2 pi
df 司 di f f ( f ) ;
h01d 00
ezp10t ( df , [0,41) 告 pl ot t i ng f unct i on and der i vat i ve
者 combi ne wi t h MATL AB gr aphi cs - dr aw a l i ne
1i oe ( [ 0 4] , [0 0) ,' C010r ' , , r' )
[ 20*x+15*y・ 30]
1/ 4*x+y- 1]
( def aul t ) 2 pi
<= 4
[ 1/ 4,
ezp10t ( f ) x
者 pl ot bet ween 0 <= x ezp10t ( f , [0,4) )
"J' ljl
百 le fol l owi ngMATL AB sessi on was capωred as a di ary file and edi ted in a text
editor. The児 fo陀 山 e MATL AB prompt does not appe紅 . 百 le bol dface words are
commands 出 at 山 ereader will typeat thecommand line.
者 def i ni ng x as a si ngl e symbol i c obj ect x ", sym( ' x' )
者 def i ni t i on of mul t i pl e obj ect s
var i abl es i n t he wor kspace
Byt es Cl ass
126 symobj ect
126 symobj ect
128 symobj ect
128 symobj e' ct
126 symobj ect
126 s戸 n obj ect
g1 g2
t ypes
Si ze
syms y f
whos %
Gr and t ot al i s 14 el ement s usi ng 760 byt es
者 const r uct i ng f (x・1) *( x- 1) *( x・2) *( x・3) f.. 12 +
12+( x- 1) " 2*( x・2) *( x- 3)
di f f ( ま}
2*( x- 1) *( x- 2) *( x- 3) +( x- 1) ^2*( x- 3) +( x- 1) "2*( x- 2)
も f i r st der i vat i ve
Y 2. 5
Addi ti onal symbo1 i c computati ons will be i ntroduced throughcode as appropri ate.
Note that 白 e result of both numeri c and symbol i c computati ons can be easily
combi ned a10ng wi thgraphi cs toprovi dea powerful computi ng envi ronment .
も t o eval uat e 9 at x 1,
subs( g,{x,y},{1,2. 5})
27. 5000
1. 7500

者 Z O   ~ ,

も not e t he chai n r ul e f or der i vat i ves
% not e t he i ndependent var i abl e i s assumed t o be x
di f f ( ま, x,2)
2*( x- 2) *( x・ 3) +4*( x- 1) *( x- 3) +4*( x- 1) *( x・ 2) +2*( x- 1) "' 2
者 t he second der i vat i ve wr t x
主 t he t hi r d der i vat i ve wr t x }










告 def i ne 9工 n















も def i ne g2
gl ,g2 can onl y have par t i al der i vat i ves
i ndependent var i abl es have t o be i dent i f i ed







も par t i al der i vat i ve
g2 '" 0. 2S*x + Y ・ 1;

4. 2. 2 Basi c Mathemati cal Concept s
The basi c mathemati cal e1ements in the di scussi on of NLP are deri vati ves, parti 叫
deri vati ves, vectors, matri ces, J acobi an, and Hessi an. We havc used the Symbol i c
Mat h Tool box in theprevi ous sccti ontocal cul atesome of thesequanti ti es wi thout
dcfi ni ng them. These topi cs will have been extensi vel y covered in the foundati on
courses on mathemati cs inmost disciplines. A brief revi ewis offeredin血 is secti on
usi ngMATLAB. 百 lis opportuni ty will alsobe utilizedtoi ncreasefamiliarity wi 出 the
Symbol i c Math Tool box and i ncorporat巴 it wi th出 eMATLAB commands that were
usedin山巴 earlier chapter冨.
Funct l on o' One Vari abl e: ft.. x) identifies a functi onof one variable:
f (x) = 12+ ( x- l i ( x- 2) ( x- 3)
is usedas a specific exampl e of sucha functi on. The deri val i veof 出 efuncti onat 山 e
l ocati onx is wri ttenas
玉江 - 1・ kx + ax) -.f(x) γ 笠
dx- i l m Ax =l l fI 1-L
ax→o ax ax->oax
x is the poi nt about whi ch 由巴 deri vati ve is compuωd. ム r is thc di stamニe tu a
nei ghbori ngpoi nt whos巴 l ocati onthereforewill be x + ax. The val ueof thederi vati ve
is obtai nedas a limit of thc ratioof 白 edi fferen田 intheval ueof thefuncti onat 曲 e
two poi nts ( 幼 ω 出 edis岡 田 separati ngthe two poi nts ( 位) ,ぉ this sep叩 tion is
reducedωzero. The computati onof thcderi vati vefor thespecificex創 npl eis usual l y
obtai n氾 dusi ngtheproduct rule. Resul ts fromtheexe民 iseon symbol i c computati on
provi dethc resuJt
ま = 均 一 例 一 例 ー 3) + (x- 1) ゆ い ) + ト 1) ^川 ー 2)
Tabl e 4.1 illustrates 出 elimitingprocess for thc deri vati veof thefuncti onchosen as
白 ecxampJ e. The deri vati ve is bei ngeval uated at poi nt 3. As thedi spJ acements are
reduced,出巴 val ueof theratioapproachcs 出巴 val ueof 4 whi ch is theaClual val ueof
thederivative. Whenax is l arge(wi tha val ueof 1) ,出 cratiois 18. Thi s issignificantly
different f romtheval ueof 4. Wi th ax of 0.1 theval ueis around4. 851. Wi th fur也 er
reducti ontoO. ∞1 theratiohas a val ueof 4. 008. Fromthesecomputal i ons it Canbc
expected白 紙 as ax approaches 0 theratiowill reachtheexact val ueof 4.0.
Numeri cal Deri vati ve Compu削 'ion: Many numcri cal techni qucs in NLP (1叫 ui 陀
出 ecomputati onof derivatives. Most softwarei mpl ementi ngthesetechni ques do not
usesymbol i c computati on. Automati on and回 seof use毘 qui re血 at 也 esederi vati ves
becomputed numeri cal l y. 百 leresults inTabl e 4.1justify出 enumeri cal computati on
of a deri vati vethrougha techni quecal l edthefi rsl forwardd, 砂量 rence[5]. Usi ng a very
smal l pe目 urbati onax, lh号 deri vati veat a poi nl is numeri cal l y cal cul atedas theratio
Tabl e4.1 Cal cul ati onor Deri vaUve
x dx x+dx M
dfl dx Derivative
3 4 18 18 4
3 0.1 3.1 0.4851 4.851 4
3 0.01 3.01 0. 0408 4. 0805 4
3 0.001 3.001 O. ∞4 4.008 4
of thechange inthefuncti onval ue, 4f, 10出 echange in恥 di spl acemenl ,ax. For 出 E
exampJ e, thederi val i veat x = 3 wi thax = 0. 001 is obtai nedas
El I _f.: (3+ 0. 001) ー1(32- .11 nnSl
dxl .rc3 0. 001 ー
The deri vati vc for lhesi ngJe-vari abl efuncti onal any val uc x is aJsocaJl edlhesl ope
or thegradi etl t of lhc functi on創 出 al poi nt . If a lineis drawn tangenl 10 thefuncti on
al theval uex, thetangent of 白 E叩 glethat 出 is Jinemakes wi 出 thex-axi s will have
thes創 neval uc as thederivative. If 血 is angl eis 9,出 en
ま =tan9
Fi gure4. 2illustrates 出 etangency propertyof thederi vati ve.
MATLAB Code: Fi gure4. 2was creal edby thecode tangent. m. Inthesame di rectory,
出 e code derl vati ve. mprovi des the illustration of tangency through a fi 伊 re
ani mati oninMATLAB. In出 efigurewi ndowboththefuncti on(drawnas thecurve) as
wel l asthelinerepresenti ng4f and ax aredrawn. As thenei ghbori ngpoi nts arccl oser,
it is evi de似 合 omthe figure that thecurve can be approxi matedby a straight line.
When ax = 0. 001,出 ederi vati veis tangent tothecurveas 出 eyarei ndi sti ngui shabl e.
H匂her Deri vati ves: The deri vati ve of the dcri vati ve is the called 出 e second
deri val i ve. Formal l y it is dcfi ned邸
互主-!!..(笠 -l i " ,坐笠生 L!;rn 炉 心 1. . 出 - 4yal x
tJi l - dxldx) 一心--10 ax 似:ゐ ax (4.16)
Si mi l arl ythel hi rdderi val i veis defi nedas
201- ・"
o 0.5
tangent - slope- derlvativeat x=3
lo12+(x・t)・(x-l)・(x- 21・(x-3)
1.5 2
2. 5
Fl gure4.2 lII uslrallon01thel angent .
3 3.5 4
Thi s can go on provi ded出 efuncti onhas suffi ci ent hi gher-order dependence on出 E
i ndependent vari abl ex.
Functi on of Two VBri Bbl es: The l wo-vari abl efuncti on
j{x川 :x214+y2- 1 (4.18)
is chosen toiIl ustratetherequi redmathcmati cal concepts. The first i mportant fcature
of t wo or mor e vari abl es is 白 紙 thederi vati ves defi nedfor a si ngl evari abl e(ordi nary
deri vati ves) do notappl y. 百】 eequi val ent concept hereis theparti al derivative. Partial
deri vati ves aredcf med f orcach i ndependcnt vari abl e. 百 lepartial deri vati veisdenoted
by the symbol d. These dcri vati ves are obtai ned in the same way as the ordi nary
deri vati veexcept thc other vari abl es arehel dat a constant val ue. I nthecx創 npl e,when
compul i ng白 epぽ tial deri vati veof x thc val ue of y is not al l owed tochange.
dfl   + ax, y) - f(x, y) (4.19)
dxl ^二ム /:u
1(.<. ) 1 ) 山 山
Thc above rel ati onexpresses thepartial deri vati veof thef uncti onf wi th respect to出 E
vari abl ex. Graphi cal l y this suggcsts that thc t wo poi nts are di spl aced hori zontal l y
(assumi ng thex-axi s is bori zonta1). Intheabove expressi on(andpri or ones too) , the
subscri pt after theverti cal l i neestabl i sbes tbepoi nt a加 utwhi ch出 epartial deri vati vewi th
郎 pect tox is bei ngeval uatcd. (x,y) 即 応 se附 any/al l points. A si mi l ar expressi onc四 be
wri ttenfor thepar首al deri vati veof f wi th陀 spect toy. For this exampl e:
For thepoi ntx = 2, y = 1:
ー 司
- 一一 : "=2y
dx - 2' dy
df • df
f (2,I) = 1; 一 =1・ ー =2
dx- " dy
Unti l this poi nt wc have made use of symbol s reprcsenti ng changes in val ues
(functi ons, vari abl es) wi thout a forrnal defi ni ti on. In山 is book
s( ) : 犯 presents fi ni tel si gni fi cant changes inthequanti ty ( )
d( ) , o( ) : represents di ぽercnti al / i nfi ni tesi mal changes in( )
Changes in functi ons occur due to changes in the vari abl es. Fr omcal cul us [1] thc
di 仇 rential change inj{x,y) (ゆ due to thedi fferenti al change in血 vari abl es x (dx)
and y (dy) is expressedas
L" dj ,. df
df =-:J dx + -:J dv
"J dx _. . dy- J
(4. 20)
For convcni ence and si mpl i ci ty, the subscri pt representi ng 白 e poi nt whcre the
exprcssi on is eval uated is not i ndi cated. The defi ni ti on of the partial deri vati ve is
apparent intheabove expressi onas hol di ngy at a constant val uei mpl i es 出 at の =0.
Another i nterpretati on for the partial deri vati ve can be observed f romthc above
defi ni ti on: cl tangeof tltefuncti onper unit ci tangeintltevariables.
Gradi ent 01theFuncti on: Inthe functi on of a si ngl e vari abl e, the deri val i vc was
associ ated wi 出 the sl op古. In t wo or mor e vari abl es the sl ope is equi val ent 10 山 e
gradi ent . The gradi ent is a vector, and at any poi nt rep児 sents thedirectioninwhi ch
thefuncti onwill i ncreasemost rapi d, か Exarni ni ngtheconventi onal obj ecti veof NL P,
mi ni mi zati on of obj ecti veβmcti ons, the gradi ent has a natural part to pl ay in the
devel opment of methods tosol vetheprobl em. The gradi ent is composed of thepartial
dcri vati ves organi zedas a vccl or. Vectors in白 is book arecol umn vectors unl ess othcr
noted. The gradi ent has a standardmathemati cal symbol . lt is dcfi nedas
Vf 日=阿
At this stage it is appropri ate to consol i date 白 is i nformati on usi ng graphi cs. The
graphi caI d巴scri pti on of 白 e ex創 npl e defi ned in Equati on (4.18) has to be three
di mensi onal as we need one axi s for x, ano出 er for y, and thethirdforj{x,y). Chapter
2 i ntroducedthree-di mensi onal pl otti ngusi ngMATLAB. A mor e useful descri pti onof
theprobl 巴 mis touse contour plots. Contour pl ots aredrawn for speci fi c val ues of the
functi on. IntheiII ustrati ontheval ues for thecontours of f are 0, 1,2,叩 d3. I nFi gure
4.3, t wo ki nds of contour pl ots are shown. In the tophal f a three-di mensi onal (3D)
contour pl ot is shown, whi l e on白 l owerhal f 恥 samei nformati onis presen凶 ぉ a
two-di mensi onal (2D) contour pl ot (this is consi de陀 dth巴 standardcontour plot). Thc
2D contour pl ot is mor e useful for graphi caI support of some of thei deas inNLP. The
3D contour wi l l be di spens巴 dwi thin山 eremai nder of thc book.
MATLAB Code: Fl g4_3. m. The annotati oninFi gure 4.3(i ncl udi ngthetangent linc
and thegradi ent 町 ow) wasdone出 roughthepl ot 吋 itingcommands avai l abl eon山 e
menu bsr in the fi gure wi ndow( MATLADversi on 5.2 and later). A m吋 01" pOl"ti onuf
白 cpl ots a問 generatedthrough thestatcments inthem- fiI e i ndi catedabove. The codc
mi xes numeri c and symbol i c computati on. It al l ows mul ti pl e pl ots and targets
graphi c commands tospeci fi c pl ots.
Di scussi on o( Fi gure 4. 3: In Fi gure 4.3 poi nt P is on contour f = O. Poi nt Q is on
thecont ourf =2. Poi nt S is on thecontourf =1. Poi nt R has 出 esame y val uc as poi nt
P 佃 d白 esame x vaJ uc as poi nt Q. For 出 eval ues inthefi gurethecontour val ue is 0. 75
(this val ue shoul d be di spl ayedinMATLAB wi ndowwhcn thecod巴 Fl g4_3. mis run).
lineP Q is a measure 4f
linePR represents changes indf whcn d. y is 0
l i neRQ問 presents changes indf when d.xis 0
Mathemati cal l y, ム f can onl y be esti matedby addi ng thechangcs al ongthe lines PR
andRQsi nce出 cdefi ni ti onof thepartial deri vati ves onl y pcrmi ts cal cul ati ngchangcs
al ong thecoordi nate di recti ons.
At poi nt S thedottedline is thetangent. The gradi ent at thc samc poi nt is normal
(pcrpcndi cul ar) to 山 c tI.ngent 岨 d is dir田 ted toward i ncreasi ng the vaJ ue of 白 e
functi on(i ndi cated by 白 earrow). By defi ni ti on, if dfrepresents a di fferenti aJ move
aJ ong thegradi ent at any poi nt,出 en(dx,めP副 官 measuredal ongthegradi ent vector)
df =空白+さ dy
3Dconl our for f (川 1=x2/ 4+I - l
。 。
2Dconl our
gllldlenl 01S
- ・ , longenl 815
2 3
Fl gur, 由 4.3 Gradi enl andl angen! IIn自 a! a point .
Fr om恥 fi guretheval ue of f wiIl change al ong gradi ent di recti on. If dfrepresents a
di fferenti aJ move aJ ong恥 畑 gent line,白 en(dX,の aremeasured aJ ong刷 gent line)
df =   主 の
批 e J y
Equati on (4.23) shoul d bc zero because movi ng tangcnti al l y (a smal l amount) the
vaJ u巴 offi s not changed.
The J acobi an[J] defi nes a useful way toorgani ze thegradi ents of sevcraJ
functi ons. Usi ng threc vari abl es and t wo func!i onsj{x,y,z) and g( x,y,z) thedefi ni ti on
of thc J acobi anis
I eJf 互 主 │
[JJ =1 1
星 笠 笠
InEquati on(4.24) thegradi ents of the functi onapp町 in出 esame row. The first r ow
is 出 egradi ent of f whi l e thesecond r owis thegradi ent of g. lf thet wo functi ons are
coUected i ntoa col umn vector. thedi fferenti al changes [df dg]Tin山 efuncti ons, due
to 出 e di fferenti al change in thc vari abl es [dx dy dz] , can be expressedas a matri x
mul ti pucati onusi ngtheJ acobi an



4 [






whi ch is si mi l ar toEquati on(4.20).
Hessi an: The Hessi an matri x [H] is thesame as thematri x of second deri vati ves of
a functi onof several vari abl es. For ft... x,y)
I dγaγ│
[問 =1ぜ ぬ │
I iJ L[ iJ L[ I
I dydx ザ │
The Hessi 叩 matri x is s ymm凶 ic. For 出巴 exampl edefi nedby Equ瓜 i on(4.18).



Funct l on of n Varl abl es: For ft... X). where匹 ] =[XI ・X2. . . . • x,.rr. 白 egradi ent is
The Hessi an is
笠 d'i ••• d立一
à ~ dx
iJ x
iJ x1ax
~ a'i
問 =1dX2ax1 à ~
a γ
iJ xndx, d..c
Equati ons (4.27) and (4.28) will app回 r qui te often in succeedi ng chapters. A f ew
mi nutes of fami l i ari zati onwill provi de sustai nedcompreh巴nsi onlater.
4. 2. 3 Tayl or' s Theor ern/ Serl es
Sl ngl e Varl abl e: The Tayl or seri es is a useful mechani smtoapproxi mate出 eval ue
of thefuncti onft... x) at 山 poi nt (x
+お ) if 白 efuncti onis compl etel y known at poi nt
• 百 l e exp叩 si onis (for finitc n)
手   ~ ( 制 2+ ト古家│う州
的 + 釘 ) 叫 , ) + ま I( お ) サ
The seri es is wi del y used inmost di sci pl i nes toestabl i shconti nuous model s. It is 出 E
mai nstay of many numeri cal techni ques, incI udi ng those in opti mi zati on. Equati on
(4.29) is usual l y truncatedωt he first t wo or 山 町 terms wi 白 血 eunderstandi ng批
approxi mati on wi l l suffer some error whose order depends on恥 term伽 t is bei ng
+ 玉江 I { AY ¥ .1.1. fll
[ ( Xp+At) 三 f ( X J &│ J h ) +2 1 d l J 釘 ) 2+O(位 )3
lf thefirst termis brought to山 eleft,出 eequati on. di scardi ngtheerror term, canbe
wri ttenas
M何 =イ 吠 f六(うx
+釘削) 一f川(令句x巧ろ引 :X/,) = 戸,,)片)片= 判 (仰刷釘削)+上 立│ 刷
dxL目 21 dX- 1
- A b - x h
InEquati on (4.31) , thefirst termon白 eri ght is cal l edthefi rst-order/li near vari ati on
whi l e thesecond termis 出 esecond-order/quadrati c vari ati on.
Fi gure 4.4demonstrates theapproxi mati ons usi ngTayl or' s seri es of vari ous orders
at poi nt 2.5wi threspect to白巴 ori gi nal functi on(red). The pri nci pal i deais todeal wi 出
theapproxi mati ng curve whi ch has known prope凶 es i nsteadof theori gi nal curve.
百 lC constant Tayl or seri es is woeful l y i nadequate. 百 le l i near expansi on is onl y
margi nal l y better. The quadrati c expansi on approxi mates theright si deof thefuncti on
comp釘 吋 to theleft si de whi ch is insi gni fi cant eπor. 百 le fifth・order expansi on is
defi ni tel y acceptabl e in the range shown. To see how the Tayl or seri es is used,
consi der thequadrati c curve about 出 c poi nt x = 2.5. At this poi nt 出巴 val ue of the
functi onft... 2.5) = 11.4375, theval ueof th巴 first deri vati vef' (2.5) = -0. 75,皿 dthcval uc
of thesecond dcri vati veisf"(2.5) = 4. Usi ng Equati on (4.31)
ft... 2.5+制 =1間 5+(- 0 万 ) 釘 +主 榊
For di fferent val ues of At. bo山 posi ti vcand ncgati ve, theval ue ofj(x) canbe obtai ned.
百 leplot of Equati on(4.32) shoul dbe thesame as 恥 onel abel edquadrati c inFi gure4.4.
f ( xp+dx,ゐ+勾 ) =f ( xp' ゐ ) +町 川 引 +

蜘 ,
For nv副 abl es,wi thろ thecurrent poi nt and d.Xthedi spl acement vector,
f ぽ p + Il X) = f ( X
) +町 鳥 川 +j mTm
百 legraphi ca1sol uti ons arepresentedfor threeki nds of probl ems: uncons位 ai ned , equality
constrai ned, andinequalitycons凶 i ned,basedon thefuncti ons inExampl e 4.1.
o 3; 0 3 (4.37)
The sid巴 constrai nts s巴rve to l i mi t the desi gn space. Consi deri ng the obj ecti ve
functi on in (4.36) , it is cl ear that the mi ni mumval ue of f wi1l be r巴al i zed if the
vari abl es are at the maxi mum(Xi = 3,巧=3). The val ue of the functi on is -9.
Fi gure 4.5 il1ustrates theprobl em. The sol uti on is at the boundary of the desi gn
space. Fi g4_5. mis theMATLAB code山 at wi l l produce Fi gure 4.5. The maxi mum
val ueof thefuncti onis 0 (basedon thesi deconstrai nts). InFi gure 4.5thetangent
and the gradi ent to the obj ecti ve functi on at the sol uti on are drawn in usi ng
MATLAB pl otedi t commands. l D也 is parti cul ar exampl e, the si de constrai nts are
necessary todetermi ne thesol uti on. If thedesi gnspace were i ncreasedthesol uti on
woul d correspondi ngl y change.
The巴xampl echosentoillustrate曲 eunconstrai nedprobl emdefi nedby (4.36) and
(4.37) is not usual l y empl oyed to devel op the opti mal i ty condi ti ons for general
unconstrai ned probl ems. Two requi rements are expected to be satistied by such
probl ems: (1) thesol uti onmust be intheinterior of thedesi gnspaceand(2) theremust
b巴 a uni que sol uti on, or 白 e probl emis uni modal . Neverthel ess, in practical
appl i cati ons thesecondi ti ons may not exi st. Most softwarearedevel opedtoappl y the
opti mal i ty condi ti ons and they rarely verify 出 eserequi rements are bei ngmet. It is
usua1l y thedesi gner' s responsi bi l i tytoensuretheserequi rements aremet. To devel op
也 eopti mal i tycondi ti ons anaI temate unconstrai nedprobl emis presented:
(4.36) JtXI ,x2): - XI X2 Mi ni mi ze
(4.38) JtXI ,x2): ( XI - 1)2+ ( X2 - 1)2- XI X2 Mi ni mi ze
,1t 1
キ I
fifth-order orlg!nal
Approxlmat!onus!rtgTaylor serles

Unconst ral ned Pr obl em

MATLAB Code: Fi gure4.4iscompl etel y createdusi ngthecode Fl g4_ 4. m. 1t uses the
MATLAB-provi ded symbol i c “tayl oc" functi on to generate the vari ous expansi ons
about poi nt 2.5.
3.5 3 2.5
Taylor series approximalion.
Fi gure4.4
0.5 D
Two or Mor e Vari abl es: The series 蹴 onl yexpanded to臨 qu拙 'aticterms. The
truncati onerror is i gnored.τ 'he two-vari abl efuncti onexpansi onis shown indetai1and
alsoorgani zedinterms of vectors and matri ces. The first -order expansi onisexpressed
interms of thegradi ent . The second-order expansi onwill be expressedinterms of tbe
Hessi anma甘 ix
f(x'J +似 Y/J + Il y) = Jtx'J' y,) + 1 さ │ 釘 +
Q, 冗 1 (.I"pろ}
手Ittp,Y/ 叶
e)2f I
耳石 ( w- Y+
1 r笠 │ …2・ q
21   , 川 、 ー 〓 ・ー (4.33)
I f 批 di spl acements are organi zed as a col umn vector [ 似 勾 ]T,the expansi on in
(4.33) canbe expressedina condensed manner as
, .N 2

0.5 1.5 2
Fl gure4.5 Unconsl rai nedsolution.
o $XI $ 3; O$ X2 $ 3
3 3.5 4
Fi gure 4. 6di spl ays theprobl emand its sol uti on. Fl g4_・. 6. mprovi des thecode.that wiII
general e most of Fi gure 4.6. It is cJ ear 山 at the sol uti on appears to be ot
xi = 2 and xi = 2. The opti mal val ue of 山 e obj ecti ve functi on is -2. Inthecontour
pl ot,出 eopt i mumis a poi nt maki ng it difficuJ t todrawthegradi ent. I f 白 is is thecase,
也 eninthreedi mcnsi ons, at theopt i mum,出 cgradi ent wil¥Ii e ina pl ane tangent tothe
functi on surface. Movi ng in出 is pl ane shoul dnot change the val ueof theobj ecti vc
functi on. This observati onis uscd todevel op then田 essarycondi ti ons later.
4. 3. 2 Equal l ty Const r ai ned Pr obl em
For a two-vari abl c probl emwe can onl y utilize one constrai nt for a meani ngf ul
opti mi zati onprobl em:
Mi nI mi ze j{XI ,x2): ー'XI X2
Subj ect ω hl(XI ,x2): X!/ 4 + - 1
円 gure4. 6 Unconstrai nedprobl em: interior 回 1lI llon.
$3; OSx
S 3
Fi 思u" e 4.7 (由 rough FI g4ー7. m) i1I ustrates the probl em. The dashed l i ne is 白 e
∞ns住 ai nt . Si nce it is an equaJ ityconstrai nt, thesol uti onmust bea poi nt on thedashed
line. The contour of f =ー 1appears toj ust gr回 巴 出 e constrai nt and therefore is the
mi ni mumpossi bl evaJ ue of thefuncti onwi thout vi ol ati ngtheconstrai nt . InFi gure4.7,
血 egradi ent of theobj ecti vefuncti onas weIl as 曲 egradi ent of 白 econstrai nts at the
sol uti onar官i l I ustrated. It appears 白 at at 血 is poi nt thesegradi enl s areparal l el cvcn
though 出 ey aredi rectedopposi te to cach other. By defi ni ti on thegradi ent is inthe
di recti onof thc most rapi di ncreaseof thefuncti onat thesel ectedpoi nt . Thi s is not a
coi nci denc巴. This fact is used toestabl i shthenecessary condi ti ons for theprobl em.
4. 3. 3 I nequal l ty Const悶 I ned Pr obl em
The number of in巴qual i tyconstrai nts is not dependent on thenumber of vari abl es. For
i1I us岡 山 n,bo白 cons佐直 int functi ons of Exampl e 4.1 ar芭 formul ated as i nequal i ty
cons町 amts.
Mi ni mi ze j{x) , X2): - XI X2 (4.36)
対 刷 2
1. 5
0.5 1.5 2
Fl gure4. 7 Equality∞nstralnedprobl em.
Subj ectto: gl (XI ,x2): 20xI + 15x2- 30S0
g2( Xl>X2): xI / 4 +えー I SO
3 3.5 4
Fi gure4. 8 ilI ustrates thegraphi cal sol uti on. Thesol uti on i sat thei ntersecti onofthe
dotted lines. The code is a vai l abl einFl g4_8. m. 百 legradi ents aredrawn usi ngthe
pl otedi t functi ons on themenubar inthewi ndow. Al so font styl eand font sizehave
becn adj ustedusi ngthepl otedi t commands. The opti mal sol uti onmust lieon or to山 E
left of the dashed line. Si mi l arl y it must I i e bel owor on the dashed curve.
Si mul taneousl y it shoul d a1so decrease the obj ecti ve functi on val ue as much as
possi bl e.
4. 3. 4 Equal i ty and l nequal l ty Const ral nt s
Exampl e 4.1is devel oped wi thbothtypes on constrai nts present . Il is問 producedhere:
1.5 2
Fl gure4. 8 Inequalityconstral nedprobl em.
Mi ni rni ze j{X¥,x2): -xl x2
Subj ect 10: h
(XI ,x2): 20 XI + 15X2 - 30 = 0
gl (XI ,x沙: (xI / 4) + ~   一 I S 0
o S XI S 3; 0 S X2 S 3
The graphi cal sol uti onof Secti on4.3.3suggests that thei nequal i tyconstrai nt will be
acti veat thesol uti on. I f 白紙 is thecase, thenFi gure 4. 8wi l l once agai nrepresent 出 e
graphi cal sol uti ontothis secti on凶 so. This wi l l be vcri fi edinthcncxt scclI on.
Analytical condi ti ons rcfer to恥 n回 essaryand suffi ci ent condi ti ons 伽 twill permi t
the recogni ti on of the sol uti on to the opti mal desi gn probl cm. Thc condi ti ons
devel oped hcre empower the numeri cal techni ques to fol l ow later. They are
i ntroduced in 由巳 same sequence as in 山 e previ ous secti on. l nstead of fonnal
mathemati cal details, the condi ti ons are establ i shed less fonnal l y f romthe
geometri cal descri pti on of 由 。 probl emandl or 出 rough i nωi ti ve reasoni ng. Fonnal
dcvel opment of thc anal yti cal condi ti ons can be f ound in Refercnces 6-8. For
establ i shi ng the condi ti ons it is expected that the sol uti on is in the i nteri or of 山 e
feasi bl 巴 regi on,and thereis onl y one mi ni mum.
4. 4. 1 Unconst r ai ned Pr obl em
The probl emused for i l l ustrati onis
Mi ni mi ze J(x.. X2): (XI - 1)2 + (X2- 1)2- XI X2 (4.38)
(4.37) OSx
S3; OSX2S 3
Fi gurc 4. 6 provi ded a contour pl ot of the probl em. Fi gure 4.9 provi des a
three-di mensi onal pl ot of thesame probl em. It is a mesh pl ot. The commands tocreate
Fi gure 4. 9areavai l abl einFl g4_9. m. The fi gureneeds tobe rotatedi ntemcti vcl y to
appear as illus回 ted. A tangent pl aneis dmwn 鉱 山 巴 mi ni mumfor emphasi s. Fi gure
4. 9wil1be us巴 dtoi denti fy theprope凶 es of the functi onJ(xl ,xz) at themi ni mum.
The mi ni mumis i denti ti ed by a supe凶 cript asteri sk ( X・or lxi , X2J
). Studyi ng
Fi gure 4. 9, the functi onhas a mi ni mumval ueof - 2, whi l e xj = 2 and x2= 2. If 白 e
val ues of X1 andl or X2 we民 tochange even by a sl i ght amount f rom自己 opti mal val ue,
inany di recti on, theval ueof the functi onwil1certai nl yi ncreasesi nceX' is thel owest
poi nt of the concave surface rep閃 senti ng 出 e functi on f Reprcsenti ng 出 E
di spl acemcnt f romtheopt i mumval ues of thevari abl es as dX, and thc changc inthe
functi on val uc f romthe opt i mumas llf, f romdi rect observati on it is cl ear that the
opti mal sol uti onmust be a poi nt 由 at satisfies
4f>0, for al1dX
Fl rst- Order Condl tl ons: The same i dca can bc appl i ed in the limit, that is, for
i nfi ni tesi mal di spl acement dx
and dx
about X・ . The functi on itself can be
approxi mated by a pl anetangent tothefuncti onat thesol uti on( shown inFi 酔 re4. 9)
(consi dcr 血 is ぉ afi rst-ord巴r Tayl or seri es expansi on). Movi ng to any poi nt in the
pl ane f romtheopt i mum(see Fi gure 4.9) will not change theval uc of the functi on,
白 E閃 foredf = O. Movi ng away f romopt i mumi mpl i es 白 紙 dx
and dx
are not zero.
Rewri ti ng Equati on(4.20) intcnns of X1 and X2
df =か 1+か =0
df =医者 ][:J=O



- 2
F1gure4.9 Three-di mensi onal plot 01Fi gure4.6.
Si nce this shoul dhol dfor al1poi nts in白 epl ane, dx
1:- 0 and dx
1:- O. Therefore,
df - f l 主 - f l
- v, dX2 - v
or thegradi ent of f at theopt i mummust be zero. 百 at is,
V f   x ~ , x ; ) = 0
Equati on (4.43) expresses the necessary condi ti on, or first・order condi ti ons (FOC) ,
for unconstrai nedopti mi zati on. It is termed thefrrst・order condi ti onbecause Equati on
(4.43) uscs 出 egradi ent or thefirst deri vati ve. Equati on (4.43) is usedto i denti fy thc
possi bl esol uti ons totheopti mi zati onprobl em. I f 出巴 functi onwere toflipover so出 at
the same functi on wc陀 to maxi mi ze the val ue of 出 e functi on, the sol uti on for the
vari abl 巴s wi l l be at the same val ueof the desi gn vari abl c. It is cl ear that Equati on
(4.43) appl i esωt he maxi mumprobl emalso. Equati on (4.43) by itself will not
determi ne themi ni mumval ue of thefuncti on. Addi ti onal consi derati ons a偲 necessary
toensure山 at thesol uti onestabl i shedby 出巴 fi rst-order condi ti ons is opti mal , in出 is
casc a mi ni mum. For a general unconstrai nedprobl cm, the necessary condi ti ons of
Equati on(4.43) can be statedas
Vf (X'・) =0
whi ch is pri mari l y a vector expressi onof the rel ati on in Equati on (4.43). Equati on
(4.44) is usedtoestabl i shtheval ueof thedesi gnvari abl es X' bothanal yti cal l y and
numeri cal l y.
Second- Dr der Condi ti ons: The second-order condi ti ons ( SOC) are usual1y
regardedas suffi ci ent condi ti ons. It can be i nfe町 'ed出 at thesecondi ti ons wi l l i nvol ve
second deri vati ves of th巴 functi on. The SOC is often obtai ned through theTayl or
expansi on of thefuncti ontosccond order. If X申 is 出巴 sol uti on,and d.Xrepresents 白 e
changeof 出巴 vari abl cs f rom出巴 opti mal val uewhi ch will yi el da change d i f , then
年 刊 念 +6X) ー が ) =可 ( X市 川 + t a h r凶
Thi s is sirnilar to Equati on (4.35) except 出 eexpansi on is about 出 c sol uti on. 6f
must be grcater thanzero. Empl oyi ng thenecessary condi ti ons (4.44) , thefirst term
on theri ght-handsi deof Equati on 4.45 is zero. Thi s I cavcs 出 efol l owi ngi nequal i ty
a f =j a T H
wher eH( X・) is theHessi an matri x (thema紅 ixof second deri vati ves) of thefuncti on
f at thc possi bl 巴 opt i mumval ueXへ For 出 erel ati ons inEquati on (4.46) to hol d, the
matri x H( X勺 must be posi ti ve definite. There are three ways to 巴 stabl i sh the H is
posi ti vedefinite.
(i) For all possi bl e.6X, d.X
H( X' ). 6X> O.
(ii) The ei genval ues of H( X' ) areall posi ti vc.
(i u) The determi nants of 叫 ] l ower orders (submatri ces) of H(X' ) that i ncl udeth巴
mai n di agonal areaIl posi ti ve.
Of t he出 ree,onl y (ii) and (iii) can be practi cal l yappl i ed, whi ch is illustratedbel ow.
Exampl e
Mi ni rni ze j(XI,x2): (XI - 1)2+ (X2- 1)2- XI X2 (4.38)
ヂ=2( x
- 2) - x
ま=2仇 - 1) - x
EQuati ons (4.47a) and (4.47b) represent a pai r ofl i nearequati ons whi ch can be sol ved
as- X1=2,I 2=2. The val ue of the t-uncti onf i s- 2. So far onl y the necessary
condi ti ons have been satisfied. The val ues above can al so refer to a poi nt where the
釦 ncti onis a maxi mum, or where thereis a saddl epoi nt .
SDC: For 出 is probl em[seeEquati on (4.26)]



Is it posi ti vedefi ni te?
(i) Not possi bl e10test all d.X
(ii) To 巴 ei genva¥ues of H,
一 λ 一 → 卜 山 川 一 = 山 川 - 3め)仏恥い川-→- ¥)り)
- 1 2- λ川l 、 ハ
τ'he ei genval ues ar巴 λ =¥ , λ = 3 and thematri x is posi ti vedefinite.
(ui) To cal cu¥ate deterrni nants of all order富 山 al incJ ude the mai n di agonal and
incJ ude theel ement in白 efrrst r owand first col umn,






The matri x is posi ti vedefinite.
Sec_ 4_ 4- 1. mprovi des 山 econfi rmati onof thenumeri cal val ues for this exampl e
usi ngSymbol i c Tool box and basi c MATLAB commands.
4. 4. 2 Equal i ty Const r ai ned Pr obl em
Thc probl em:
Mi ni rni ze f(XI ,x2): -XI X2
切 ω ,
Fi gure 4.7i l l ustrated 白 e graphi caJ sol uti on. It wお noti ced 山 at at thesol uti on, the
gradi ent of 恥 obj ecti vefuncti on叩 dthc gradi ent of theconstrai nt we問 paral l el and
opposi tel y di rected. Exami ni ng other feasi bl e poi nts in Fi st I m4. 7( on the dashed
curve) it can be ascert剖 nedthat thespeci al geometri cal rel ati onshi pis onl y possi bl c
at thesol uti on. At thc sol uti ona proporti onal rel ati onshi pcxi sts between thegradi ents
at the sol uti on. Usi ng thc constant of proporti onal i ty AI (a posi ti ve val ue) thc
rel ati onshi pbetween thegradi cnts can be expressedas
町=ー λI Vhl or Vf +川 町 =0
Equati on (4.48) is usual l y obtai ned in a more formal way usi ng thc met hod of
Lagrange mul ti pl i ers.
Met hod of Lagr ange: Inlhis met hod, theprobl emis transformedby i ntroduci ng
an augmented functi on, cal l edtheLagrangi an. as theobj ecti vefuncti onsubj ect tothe
same 叫 ual i ty cons甘泊 nts. The Lagrangi an is defi ned as the sumof the ori gi nal
obj ecti vc functi onand a l i near combi nati on of theconstrai nts. 前 lecocffi ci cnts of lhis
l i ncar combi nati on 制 官 known 創 出 巴 Lagrange mul t伊 liers. Wi t h rcfcrence 10 thc
exampl c in山 is sccti on
Mi ni mi ze F(拘,x2,AI ) =f(xl ,xu + AI h
(XI ,x2) (4.49)
τbe compl ete probl emis devel oped as
Mi ni mi ze F(XI ,x2,AI) = - XI X2 +λ1(.寸/ 4+4- 1) (4.50)
Subj ecl to: h
(XI ,x2): 4 / 4 +4 - 1
O::;;XI ::;;3; 0::;;x2::;;3
Is 曲 esol uti ontothe回 l sformedprobl cm[Equal i ons (4.50) , (4.39), (4.37)] thesame
as thesol uti ontotheori gi nal probl em[Equati ons (436),(4.39),(437)]?I f thedesi gn
is f i回 si bl e,也 en most defi ni tel y yes. For feasi bl e desi gns, hl (x" X2) = 0,組 d 白巴
o均配 tive functi ons in Equati ons (4.36) and (4.50) are the same. If desi gn is nol
feasi bl e, thcnby defi ni ti onthereis no sol uti onanyway.
τbe F OC areobtai nedby consi deri ngF( x" X2,入1) as 佃 unconstrai nedβmcti ol l in
出 evari abl cs XI ・X2,λl' This provi des 血 reerel ati ons tosol vefor xj , xi ,川
dF i)f • ah,
ι +λ . ー- ! . =o

' .'. ax
aF af . " ahl
一一=一一 +λ ,一一 =0
aA2 ax
" '. aX
一一 =h. =O
Equati ons (4.51) express 出 eFOCor n国 民 S釘 y condi ti ons for 叩 巴 qual i tyconstrai ned
probl emint wo vari abl es. Thc last equati oninthc above set is theCOns回 int equati on.
百 is ensures the sol uti on is feasi bl e. The first two equati ons can be assembl ed in
vcctor f ormtoyi cl dthesame i nformati onexpressedby Equati on (4.48), whi ch was
ob凶 nedgraphi cal l y. 百 leI eft-handexp陀 ssi ons in山 efirst l wo equati ons abovc are
thegradi ent of theLagrangi an functi on:
VF =可 +λI Vh. =0
Appl yi ng Equati ons (4.51) tothc exampl e of 出 is secti on:
a F λ I X1
?一 =- x. +一七ム =0
? 一 =- x. + 2λ x. =o
".:¥__ - -""2 ' "'""¥ol
h 1 4 4 - l =o
Equati ons (4.52) 陀 present 伽 田 equati ons in伽 田 vari abl es whi ch shoul ddetermi ne
the val ues for xj. xi ,λj. Note 白 紙 Equati ons (4.52) onl y defi ne the necessary
condi ti ons, whi ch means thesol uti oncoul d be a maxi mumalso.
Thcreis one probl emwi threspect totheset of Equati ons (4.52). Th巴 equati ons are
a nonl i ncar set . Most prerequi si te courses on numcri cal methods do not attempt to
sol v巴 anonl i near systemof equati ons. Usual l y. l hcyonl y handl e a si ngl eone through
Newt on- Raphson or the bi sccti on method. In fact, NL P or desi gn opti mi zati on is
pri mari l y about tcchni ques for sol vi ng a systemof nonl i near 叫 uati ons,albeit of
spωi fi c forms. Thismeans Equati ons (4.52) cannot be sol vcduntil we advancc釦 rther
in 由 is course. Fortunatel y, havi ng a tool Ii ke MA札 ABobvi ates this di ffi cul ty, and
provi des a s位 。 ngj usti fi cati onof usi nga numeri cal/ symbol i c tool for supporti ng白巴
devel opment of thecourse.
MA礼 ABCode: In MATLAB. there are t wo ways of sol vi ng Equati ons (4.52) , usi ng
symbol i c support functi ons or usi ngthc numeri cal support functi ons. Both procedurcs
havc l i mi tati ons when appl i cdtohi ghl y nonl i near functi ons. The symbol i c functi onis
sol veand血巴 numeri cal functi oni sfsol ve. The numeri cal tcchni queis an iterativeone
and requi res you tochoose an initial gucss tostart theprocedure. Qui te oftensevcral
di fferent guesses may bc 隠 qui red to fi nd 出 c sol uti on. Sol uti ons of systcms of
equati ons aref undamental totherest of 出 ebook. The fol l owi ngcode is i ncl uded悩
a hands-on exerci se. It is avaiI abl e as Sec4_ 4_2. m. It requi res eqns4_ 4_2. mto
executeβ ol vecommand. Thc co蹴 附 of 血 efilcs arelistcdbcl ow
111C問 nrcolher 10018besldcsMATLADsuch田 MUl hcnd副 Id臥 ccl.
S町 4_4_2. m
者 Necessar y/ Suf f i ci ent condi t i ons f or
者 Equal i t y const r ai ned pr obl em

者 Opt i mi zat i on wi t h MATLAB, Sect i on 4. 4. 2
% Dr . P. Venkat ar aman

者 Mi ni mi ze f ( X1,x2) - x1

者 ー ・ ー - - - - ーーーーーーーーー- - - ・・ーーー・
者 symbol i c pr ocedur e
も ー ー ・ ー - - ーーー" ーーーー- - ーー' ーーー・・
者 def i ne symbol i c var i abl es
f or mat compact .
syms x1 x2 l am1 h1 F
者 def i ne F
F - x1*x2 + l am1*( X1*X1/ 4 + x2*x2 ・ 1) ;
h1 x1*x1/ 4 +x2*x2 - 1;
者ヒ he gr adi ent of F
gr ad1   di f f ( F,X1) ;
gr ad2 = di f f ( F,X2) ;
者 opt i mal val ues
も sa七 i sf act i on of necessar y condi t i ons
[ l ams1 xs1 xs2] sOl ve( gr ad1,gr ad2,h1,' X1,x2,l am1' ) ;
も t he sol ut i on i s r et ur ned as a vect or of
者 t he t hr ee unknowns i n case of mul t i pl e sol ut i ons
も l ams1 i s t he sol ut i on vect or f or l am1 et c.
; mOR mT : t h e r esul t s ar e sor t ed 向 habet i 叫 l y
者 f pr i nt f i s used t o pr i nt a st r i ng i n t he
幸 command wi ndow
者 di sp i s used t o pr i nt val ues of mat r i x
f - xs1. *xs2;
f pr i nt f ( ' The sol ut i on ( x1* ,x2合 , l am工*, f * ) ¥n' ) ,
di sp( doubl e( [ xs1 xs2 l ams1 f ] ) ) ー
曹 司 ・ ・ ・ ー - - - - - - - ー ー ー ー ー ・ ー ー ・ ー ー ー ・ ー ー - - -
% Numer i cal pr ocedur e
者 ・ ー ・ ・ ー ー ー ー ー ー ー ー ー ー ー ー - - ー・・・・ーーー- -
も sol ut i on t o non- l i near syst emusi ng f sol ve
も see hel p f sol ve

者 t he unknowns have t o be def i ned as a vect or
告 t he f unct i ons have t o be set up i n an m- f i l e
も def i ne i ni t i al val ues
xi ni t =[ l 1 0. 5] ' ; 者 i ni t i al guess f or x1, x2, 1am1
者 t he equat i ons t o be sol ved ar e avai l abl e i n
も eqns4_4_2. m
xf i na1   f so1ve( ' eqns4_4_2' ,xi ni t ) ;
f pr i nt f ( ' The numer i cal sol ut i on (x1* ,x2* ,l am1*) : ¥n' ) ,
di sp( xf i na1) ;
eqns4_ 4_2. m
f unct i on r et eqns4_4_2( x)
も x i s a vect or
者 x( l ) x1, x( 2) x2, x( 3) l am1
r et =[ ( - x( 2) + 0. 5*x( 1) *x( 3) ) ,
( - x( l ) + 2*x( 2) 合 x( 3) ) ,
( 0. 25*x( 1) *x( 1) + x( 2) *x( 2) - 1) ] ;
Out put InMATLABCommand Wl ndow
The sol ut i on ( x1* ,x2* ,l am1* , f *) :
1. 4142 0. 7071 1. 0000 - 1. 0000
- 1. 4142 - 0. 7071 1. 0000 - 1. 0000
- 1. 4142 0. 7071 - 1. 0000 1. 0000
1. 4142 - 0. 7071 - 1. 0000 1. 0000
Opt i mi zat i on t er mi nat ed successf ul 1y:
Rel at i ve f unct i on va1ue changi ng by 1ess t han
OPTI ONS. Tol F un
The numer i ca1 sol ut i on ( x1* ,x2* ,l am工*) :
1. 4141
0. 7071
0. 9997
The symbol i c computati ongenerates four sol uti ons. Onl y thefirst one is val i dfor this
probl em. Thi s is deci dedby出 esi deconstrai nts expressedby Equati on(4.37). Thi s is
阻i I I ustrati onof thea priori manner by whi ch thesi deconstrai nts affect theprobl em.
On theother hand. 出 enumeri cal techni ques provi deonl y one sol uti on10theprobl em.
百l I s is a functi onof the initial guess. General l y. numeri cal t田 hni ques will del i ver
sol uti ons cl osest tolhepoi nt they startedfrom. The sol uti onis
x; =1. 4141; x; =0. 7071; 吋 =1. 0
τ'he soh瓜 ons for XI 叩 dX2C佃 be veri fi edf romthegraphi cal sol uti oninFi gure4.7.
The sol uti ons canal sobe veri fi edthroughthehand-hel dcal cul ator. The val ueand si gn
for λare usual l yi mmateri al for establ i shi ngtheopti mum. For a wel l -posedprobl em
it shoul dbe posi ti vesi ncc inc陀 asi ngtheconstrai nt val uc is useful onl y if wc aretryi ng
toi denti fy a l ower val uefor 曲 emi ni mum. This forces h and f tomovc inopposi te
di recti ons (gradi ents). I ncreasi ngconstrai nt val uecan be associ atedwi thenl argi ngthe
feasi bl edomai n whi ch may yi el da better desi gn.
Lagr ange Mul t, 伊 ,l l ers: The Lagrange mul ti pl i er met hod is an巴l egant formul ati on
toobtai nthesol uti ontoa constrai nedprobl em. Inovervi ew, it seems strange血 at we
have toi ntroducean addi ti onal unknown (λ1) tosol vetheconstrai nedprobl em. Thi s
vi ol ates 曲 e conventi onal rul e for NL P 白 紙 the f ewer the vari abl es, the better the
chances of obtai ni ng 白 e sol uti on. As i ndi cated in the di scussi on earl i er, 出 E
Lagrangi an al l ows the位 ansformati onof a constrai nedprobl emi ntoan unconstrai ned
probl em. The Lagrange mul ti pl i 巴 r al so has a physi cal si gni fi cance. At thesol uti onit
expresses 山 e ratio of 蜘 change in the obj ecti ve functi on to the change in 臨
constrai nt val ue. To illustrate出 is consi der:
F =f +λI hl
dF=df +λI dl1
d F . d F . dF
一 , 一 一 命 令 :: aA.,
_. . . dX2- ' 2 • dλ - ,..
At the sol uti on, theF OC deems that dF = 0 (whi ch can al so be seen in the above
detai l edexpansi on). Hence,
λ 一一王立 L笠
1 - dh 611
The above dependence do巴 s not affect theestabl i shment of theopti mal desi gn. It
does hav巴 an i mportant rol e in the di scussi on of desi gn sensi ti vi ty in NL P
probl ems.
Gener al Equal l ty Constral necl Pr obl em: Rememberi ng n - 1>0,
Mi ni mi ze f ( X) ,匹 J'I (4.5)
(4.6) Subj ect ω[lI (X)], = 0
SXSXup (4.8)
The augment ed probl emwi ththeLagrangi an:
Mi ni m取 F( Xム' ) =f ( X) +L 入品 (X)
=f ( X) +A' h;
=f (め +l Th (4.53)
Subj ect to: [h(X)], = 0
10Equati on(4.53), threeequi val ent representati ons for theLagrangi an areshown. The
FOCar e
dF ν ムヨ h" (4.54)
一一=ーム+) A."   =0; i =1 , 2
aJEi axi z k h'
Subj ectω: 胆 (X)],=o

Equati ons (4.54) and (4后) provi de the11+ 1 rel ati ons todetermi ne恥 n+ 1 unknowns
xヘ λ・. Equati on(4. 8)i s used after thesol uti oni sobtai ned-i f necessal y. Equati on
(4.54) is al soexpressedas








Second-Order Condi ti ons: At 恥 sol uti on determi ned by 山 F OC,the functi on
shoul d i ncreasefor changes in6X. Changes in企X arenot arbitrary. They have to
satisfy曲 el i neari zedequal i tycons町 aint at thesol uti on. 1t is takenfor gr血 旬 d出 atthe
suffi ci ent condi ti ons Me usual l y appl i edin a smal l nei ghborhood of the opti mum.
Al so,changes arecontempl atedonl y wi threSPECt to X and not wi threspect to thE
Lagrangemul ti pl i er- TKLagrangemethodi s o b n cal l edthemethodqf undetermi ned
coefficient.ー i ndi cati ngit is oot a vari abl e. 1nthe叩 a叫ly 凶 “ c叫叫 d白eri 巾 va副6ωo 叩 nof t白heFOCf or
帥 :pp州 emπm 凶l 恥 Lagrangi a 釘I刊 was c ∞on帥 叫 u聞 附 n即 c ∞o ∞ns蜘 i削 B勘 0叩 r 町r側 n 暗削 g f ,加 m削 『 町 m川n t 白he
previ ouss記巴 c叫凶ω0 叩 n(凹 c ∞on 附 stra凶i ne叫dmi加n泊回1由i訟za 副t “i on 叫) ,血巴 SOCc叩 beexpec飽 dtosatisfythe
fol l owi ngrel ati ons:
岬 =F(侭X* +必訓)ト川-イF 尺'(X.) 乃乍 ) =VF( 賦'(X'噂 ) → 邸 d凶 T 可[ 内 v炉内 z 守頓怖 2F(X' 駅限献附 ( αぽ附削 X' ) ワ羽)] 瓜 〉刈O 糾
Intheabove [ V2F( X勺l i s theHessi an of theLagrangi an,wi threspect tothedeSI gn
vari abl es onl y eval uatdat thesol uti on. Al so,theF 0C mqui mt hat VF( X・) =0. Wi t h
referencetot wo vari abl es and one constrai nt:
t:.. F =核吋 +2三走州向)+言叶〉。
M = i 際問 + 2 t L位)+詞吋〉。
dh/ ox
dh1/ dx
Substi tuteEquati on(4.58) inEquati on(4.57) and theSOC requi res that theexpressi on
in br aωmust be posi ti ve. 百 四 deri vati ves in Equati ons (4.57) 叩 d (4.58) 師
eval uatedat them泊 i mum.
Appl yi ng thesecond-order condi ti on to theexampl e of 白 is secti on is 1巴 ft お 叩
exerci se for the 陀 ader. Compar芭 d to 白 e SOC for the unconstrai ned mi ni mi zati on
probl em, Equati ons (4.57) and (4.58) 町 宮 not 側 syωappl y,especi al l ythesubsti tuti on
of Equati on(4.58). Fr oma practi cal perspecti v巴 批 SOCis not i mposed for 叫 uality
constrai ned probl ems. It is left to the dcsi gncr to ensure by other means 白 川 出 e
sol uti onis a mi ni mumsol uti on.
4. 4. 3 I nequal l ty Const r ai ned Opt l mi zat i on
Thepr obl em
Mi ni mi ze f(XI ・Xz}:ー XI X2 (4.36)
Subj ect to: gl (XI ,x2): 20 XI + 15X2- 30 S 0 (4.40)
, S O (4.41)
OSXI S3; OSX2S3 (4.37)
The number of vari abl es (n= 2) 剖 dthenumber of i nequal i tyconstrョints ( m= 2) do
not depend on each other. In fact. an i nequal i ty cons釘ai ned probl emin a si ngl e
vari abl e can be useful l y desi gned and sol ved. Secti on 4.4.2 sol ved the 叫 uality
constrai ned probl em. If the above probl emcan be 回 nsf ormed to an equi val ent
cqual i ty constrai ned probl em. 出 en we have f ound the sol uti on. 百 1e standard
回 1sformati onreq凶 res a sl ack vari abl ezj for eachi nequal i tycons凶 nt g
j ・
Unl i keLP
probl ems, thesl ack vari abl efor NL P is not restri ctedinsi gn. τberefore, thesquare of
the newvari abl eis added tothel eft-handsi deof thecorr四 pondi ngcons住 ai nt . Thl s
adds a posi ti veval uetotheleft-h叩 dsi detobri ngthecons回 int up tozero. Of course
a zeroval uewi l l be added if theconstrllint is alrelldy zero.
1週目 ; 1
Transformatl ont o s n Equal i tyConstrai nedProbl em
Mi ni mi ze j(XI ・X2): -XI X2
Subj ectl o: gl (XI ' Xz}+ zi: 20xI + - 30 + zi = 0
g2(X. , X2) 這
There arefour vari abl es (XI.X2. ZI. Z2) and twoequal i ty cOllstraints. It is a val i dequa1ity
cons回 i ned probl em. The Lagrange mul ti pl i er m巴 thod can be appl i ed to this
岡 由 mau- - Tod叫 ngui sh曲 emul ti p目 的 制 oci atedwi 出 i叫 叫 ity cons凶 脳
血 esymbol is used. This is s凶 ct1y for cI 回 ty.
Method of Lsgrsnge: The augmentcd functi onor theLagrangi anis:
Mi ni mi ze

, ・ ・  

=f (x
) + s
) + zil + s2[C2(XI' ろ ) (4.61)
If the Lagrangi an is consi dered 鎚 an unconstrai ned obj ecti ve functi on. the FOC
(necessary condi ti ons) are
dF df . / l ogl . / l dg2
一 = - 4 1一 +s 一
axl a x l a X1 2 a x l
dF of . / l ogl . / l
一 = ー + + s2ー =0
dX2' I"' . dx
dF _ n
a Z
dF . n
? ー =2s. z. =0
a Z
dF ?
ー =g. +zf =O
dPI u l
iJ F ?
ー ,
U " , 必
(4. 62b)
Equati ons (4.62e) and (4. 62t) are珂 ual i tyconstrai nts. Equati onset (4.62) provi des six
句 uati onstosol vef orx1,xE,z; ,Z5.91. 向. By si mpl e recombi nati on, Equ副 ons(4. 62c)
to(4. 620 can be col l apsedtot wo equati ons, whi l e thesl ack vari abl es ZI and Z2can be
el i mi natedfromtlzeprobl em.
Fi rst mul ti pl y Equati on (4. 62c) by ZI ・Repl acez t by -81 f romEquati on (4.62e).
Dr op thenegati vesi gnas wel l ωt he coeffi ci ent 2 toobtai n
。18. =0
Equati on (4. 63b) is obtai ned by carryi ngout si mi l ar mani pul ati ons wi thEquati ons
(4. 62d) and (4. 620. The F OC can be restatedas
dF df . n d8. . n d82
一 =;J + 132ー =0
dx. dx. r1dx. '
dF dr d8I d8今
一=二: : L+ 132ー=0
1318. =0
(4. 62b)
(4. 63b)
These four equati ons have tobe sol vedfor xj , xi ,日j ,s 2 ・ Note伽 tzi , Z2areoot bei ng
det ermi ncd- whi ch suggests 白 紙 白 eycan be di scardedf rom恥 probl cma1togc白 er.
It woul d be useful topretcndz's ncvcr exi stcdinthefirst pl ace.
Equati ons (4.63) l ayout a dcfi ni tefeaturefor a nontri vi al sol uti on: ci ther sj is zero
(and8/ 手 0) or 8/ is zero ( 島刊) . Si nce si mul taneous cquati ons 蹴 bei ngsol ved,出 e
condi ti ons on 曲 e mul ti pl i ers and constrai nts must be sati sfi ed si mul taneousl y. For
Equati ons (4.63) ,出 is 甘 組 sl aωs i nto白 efol l owi ngfour cases. Thc i nformati onon 8 in
brackets is tocmphasi ze an accompanyi ng consequence.
Case a: s. = 0 [8. < 0]; 132= 0 [82< OJ
Case b: 仇 =0[ 8. <0] ; 仇 * 0[82= 0]
Casec: 13
* 0 [8. =0]; 13
=0 [ 82<0)
C邸 巴 d: 日I 棋 0[8. =0); 九 * 0[82= 0]
In Equati on (4.64), if sj * 0 (or correspondi ng 8/ = 0) , then the correspondi ng
cons位aI nt is an equal i ty. Intheprcvi ous sccti ona si mpl e reasoni ngwas usedωshow
白 鉱 山 c si gnof 出 e m叫 tiplier must be posi ti ve( >0) for a wel l -formul atedprobl em.
Whi lethesi gnof themul ti pl i er was i gno問 d for 白 eequal i tyconstrai nedprobl em, it
is i ncl udedas part of theF OC for 出巴 i nequal i tyconstrai nedprobl em. Bef ore restati ng
l 伺目訂
Equati on(4.64) , theLagrangi anis reformul atedwi thout thesl ack vari abl esぉ (weare
goi ngtopretendZ never exi sted)
Mi ni mi ze F(x. , x
) = .f{x. ・x
) + s.[g.(x..x2)] + s
[ 8T, . X
)] (4. 65)
whi ch is thesame formul ati oninEquati on (4.53). The sl ack vari abl ewas i ntroduced
to provi de 出巴 transformati on to an equ叫 ity cons回 int . It is al so evi dent 白 鉱 山 e
cons凶 cti onof 由巳Lagrangi anfuncti onis i nsensi ti veto白巴 typeof constrai nt Si nce
白 emul ti pl i ers tiedtothei nequal i tyconstrai nt arerequi redtobc posi ti ve, whi l e those
correspondi ngtotheequal i tyconstrai nts arenot, this book wi l l conti nuetodi sti ngui sh
between出巴 mul ti pl i ers. This will servetoenforcc clarity of presentati on. The FOC
for theprobl em:
e)F dr d8 I d8内
一 =二 L 4 1 4 4 H=o
dx, dx, ' I"" 'OXt . I""Zdx.
dF df . n dg. . n dg
一 = ー +   + 132一 =0
dXz dx
' 1""' d.Xz . I""Z d.Xz
Case a: 払 =0[gl <0]; 日2=0 [g2 < 0]
Case b: 13, = 0 [8. < 0]; 仇 >0 [g2 = 0]
Case c: s. > 0 [g. = 0]; s2= 0 [g2 < 0]
Case d: s. > 0 [8. = 0]; 島 >0 [82= 0]
(4. 62b)
Equati on set (4.62) and any si ngl ecaseinEquati on (4.64) provi des four equati ons to
sol v巴 for thc four unknowns of the probl em. Al l four scts must be sol vedfor the
sol uti on. The best desi gnis deci dcdby scanni ngtheseveral sol uti ons.
Thc si gn of the mul ti pl i er in the sol uti on is not a sujJi ci ent condi ti on for the
泊 叫 ual i tyconstrai nedprobl emThc val ue is un加 portant for opti mi zati onbut may be
rel evant for scnsi ti vi tyanal ysi s. General l y a posi ti veval ueof 由巳 mul ti pl i er i ndi catcs
白 紙 the sol uti on is not a l ocal maxi mum. Formal l y veri fyi ng a mi ni mumsol uti on
req凶 偲 s consi derati on of the sccond dcri vati ve of thc Lagrangi an. In practi cal
si tuati ons. if thc probl emis wel l defi ned, thc posi ti veval ue of 山 c mul ti pl i er usual l y
suggcsts a mi ni mumsol uti on. This is used cxtensi vcl y in thc book to i denti fy the
opt i mumsol uti on.
Sol utl on of the Exampl e: Thc Lagrangi anfor theprobl emis defi ncdas
) =- x. .Xz+ s,(2Ck. + 15Xz- 30) 九 お
TheFOCar e
Z; =42+2ORI +05h=0
55=41+15P l +2h=0
+ 15
- 30) =0
(0おイ +4- 1) =0
The si deconstrai nts 制 官
S3; O S
S 3 (4.37)
Case a:
sl = 0;
s2= 0: The sol uti onis 町 ivial and by i nspecti onof Equati ons (4.66n)
and (4.66b).
X1=0; x
=0; 1=0; gJ =ー30;
g2=ー (4.67)
The i nequa1itycon抑 制 ts aresatisfied. The si decon抑 aints are削 isfied. The val ues
inEquati on(4.67) rep回 sent a possi bl esol uti on.
The sol uti ons for theother cases 釘 ' c obtai nedusi ngMA11. AB・百 lecode is avai l abl e

m. For Cases b and c theappropri atemul ti pl i er is set tozeroand the
resul ti ngthrEeEquati ons inthreeunbEowns aresol ved. For casE d thecompl ete set
of four equati ons infour unknowns is sol ved. cased is al soSOl vednumeri cal l y. 明 le
MATI . AB code will a1soruncode Fl g4_8. mwhi ch contai ns th巴 commands todraw山 e
figure. The output f romtheCommand wi ndowis patchedbe10w.
The sol ut i on *** Case a ***( x1* ,x2* , f * ,
o 0 0 ・ 30 ・ 1
The sol ut i on

**Case (b) 合**
( x1* , x2合 ,
b2* f * , 91, 92) :
1. 4142 0. 7071 1. 0000 - 1. 0000 8. 8909 0
- 1. 4142 - 0. 7071 1. 0000 - 1. 0000 ・ 6日 . 8909 0
- 1. 4142 0. 7071 - 1. 0000 1. 0000 - 47. 6777 0
1. 4142 - 0. 7071 - 1. 0000 1. 0000 - 12. 3223
The sol ut i on
Case ( c) *** ( x1* ,X2* , b1* f * ,
91, 92) :
o . 7 5 0 0 1 . o o oo o . 0 5 0 0 - o . 7 5 0 0 o
o . 14o 6
The sol ut i on
***Case( d}***{xl * ,x2* , bl * b2* f * ,gl , g2) :
1. 8150 ・ 0. 4200
0. 0426 ・ 1. 4007 0. 7624 0 0
0. 8151 0. 9132
0. 0439 0. 0856 - 0. 7443 0 - 0. 0
Maxi mumnu
of f unct i on eval uat i ons
l ncr ease opt i ons. MaxFu
nEval s
Opt i mi zer i s st uck at a mi ni mumt hat i s not a r oot
Tr y a9ai n wi t h a new
st ar t i n9 guess
The numer i cal sol ut i on (x1合 ,
X2合 ,
b1* ,b2*) :
0. 8187 0. 9085 0. 0435
0. 0913
The sol uti onfor Case a was di scussedearlier and is conrtrmed above.
Case b hns four sol uti ons. The first one is unacceptabl ebecause constrni nt gl is not
satisfied. The second sol uti onis feasi bl ens far as thefuncti ona1constrai nts g, and g2
are concemed but they do not satisfy the si de constrai nts. The third and fourth
sol uti ons are unacceptabl e for the same reason. Thus. Case b is not nn opti mnl
sol uti on.
Cnse c is unacceptabl ebecause constrai nt g2 is inviolation.
Cnse d hns t wo sol uti ons thefirst of whi ch is not acceptabl efor several rensons. The
secondsol uti onsntisfies nll of thereq凶 rements
• It sntisfies theconstrai nts. 80th constmi nts 釘 宮 nctive.

The mul ti pl i ers む eposi ti ve(maybe a suffi ci ent condi ti on).
・I t satisfies thesi deconstrai nts.
百 lesol uti onis
=0. 8151; X2= 0. 9132;

=0. 0439;

=0. 0856;
1 =ー0. 7443;
gl =
g2 =
This is nl most confi rmed by出巴 numeri cnl sol uti onwhi ch nppears tohnve a probl em
wi 出 convergen田 . It is al most at thesol uti on. 1t must be nol edherethat theMATLAD
functi onused tosol ve白巴 probl emis part of thestandardpackage. The functi ons in
the Opti mi zati on Tool box will certain1y do better. Neverthel ess. the i ncompl ete
sol uti onis reportedbel owbut will be i gnoreddue to出 esymbol i c sol uti onavai l abl e
inCase d.
x,=0. 8187; x
=0. 9085;
,=O. 似 35;

=0. 0913


There arenowtwo candi dates for 白 esol uti on: 出 etrivinl sol uti oninCase a and the
sol uti oninCnse d. The sol uti oninCns巴 dis favoredns it hns a l ower obj ecti vefuncti on
va1ue. Al l of the cnses above can be expl ored wi 由 民 spect to Fi gure 4.8. It is an
excel l ent facilitator for comprehendi ng thecases and血巴 sol uti on.
4. 4. 4 General Opti ml zatl on Pr obl em
τ'he genera1opti mi zati onprobl emisdescri bedintheset of Equati ons (4.1) ー (4.8). 百 le
speci fi c probl eminthis chapter d巴fi nedinSecti on4.3is
Mi ni mi ze j(Xl 拘) : ー 'X,X2
Subj ect to: h1(Xl
V: 20 x, + 15X2- 30 = 0 (4.10)
gl (X,,xV: <4/ 4) + ~   ー I S 0 (4.11 )
OSXl S3; OSX2S3 (4.12)

島;(XIoX2・. . . ,x,J S O. j = 1.2. . . . • m
議長野忠 ZAiil 端 部 出 ;;355
(4.4) XI   j = 1.2. . . . • n
The Lagrangi an:
出;雪 Mu/ i附 附 od: The probl cm…sfi…y mi ni mi zi ng thc
Mi ni mi ze
,Xn,A. ,... ,λ/. sI' . . . . srn) = F(XJ ,
2. A. ..sU= - X .X2+入1(20x. + 15x2- 30) + s. (0. 25x? +xi - 1)
Mi ni mi ze
Subj ect 10:
Therearen + 1 + m unk.nowns. The sarnenumber of cquati ons arer .叫 ui redtosol ve
出 eprob1em. 百】 ese釘 "eprovi dedby theF OC or 出 eKuhn- Tucker condi ti ons:
f(x,・・・ . . X
) +λ,h, +・・・+入内 +s. g,+・・・ +srnCm
"1(XJ ,
2): 20xI + 15x2- 30=0
i =1. 2. . . . . n
11cquati ons areobtai nedas
dF df . dh," . , dhl . n dg,. . n dg
一 =ー +A. ,- : ; ート・+入ー +PI - - +ー・+sm-:
dx, dx," - ' , dx, - --'dx, - r , ax, - r m ax,
(4. 68b)
Z; =- X2+2瓜 I +O.5s内 =0
=- x. 十 叫 +明 x
l 叫 uati ons ar芭 obtai neddi rcctl y也 ruugh山 ecqu凶 ityconstrai nts
瓦 - =2伽 1+
m cquati ons areapp1 i ed through出巴 2
cases. Thi s i mpl i es 白 紙 出 ereare2
possi b1e
sol uti ons. Thesc sol uti ons must i ncl udeEquati ons (4.71) and (4.72). Each cas巴 sets the
mul ti pl i er 島 or 由巳 correspondi ngi nequal i ty c 叩0叩 n附 st釘m凶ai n 蹴t 身がtωo ze巴 瓜 I百ft血he 引 ml 叫t
set to zero. 出 en 出 e correspondi ng cons位置 int must be feasi b1e for an acccptabl e
sol uti on. If 出 e constrai nt is set to zero (acti ve constrai nt). then the correspondi ng
mul ti pl i er must be posi ti vefor a mi ni mum. Wi t h出 is inmi nd出 cm equati ons canb官
k = 1.2.. ...1


• ・ . X
) =0; (4.68c)
(4. 69b)
32;: 町 立 は ; ; z z u y f肌 ωa同 ui rωthesol uuo山 system
結 滞 在 i結 成 23222!な 1223;
gl <0
g. =0
s, =0;
Case a:
Case b:
If con似 ons in Equati ons (4.73) are not met. 出 e dωi gn is not acceptab1e. In
i mpl ementi ng Equati on (4.73). for each case a si mu1taneous to凶 of m va1ues and
cqual i ti es must be assi gned. Once theseF OC condi ti ons determi ne a possi bl esol uti on.
thc si deconstrai nl Shavc tobe checked. As cvi denced in出 c examp1es ear¥i er. this is
not bui l t i nto 出巴 FOC. lt is on1y conf trmed after a possi b1e sol uti on has been
i denti fi ed. Equati ons (4.71)ー(4.73) arereferredtoas theKuhn・Tuckercondi ti ons [see
Exarnpl e 4.3(Secti on4.5.2) for addi ti onal di scussi onof theKuhn-Tucker condi ti ons].
then gj < 0








設立制思議部議悲鳴; :
f(X. ,x2・・.. . x,J
"k(XI ,X2... . . xn)=O. k = 1.2... ..1
Su町民 lto:
Mi ni mi ze
Two exarnpl es arepresentedinthis secti on. The first is an unconstrai nedprobl em白 紙
has si gni fi cant us巴 indatareducl i on. Speci fi cal l y, it illustrates theprobl cmof curve
fittingor regressi on. The second is thc bearndesi gnprobl emexpl o閃 d inChapters 1
and 2.
4. 5. 1 Exampl e 4. 2
Probl em: A set of y, Z data is gi ven. It is neccssary to fi ndthe best strai ght line
出 roughthedata. Thi s exerci seis termed c町 vefittingor data陀 ducti onor regressi on.
Tok田 Pthisexarnpl esi mpl e thedatapoi nts aregeneratedusi ngtheparabol i c equati on
Z = O.5y2. The dataaregeneratedinarbi traryorder and nonun丘or mi nterval s.
Least Squar ed Error: The obj ecti vefuncti on白 紙 dri ves a l argenumber of curve
fi tti ngl regressi on methods is the mi ni mi zati on of the least squared error. 百 le
constructi on of the obj ecti ve 問 中 ires t wo enti ti ω- the data and 山 type of
mathemal i cal rel ati onshi p betw回 n the data. In this 巴 xarnpl e it is a strai ght line.
General l y it can be any pol ynomi al , or any functi on ror 山 at manner. Thi s is usual l y
theway corrcl ati onequati ons aredetcrmi nedinex. peri mcntal fluiddynami cs and heat
transfer. For this ex創 npl e血 edataarethecol l ecti onof poi nts (Y;・ Z/) ,i = 1,2, • • • ,n
Th 批1施ee. 郎 xp 戸ec也凶dstrai ght Ii n巴 c叩 巴 】   隠 句 y
where xl is thesl opeand X2 is 出 ei ntercept. Usi ng opti mi zi ngtermi nol ogy, thereare
t wo desi gnvari abl e渇. For a cubi c pol ynomi a I , therewil¥be four desi gnvari abl es. 百
Zpj is 血巴 fittedval uefor 也 ei ndependcnt vari abl eY; ,出 eobj ecti vefuncti on, whi ch is
themi ni mumof thc squ紅 eof theerror ovcr aI l of thedatapoi nts. can bc巴xp閃 ssedas
n 11
Mi ni mi ze f (x
) = L [ Z/ - zpi = L [Z/ ー (X尚 +X2)]2 (4. 74)
;=1 1=>1
It is possi bl e tocompl ete恥 formul ati onwi 出 si decons凶 nts aI 出 ough白 紙 is not
necess釘 y. 明 leF OC ar芭
ま=土 2[ Zj - ( x
ま=; 2I Zi - 仰 i叩
4.5 EXAMPLES 195
Exp叩 di ngtheexpressi ons inthebrackets and reorgani zi ngas a line釘 ma住 i xequ副 on
ヱyf 2 , Y; 司 LZ;Y;
LY; L11凶 I iz;
/=1 I=J ;=1
Not巴 that the matri ces can be set up easi l y and sol ved usi ng MATLAB. The Hessi an
matri x is thesquむ ematri x on theleft The SOC requi res 出 at it be posi ti vedefinite.
Code: Sec4_5_1. m: Exarnpl e 4. 2is sol vedusi ngMATLAB佃 dtheresults f rom出 e
Command wi ndowareshown bel ow. Compar e this code wi th出 at inChapter 1. The
results aredi spl ayedon a pl ot (not i ncl udedin由巳 book). New features in出巴 codeare
theuseof randomnumbers, usi ngthesumcommand, and wri ti ngto theCommand
wi ndow. Fr omthepl ot and山巴 tabul ar vaI ues it iscl ear that a l i near fit is not acceptabl e
for 由 is exampl e. The quaI ityof thefit is not undcr di scussi onin山 is 巴xarnpl eas it is
apparent 白 紙 anonl i near fit shoul dhave bcen chosen. Thi s is lefl as anexerci sc. Thc
fol l owi ngwill appear in山 eCommand wi ndowwhcn出 ecode is run.
Resul t s f r omLi near f i t
obj ect i ve f unct i on: 16. 5827
desi gn var i abl es x1, x2:
2. 3458
- 1. 5305
yi Zl. Zp di f f
4. 7427 11. 2465 9. 5951 1. 6514
3. 7230 6. 9304 7. 2031 - 0. 2727
4. 3420 9. 4266 8. 6552 0. 7714
0. 3766 0. 0709 ー0. 6470 0. 7179
2. 4040 2. 8895 4. 1089 - 1. 2193
4. 1054 8. 4272 8. 1002 0. 3270
4. 1329 8. 5404 8. 1646 0. 3757
3. 3369 5. 5674 6. 2973 ー0. 7299
2. 7835 3. 8739 4. 9991 - 1. 1253
4. 2032 8. 8334 8. 3296 0. 5039
3. 1131 4. 8456 5. 7723 - 0. 9267
3. 1164 4. 8561 5. 7802 - 0. 9241
3. 2415 5. 2538 6. 0737 ー0. 8199
0. 1954 0. 0191 - 1. 0720
1. 0911
0. 7284 0. 2653 0. 1781 0. 0871
4. 5082 10. 1619 9. 0450 1. 1168
1. 9070 1. 8183 2. 9430 - 1. 1247
0. 4974 0. 1237 - 0. 3638 0. 4874
1. 9025
0. 1749
1. 8098 2. 9325 - 1. 1227
0. 0153 - 1. 1203 1. 1356


- r















4. 5. 2 Exampl e 4. 3
Thi s exmnpl c is thesame as Exampl e 23. t he fl agpol epmMem. I t is qui tedi ffi cul t to
器部諸説 3315日程盟百
総 itzi 締 結 鵠 F限 切 c z
Mi ni mi ze f(xl 占 ) = 6. 0559 E -
Subj ect to:
gl (X内 ): 7. 4969E05.x7+ 40000x
- s 0
g2(XI' ろ ):(5卿 +1. 4994E 05x
)(.x7+ X内 吋 ) ー 1・7083E07( Xi S 0 (4.77b)
g3(Xl' x2): 1. 909I E- 03xl +6. 1116E- ω - 0. 05( x1- xi) S 0
  - x
+ 0. 001 S 0
(4. 77c)
(4. 77d)
0. 02S x
S 1. 0; 0. 02S x
S 1. 0
K駒 'uh 加n- 7 i 町'ucke. 町 r マ ω'ondii削 胸 N陶 ons 釘 : F , 而ou町 叫 r La砲grang 酔cmL 机t刷 pli附釘 ersa 蹴 r陀 川 泡 i加n附n則Ir 刷r
血 珂 u油 tザy c ∞o由 t佐reU創 nt匂s 佃 dt白heLagra 加n沼頃gi an 叩 nis
F( X,s) = 6. 0559 E 05(xf +
s p .4969E 05xf + 40, ∞OX
- 9. 7418E06( x1- xi ) ) +
((50∞+ 1.4994E 05x
)(xf + 内 ー +
臥(1. 9091E- 03x. +6. 1l 16E- ω ー 0. 05( x1- 4) +
s4( X
- x . + 0. 001)
4.5 EXAMPLES 197
where X = [X¥oX2] and s = [131, 132,仇, s4]T. 百 leKuhn-Tucker condi ti ons requi ret wo
叫 uati ons usi ngthegradi ents of IheLagrangi an wi 出 向 spectl o白 edesi gnvari abl es


aF _ aF
一一 =0: 一一 =0

The acωaI expressi onfor 白 egradi ents obtai nedf romEquati on (4.80) is left as 叩
cxerci sefor thestudent. 百 leKuhn-Tucker condi ti ons areappl i edby i denti fyi ngthe
vari ous cases based on theacti veness of vari ous sets of constrai nts.
For this probl emn = 2 and m = 4. Thcre are 2削 =2
= 16 cases that must be
i nvesti gatcd悩 part of 出巴 Kuhn・Tucker condi ti ons gi venby Equati on (4.73). Whi l e
some of thesecases aretrivial , neverthel ess it is a formi dabl e task. Thi s probl emwas
sol vedgraphi cal l y inChapter 2 and thesame graphi cal sol uti oncan be expl oi ted10
i denti fythep紅 ticular case白 紙 needs tobe sol vedfor thesol uti on. Vi si ti ngChapter 2
and sc叩 ni ngFi gure 2.9, it c釦 bei denti fi ed白 at cons回 ints 81 and 83areacti ve(as
conf i rmed by 出 ezoomed sol uti on inFi gu問 2.10). The sol uti on is xT= 0. 68m and
xi = 0. 65m.
lf 81and 8J areacti vecons凶 附 , 白 E叩n由恥emL 刷 u山l
By thc sa 邸 mereasoni ngthemu 凶ltipμli旬ers associ atedw 別 i由 出 巴 inacti ve constrai nts 82叩 d
E伽4恥 ' 伽 t山 i旬S,132and 134, must bc set tozero. This i nformati onon theacti veconstrai nts
can be used tosol vefor xj , xi 鑓 白 is is a sy山 mof t wo叫 uati ons int wo unknowns.
Thi s d脚 not , however, compl ete印 刷 sfacti onof 白 eKuhn- T山 ker condi ti ons. 131
and sJ must be sol vedand veri fi edthat they are伊 副 ve. 82and 84must be eval uated
and veri fi edthat 出 ey areless thanzero.
Sec4_5_2. mis thecode segment 血 at is usedtosol ve白巴 parti cul ar casedi scusscd
abovc. Thc probl emis sol vcdsymbol i cal l y. Two vcrsi ons of theprobl emaresol vcd:
theori gi nal probl emand a scal edversi on. 百 1巴 obj ecti vefuncti on(4.76) is lJl odi fi ed
tohave a coeffi ci ent of uni ty. Thi s shoul dnot ch回 getheval ueof thedesi gnvari abl es
(why?). You can experi ment wi th恥 codeand veri fy if 白 is is i ndeed凶 e. Note白 c
val ues of theLagrange mul ti pl i ers duri ng血 is cxerci seand i nfer therel ati onbetween
the m凶 tipliers and the scal i ng of 血巴 obj ecti vefuncti on. The fol l owi ngdi scussi on
assumes that thecode has been runand val ues avai l abl e.
Referri ng to 山 e sol uti on of thc casc above, the opti mal va)ues for 山巴 desi gn
vari abl es are xj :::: 0. 6773 m and xi = 0. 6443 m. Thi s is very cl ose to the graphi cal
sol uti on. Bef ore we concl ude白 紙 thesol uti onisachi eved, takenote of themul ti pl i ers:
向 =5. 2622e - 010叩 d向 =-0.1270. If themul ti pl i ers arenegati ve,白 enit is not a
mi ni mum. The val ues of 由 econstrai nts at theopti mal val ues of thedesi gnare82=
- 4. 5553e + 007 and 84 =ー 0. 0320. Bot h val ues suggest that the cons凶 nts are
i nacti ve. At least 出 esol uti onis feasi bl e-al l cons回 ints aresatisfied. The val ueof
st is zeroand it is a1soan acti vecon
. ,
consi deredacti ve. 百 1esol uti on, however, vi ol at岱 thesi decons往 aints.τ 'hegraphi cal
sol uti onis rather expl i ci t ini denti fyi ngthemi ni mumat 出 epoi n包 establ i shedinthe
first sol uti on. 百 1e凶 vial caseof themul ti pl i er bei ngzerowhen theconstrai nt is acti ve
is observcd wi threspect to the first t wo constrai nts 白 紙 express stress rel ati ons. ln
general ,出 c tcrms inthesei nequal i ti es have l argeorders of magni tude, especi al l y if
thereareoth巴requati ons 出 at express constrai nts on di spl acements, 制 芭 as,or thelike.
Large val ues exert a l argei nfl uenceon theprobl emsomuch as tocause l ower val ue
constrai nts tohave littleor no signifi 四 ncc for 曲 eprobl em. This is usual l y a severe
probl eminnumeri cal i nvesti gati ons. The remedy for 白 is is scal i ngof thefuncti ons.
Scal l ng: Consi dcr 出 e order of magni tude in Equati ons (4.77a) and (4.77c).
Numeri cal cal cul ati ons are 合 i ven by l arger magni tudes. l nequal i ty (4.77c) wi l l be
i gnoredinrel ati ontotheothcr functi ons even though thegraphi cal sol uti oni ndi cates
白 紙 お is acti ve. 百世 sis a frequent 侃 curren田 inall ki nds of numeri cal t配 hni quω. The
standardapproach to mi ni mi ze白 e i mpact of l argevari ati ons inmagni tudes among
di ffe悶 1t equati ons is tonormal i ze出 erel ati ons. Inpracti cethis is al soextended to出 e
vari abl es. 百 is is referredtoas scal i ngl hevari abl es and scal i ngthefuncl i ons. Many
current software wi11scal etheprobl 巴mwi thout user i nterventi on.
S伺 l i ng Vari abl es: 百 1eprcsencc of si deconstrai nts inprobl emformul ati onal10ws
a natura1dcfi ni ti onof scal edvari abJes. The user-defi nedupper and l ower bounds are
used toscal eeach varl abl ebetween 0四 d 1. Therefore,

え=→- - ' ,; x: = scal edithvari abl e
Xi - Xi
X,=x;(.ヰ -x:) +X:
(4,8I b)
Intheori gi nal probl emEquati on (4.81b) is usedtosubsti tutefor theori gi nal vari abl es
after whi ch the probl emc叩 be expressed in terms of scal ed vari abl e. An al temate
formul ati onis to use on1y theupper val ue of thesi deconstrai nt to sca1e thc desi gn
vari abl e:
si =xi / 4e
xj =xj 寓
(4. 82b)
Whi l e曲 is l i mi ts 山巴 hi gher scal edva1ue to 1, it does not set 出 el ower scal edval ueto
zero. For 出 e exampl e of 山 is secti on, there is no necessi ty for scal i ng the desi gn
vari abl es si ncetheir ordeJ' of magni tudc is one, whi ch is exactl y what sca1i ngatl empts
toachi eve.
Scal i ng the Constrai nts: Scal i ngof thefuncti ons in出 eprobl emis usual l y critical
for a successful sol uti on. Numeri cal techni qucs usedinopti mi zati onareiterative. In
4. 5 EXAMPLES 199
ωch i terati onusual l y白 egr叫 i ent of thefuncti ons at thc current val ueof thedesi gn
vari abl es isi nvol vedinthecal cul ati ons. Thi s gradi ent, exp陀 ssed凶 amatri x, is ca1l ed
出 eJ acobi 仰 matri x or si mpl y theJ acobi al l . Sophi sti cated scal i ngtechni ques [7,8]
empl oy thedi agonal en凶 es of this matri x as metri cs toscal etherespecti vefuncti ons.
Theseen肘 ωar eeval uatedat thestarti ngval ues of thedesi gnvari abl es. The functi on
canal sobe sca1ed in山 esame manner as Equati ons (4.81) and (4.82). For thef ormer
anexpectedl ower and upper val ues of theconstrai nts ar芭 necessary. Inthis exerci sc,
出 e constrai nts will be scal edusi ng rel ati ons si mi l ar to the rel ati ons expressed by
珂 ualities (4.82). The scal i ngfactor for each cons位回 nt wi l l be determi nedusi ng白 E
starti ng val ueor theinitial guess for thevari abl es. A starti ng val ueof 0.6for both
desi gnvari abl es is sel ectedtocomput e白 eval ues n巴 cessaryfor scal i ngthefuncti ons.
The scal i ngconstants for theeq凶 凶 ons arecal cul ated凶
g¥O= 293888.4; g20=102561.12; g30=1万 70E- 03; g40=1
百 1e first three constrai nts are di vi ded through by their scal i ng constants. The last
珂 uati on is unchanged. 百 1eobj 民 tive functi on has a coeffi ci ent of one. The scal ed
probl emis
Mi ni mi ze 1イ -4
8.:   イ 壬
(4. 84a)
: (0. 似 88+ 1. 4619x¥)(イ+x内+ヰ) ー 166.5641 (xi - 0 (4. 84b)
あ: 1. 0868x. + 0. 3482 - 28.4641 (x1- 0
C4: X2- XI +0. 001
0. 02 XI 1. 0; 0. 025; x2 1. 0
(4. 84d)
Sec4_5_2_scal ed. m: The code (coul d have becn part of Sec4_5_2. m)
i nvesti gates the scal ed probl emin 出巴 same way 伽 t the ori gi nal probl emwas
i nvesti ga凶 . Pri mari l y,出 is expl oi ts 由巳 i nformati onf romthegraphi cal soI 凶 on. 1t is
expected 出品 t the sol uti on for the desi gn vari ab1es will be the samc al though the
mu1tiplier va1ues areexpected tobe di fferent . Fr omthei nformati onin出 eWorkspace
wi ndowtheopti mal val ues for thedesi gnvari abl es arexi = 0. 6774 and xi = 0. 6445.
AIl of 帥 constrai nts arefeasi bl e. The opti ma1val ues of 白 emu1ti pl i ers have changed.
The問 is no l ooger a zero mul ti pI i er-i t has t umed posi ti ve. There is still one
mu1tiplier wi th a negati ve si gn i mpl yi ng the poi nt has not met 白巴 Kuhn- Tucker
condi ti oDS. Actual l y, it shoul dhave been anti ci patedsi ncechangi ng ofl he sigll of the
mul ti pl i er is not possi bl ewi thscal i ng.
Thereis sti lI thematter of i nconsi stency betw田 n thegraphi cal sol uti onand the
di ssati sfacti onof theKuhn- Tucker condi ti ons. Expl ori ngthis parti cul ar exampl e has
been very torturous but has provi ded白 eopportuni ty toexpl ore i mportant but rel ated
quanti ti es Ii kc scal i ng. Another i mportant consi derati on is 出 e scopc of the
Kuhn- Tucker condi ti ons. These condi ti ons appか onl yat regl l l ar poi nts.
Regul ar Pol nts: Regul ar poi nts [8] arise when eq.uality constrai nts, h(X) , are
P陀 sent 泊 白 eprob1em. 百 l eyar芭 a1soextended toacti vein叫 回 I i tycons住必 nts(pseudo
equa1i tycons位剖 nts). Kuhn- Tuckcr condi ti ons arc val i don1y for regu1ar poi nts. 百 le
t wo csscnti a1features of a regul ar poi nt X. are
• Thcpoi nt X・is feasi bl e(satisfies alI constrai nts).
. Thc gradi ents of the 叫 ua1ity ∞nstr温 ints ぉ wclI as the acti ve in叫 回 lity
constrai nts at X. must f orma l i ne釘 i ndependent set of vectors.
In the sca1ed probl emof Examp1c 4.3 thc constrai nts g I and 83 are acti ve. The
sol uti on xj = 0. 6774 and xi = 0. 6445 is feasi bl e. The gradi ent of g I is Vg. =
[- 37. 631435.454司T. The gradi ent of g2is Vg
= [- 34. 3119 30.4786]T. lt appears
that thet wo gradi cnts arc a1most paral1cl toeach oth巴 r. Thi s is al socvi dent inthe
graphi cal sol uti on. The gradi ents thereforc are not Ii nearl y i ndepcndcnt -
Kuhn- Tuckcr condi ti ons cannot bc appl i ed at this poi nt . Thi s does not mean the
poi nt is not a l ocal mi ni mum. Th巴r巴 is graphi cal evi dence that it is i ndeed so.
Kuhn・Tucker are the on1y avai l abl e f orma1 condi ti ons for recogni ti on of the
opt i mumva1ues. It is uni vcrsa1ly appl i ed wi thout regard to regul ari .か Some
addi ti ona1consi dcrati ons shoul dbc kept inmi nd [8].
If equal i ty constrai nts are present and al1 of 白 e i nequa1ity cons釘 'aints are
i nacti ve, 出 en the poi nts sati sfyi ng 白 c Kuhn-Tucker condi ti ons may be
mi ni mum, maxi mum, or a sadd1epoi nt. Hi ghcr-order condi ti ons areneccssary
toi dcnti fy thc typeof sol uti on.
If the mu1ti pl i ers of the acti ve i nequa1ity constr剖 nts are posi ti ve, the poi nt
cannot be a 10cal maxi mum. It may not be a local mi ni mumeither. A poi nt may
bc a maxi mumif 出巴 mul ti pl i cr of an acti vc in叫 ual i tyconstrai nt is zero.
1. Stcin, S. K. Cal cul us and Anal yti cal Geometry. McGmw- Hi I I , NewYork. (1987)
2. Hostetl er, G. H. , Santi na, M. S. , and Montnl vo, P. 0. , Anal yti cal . Nwneri cal . and
Computati onal Methods f or Sci enceand Engi neeri ng. Prenti ω- Ha1I. Engl ewood CIi飽,
NJ . (1991)
3. Mol cr, C. andCosta. P. J. , Symbol i c Mat h Tool box- f or ωe wi thMATUB- User s Gui de,
MathWorks Inc.. MA.
4. MATLABー Demos Symbol i c Tool Box-I ntroducti on. onl i ne resource in MATLAB.
MathWorks Inc. , MA.
5. Burden, R. L. andFaires, J. 0. , Numeri cal 刷 物 'sis. 4th ed.. PWS_KENT Publ i shi ng
Company, Boston. (1989)
6. Fox, R. L.. Opti mi wti on Methodsf or ElI gi neeri ng Desi gn. Addi son-Wesl ey, Readi ng,
MA. (1971)
7. V飢 derplaats,G. N. , N/l meri cal Opti mi wti on Techl l i ques for ElI gi neeri ng Desi gl l .
McGmw-Hi 11, NewYork. (1984)
8. Arora, J. S. ,l ntroducti ontoOpti maI Desi gn. McG mw- Hi ll , NewYork. (1989)
9. Kuhn, H. W. andTuckcr, A. W. , Nonl ul ear Programmi ng. Proceedi ngs 01theSecond
Berkel ey Symposi umon Mathemati cal Statistics al l d Probabi/i.η, J. Ncyman (cd.),
Uni vcrsi tyof Cal i forni aPrcss, 1951.
(In many of the probl ems be1ow, you are requi red to obtai n the numeri cal
sol uti on.)
4.1 Defi nc two nonl i near functi ons in two vari abl es. Fi nd thcir sol uti on through
contour plots.
4.2 For thc functi ons inProbl em4.1 obtai n the gradi ents of the functi on and UI 巴
J acobi an. Conf i rmt hemusi ngtheSymbol i c Mat h Tool box.
4.3 Def i ne thedesi gnspace (chooses si deconstrai nts) for a two-vari abl eprobl cm.
Dcf i ne t wo non1i near functi ons in t wo vari abl es that do not have a sol uti on
wi thi nthis space. Graphi ca1lyconf i rmth巴 rcsul t .
4.4 Defi ne a desi gn space for a two-vari ab1e probl em. Defi nc t wo nonl i near
functi ons int wo vari abl es 白 紙 haveat least t wo sol uti ons wi thi n白 espacc.
4.5 Def i ne a nonl i near functi onof t wo vari abl es. Choose a contour val ueand draw
thc contour. I dcnti fy a poi nt on thc contour. Cal cul atetheva1uc of thegradi ent
at that poi nt. Dr awthegr温di ent at thepoi nt usi ngthecomputed val ue・Cal cul atc
由巳 Hessi 回 鉱 山 c above poi nt .
4. 6 Usi ng therel nti oninEquati on(4.23), establ i sh伽 t thegradi ent is normal tothe
tangent [aphysi cal i nterpretati onfor Equati on (4.23)].
4.7 Def i ne a nonl i near functi onof 伽 民 vari abl es. Choose a poi nt in the desi gn
space. Fi nd 出 e gradi ent of 出巴 functi on at the poi nt . Ca1cul ate the Hessi an
matri x at thesamc poi nt.
4. 8 Exprcss theTayl or seri es expansi on (quadrati c) of thefuncti onftx) = (2- 3x +
x2) sinx about 由巳 poi nt x = 0.707. Conf i rmyour resul ts throughtheSymbol i c
Mat h Tool box. Pl ot theori gi nal functi onand出 eapproxi mati on.
4.9 Expand thefuncti onftx.y) = 10(1- x2)2+ (y - 2)2quadrati ca1lyabout the ~   n t
(1,1). How wil1youdi spl ay thei nformati on? Dr awthecontours of theori gi na1
functi onand theapproxi mati on.
4. 10 Obtai n thesol uti ongraphi cal l yforthecase when Equati on(4.10) is added to肱
probl emdefi ned by Equati ons (4.36), (4.39), and (4.37). Obtai n出 e sol uti on
usi ng白 eSymbol i c Mat h Tool box.
4. 11 Obtai n白 csol uti ontothc opti mi zati onprobl eminSccti on4.4.4usi ngMATLAB.
4. 12 1nSecti on4.5.1, obtai nthecoeffi ci ents for a qua合 凶 C fit to出 eprobl em.
4. 13 Use an exampl e toshow白 紙 opt i mumval ues for thedesi gnvari abl es do not
cb甜 ge江 theobj ecti vefuncti onis mul ti pl i edby a cons凶t . Prove thesame if a
constant is added to出 efuncti on.
4. 14 How does scaI i ngof theobj ecti vefuncti onaffcct theval uc of 山 emul ti pl i ers?
Usc ancxampl c to i nfer the陀 sul t. How does scal i ngtheconstrai nt affect the
mul ti pl i ぽ s ? Check wi than exampl e.
τ'he one-di mensi onal unconstrai ncd opti mi zati on probl emis often i ntroduced 阻 d
sol vednumeri cal l y in most courses 山 at deaI wi th numeri cal anal ysi sJ techni ques. It
appears i ndi rectl y, notabl y as fi ndi ngthe root of a nonl i near equati on, whi ch is 山 e
same as satis今 i ng山 eFOC for an unconstrai nedopti mi zati onprobl em. Two of the
pop叫 ar techni ques are白 eNewton- Raphson and出 ebi sccti on techni que.τ 'hey 釘 E
i ntroduced in Secti on 5.2 to provi de a compari son wi th the methods used in
opti mi zati on. In engi neeri ng dcsi gn 山 e si ngl e-vari abl e opti mi zati on probl emis
prob曲, l ya 回 viality,but one-di mensi onal opti mi zati on is a criticaI component of
m凶 ti vari abl e desi 伊 and is di scussed in S回 凶 on 5.3. Meanwhi l e the cI assi cal root
fi ndi ng methods i denti fi ed above do not pl ay any si gni fi cant part in numeri cal
opti mi zati on techni ques as they are consi dered computati onal l y expensi ve and not
robust enough over a l argecI ass of probl ems. An i mportant consi dcrati oninnonIi near
desi gnopti mi zati onisthat duri ngtheinitial i terati ons, when you aretypi cal l yfar away
合 o m白 e sol uti on, accuracy c皿 bc 位置 ded for speed. In these i terati ons it is al so
i mportant tomove throughto山 cncxl itcration, i nstcadof fal teri ngat 白 ecurrent one.
These.criteriahave陀 qui 悶 dthat thcone. di mensi onal techni ques be si mpl e inconcept
as wel l as easi l y i mpl emcntabl e. Two di fferent techni ques, the pol ynomi al
approx凶 l ati onand曲 egol densecti on, provi de popul ar support for most opti mi zati on
softw訂 e. Very oftenthet wo techni ques are i mpl emented incombi nati on. They are
di scussedinSecti on5.2.
In order to connect wi ththe di scussi ons later, theone-di mensi onal vari abl e will be
i denti fi ednsαi nstead of X or XI ・ Inthefol1owi ng chapters 血 is is a1soreferredtons
theonc-di mensi onal stepsi zecomputati on.
Exampl e 5.1 This exampl e is si mi l ar toone inChapter 4:
Mi ni mi ze f(α) =( α_ 1)2(α- 2) ( α- 3)
Subj ectto: 0 S α S 4
Theprobl emdoesnotrepresentanyreal desi gn. It wasconstructedtohavemul ti pl e
l ocal mi ni mums wi thi n the area of i nterest. Expressi on (5.1) reprcsents an
unconstrni nedprobl em. Thesi deconstrai ntsaccompanyal1probl cmsi nthi sbookto
con vey thei dea that therearenotru Iyu nconstrai ned prob l ems. Si deconstr ai ntsal so
defi ne an acceptabl e desi gnregi on for all probl ems. Secti on5.2will expl ore this
prob l emthroughthcvar i ousmet hodsmen ti onedcar I i er .
5. 1. 1 Const r al ned One- Dl mensl onal Pr obl em
The onl y that Exampl e 5. 1 , a si ngl e-vari abl eprobl em, cana, ∞ommodat e is
i nequal i ηconstrai nt . Fr omthc previ ous chaptcr there can be mo隠 白 血 onc such
cons回 int . A si ngl e-vari abl eprobl emcnnnot have如 何 ual i tyconstrai nt (Chapter 4).
Ina constrni n巴d mul ti vari abl eopti mi zati onprobl emit will oftenbe necessary l nd
白 estepsi zeαsuch白 紙 山 econs位置 int j ust becomes acti ve. It is possi bl etodef me出巴
probl em.
Exampl e 5. 1a
Mi ni mi ze f(α. )=(α ー 1)2(α- 2) ( α- 3)
Subj ect to: g(α): 0. 75α2- 1. 5α ー I S0
OS αS 4 (5.2)
In actual appl i cati ons it is mor e I i kel y 出 at cxpressi ons (5.1)ー(5.3) establ i sh t wo
probl ems that provi de di fferent sol uti ons to出 c stepsi zcα. The first is 白 esol uti onto
expressi ons (5. 1) and (5.2) , whi ch is the samc as Exnmpl c 5. 1. The second is the
sol uti onto
g(α. ): 0. 75α2- 1. 5α ー 1=0
OS α S 4
The second sol uti onisuseful onl y if expressi on(5.3) isan acti veconstrai nt Otherwi se
thesol uti ontoExampl e 5.1wil1prevai 1.百 lefinal sol uti onchosen ensu陀 sexpressi ons
(5.1)ー (5.3) aresatisfied. A graphi cal i nvesti gati onin山 enext secti onshoul dillustrate
this featurec1earl y.
The sol uti ontoExampl es 5.1佃 d5. 1ai s exami nedgr司 phi caIIy. Next thec1nssi cal root
fi ndi ngtec加 i qucs of Newt on- Raphson and山 ebi secti onmet hod arepresented. These
M芭 fol l owedby thepol ynomi al approxi mati on and山 egol den secti onmcthod.
5.2.1 Graphl cal Sol ut l on
The exampl e has al ready made its appearance inChapter 4. Fi gu閃 5.1 is a pl ot of the
obj ecti vefuncti oninExampl e 5. 1. It al so i ncl udes 出 c sol uti onωExampl e 5.1a. If
expressi on (5.3) represents an i ncqual i ty constrai nt , the feasi bl e regi on for one-
vari abl e functi ons is qui tedi fferent f romthe two-vrui abl e probl ems encountered in

" , '




constral nedmi ni mum

、 .
絡 砂 舟  
H J " ' l ι g l o b a l
unconstrai nedmi ni mum
;.. - -

3.5 0.5 1. 6
円 gure5.1 Graphi cal solulion: Exampl e 5.1.
::tz;; ロ: ; 立とむねなお品 321i222211出;宮
探 知 1 F Z Z ; i t mt 品 川 双 山 口
Exampl e5. 1
α事=2.6484; f * =ー 0. 6195
Exampl e 5.1a
α* = 2.5275; f. = - 0. 5815
5. 2. 2 Newt on- Raphson Techni que
The Ncwton- Raphson techni que, alsorcferrな dtoasNcwton' s techni que, isa gradi ent-
bascd 剣 山 n to fi ndi ng 山 c root of a si ngl c nonl i near eq剛 on. Root fi ndi ng
lおおむ lニi t U 1 2 2 Z Z ; : : r h刷 出 eequati 川 お aぷ。 ;:;L
中( α) =0
wher官ゆ (α) is a nonl i near equati oninthevari abl eα.
百 letechni quehas addi ti onal featurcs:
・ It hasag回 metri cunde甲 i nni ng.
• It usωt he Tayl or series expanded I i nearI y.
・It is iterative.
・ It has thepropertyof quadrati c convergence.
Any iterativc tcchni queaddrcssi ngEquati on(5.5) canbesummari zed as
Generi c Al gori tl rm(A5.1)
Step 1. Assum巴 α
Step2. Cal cul ate.1α
Step3. Update a =α +.1α
If converged( 中 (δ) =:0) 出 enexit
If not converged(ゆ(匂 :;f: 0) 白 enα ←a
go toStep2
iI ω
10bedownl oadedf wmthcwebsile町 宮 indicaledbyboldfnccsnnsserif Iype.
The above sequen田 is fairly standard in iterative techni ques for many cl asses of
probl ems. It al socapl ures theessencefor most of thetechni ques tofol l owinthisbook.
Sl ep 1 i ndi cates thepri mary f i伺 ωrc of iterativeme白 ods- t hestartingsolution. This
is an initial guess provi ded by the user 10 slart the iterative process. The iterative
process is conti nued10thesol uti onby cal cul ati ngchanges inthevari abl e(.1α.). The
val ueof thevari abl eis updated. Convergence ischecked. If Equati on(5.5) is satisfied,
thenthe sol uti on has been found. If convergence is not achi eved, 出 叩 the updated
val ue provi des the val ue to proceed wi th 出巴 next iteration. Of course, .1αcan be
cal cul atedinmany ways and Newton- Raphson isjust one of theways ωestabl i shthe
I change inthevariable.
費l j
Cal cul atl on 01 Aα ( Newt on- Raphson) : Let α b e the current val ue of the
variable. It is assumed that Equati on(5.5) is not satisfiedat 出 is val ueof thevari abl e
(出 e陀 asonfor us toiterate). Let a be a nei ghbori ngval ue. Ideal l y we woul d like10
achi eveconvergence at this val u巴 (even 邸 we are aware it rni ght take us several
itcrations 10achi evc converg巴 nce). Usi ng Tayl or' s theoremexpanded tothe linear
terrnonl y,
ゆ(α) = ct(α+似) =中 ( α) +25Aα=0
Aα=一魁旦 1=- ' 坐 I cb( o.)
坐 Id
l α│τ
For Equati on(5.7) tobc effccti ve,山 egradi ent of thefuncti onshoul dnot be zcro. lt
is a1soi nevi tabl e出 at changes i nαwi l l be l argewhere ct is flat, and smal 1where the
sl opeis large. For theNewton- Raphson method tobe effccti vetheiterations shoul d
avoi d問 gi ons where thesl opeis smal l . Thi s is a scri ous di sadvantageof 出 is mcthod.
Fi gure5.2ilI ustrates thegeometri cal constructi onof tbetechni quc at thepoi nt α = 3.
Exampl e 5.1 The Newton- Raphson tcchni queis appl i edtothefirst-order condi ti ons
( FOC) of Exampl e 5. 1.
Mi ni mi zc f (α)=(α ー1)2(α ー 2) ( α- 3) (5.1)
Subj ect to: 0 SαS4 (5.2)
The FOCi s
中 刷 { ω叫 =去=判均α ト一 l 州 - 2 引吻州 2均伽釧)い州彬ベ(似 ωα -3) 司)+(似α トい川一→1り) 向州勺恥 3 勾)+(似α 十日一→lり) 州 αrμ-2 十 2

- 2
0 0.5 1.5 2 2.5 3
Fl gure5.2 Devel opment 01Newton. Raphsontechni que.
3.5 4
ηJ e Technl que: Two i terati ons areshown here. The compl ete appl i cati onis done
through MA11AB codesecs- 2- 2. m. The code uses symbol i c cal cul ati on.I nr umi no
悦 codethefi gurewi ndowdi spl ays印 刷 rnati onof theiterati 削 ravel ω 曲 目 仰 60;
Al so shown is the change in 6αwi t h each iteration. The iterative resul ts are
summari zcd ina tabl e8t 山 ecnd. The ca1cul ati ons requi rc thegradi ent of 中(α):
ゆ' ( α) =2市 ( α- 2) ポ ( α- 3) +4*(α ー 1)*(α ー 3) + 4*(α ー 1) *( α- 2) +2*(α ー 1)^2
Iterati on 1:
α =3; f (α) =0. 0; ゆ(α) = 4 I t' ( α) =16 6α=- 0.25
I t ( α+L1α.) = 0. 875- not converged, at least another i terati onneccssary
Iterati on2:
α =2. 75; f(1α. )=-0. 5742; 中(α. )=0. 0875; 帆 α. )=9. 25; 却 = ー O. ω46
。( α+6α) =0. 1040ー not converged, at least anothcr i terati onnccessary
τ'he code Sec5_2_2. mr珂 ui res fi ve i terati ons to converge the va1ue of I t to
1. 0E心 8(consi deredzerohere). T悩 svaI ue of 1. 0E・08is termed theconvergence (or
stoppi ng) criterion. Thc acccl erati on of convergence or rate of convergence to thc
sol uti onis formal l y defi ncdinmany di fferent ways. One of thc ways is tomoni tor the
ratio of ゆ( α+6α )1中 (α). Anothcr i sωt r ack 6α. ln appl yi ng and understandi ng
numeri caI techni ques it is i mportant todevel op an i nsti nct and feel for thenumbers
correspondi ngtovari ous functi ons 脳 血 ei terati ons progress. The speedor 白 equal i ty
of convergence can be gauged by 出 e shift in the dcci maI poi nt in the quanti ty of
interest through the iterations. Consi der the vaI ue of I t through the fi ve i terati ons
[4. 0000,0. 8750,0.1040,0. 0023,0. 0000]. Defi ni tel y白 echangcs in中are nonl i near.
The Newt on- Raphson techni quecan be di recUy appl i edto Equati on(5. 3)-after
aI l themeUl od is desi gnedtooperateon theseki nds of probl ems. It is left as anexerci se
for 山巴 r回 der. 百 lere is a very i mportant feature of most itcrative techni ques that
exp!ore m叫 tiple sol uti ons- as is 白 e case of Exampl e 5.1. Rerun the code
Sec5_Zι 2. mwi tha starti ngvaI ue of 0.5 for a.. After six i terati ons,αconverges to
出 evaI ue of 1.0, whi ch is a l ocaI mi ni mum. Wherewoul d i terati ons l eadif thestart
vaI ue is 1.5 ? These answers are 回 sy to s田 graphi caI ly in a one- or maybe a
two-vari abl eprobl em. Ina typi caI engi neeri ngdesi gnprobl em, si gni fi cant experi ence
and i ntui ti on are requi red to recogni ze Ul esc si tuati ons when 出 cy occur. A heal thy
S出 pi ci onof thesol uti on must aI ways be intheback of one' s mi nd when expl ori ng
mul ti vari abl e probl ems. In summar y, most iterative numeri cal techni ques are
desi gnedtoconverge tosol uti o,1Sthat are c/ ose 10 where theystartfrom.
5. 2. 3 Bi sect l on Techni que
This is another popul ar tcchni quetofi nd曲 目 ∞tof a functi on. It is al socaI l eda bi nary
search or i ntervaI haI vi ng procedure. Unl i ke the Newt on- Raphson met hod this
procedure does not 民 qui retheevaI uati onof 白 egradi ent of 曲 efuncti onwhose root
is bei ngsought. One of thereasons themet hod isbei ng i ncl udedis tocompare it wi th
thegol den sccti onmethod di scussedlatcr. The numeri cal techni que in山 is method is
basedonthe問 お oni ng白 紙 therl

t(or zero) of a functi onis trappedor bound between
a posi ti veand a negati vevaI ue of thefuncti on. The actuaI sol uti onachi evedby 血 E
method is an i ntcrval inwhi ch出 ezero or thesol uti oncan be l ocated. Thc finaI i nterval
is typi 回l I y dctcrmi ned tobe a vcry smalI val ue, cal l cdthc tol erance, of theordcr of
l OE-08. S泊 ce 出 is is a root fi ndi ng procedure, when appl i ed to opti mi zati on it is
appl i cdto恥 gradi ent of 批 ot訪問 tive whi ch we i denti fi cdas ゆ(α.) Equ副 on(5.5).
Establ i shi ngthemi ni mumof theobj ecti veis thereby transl atcdtol ocati ng曲 erootof
出 e gradi ent functi on. It is possi bl c to devcl op a bi secti on tcchni que for di rectl y
handl i ngthemi ni mi zati onprobl em.
The met hod r珂 ui res t wo poi nts to start , sayαn and αb・The vaI ue of It at 由 民 E
poi nts must be opposi te in si gn. τ 'hesc can be the si dc constrai nts of the
one-di mensi onaI opti mi zati onprobl em. lt is assumed白 紙 at Icast one sol uti onexi sts
in白 is initi 凶 interval. Duri ng each i terati on,出 is i nterva1is haI ved wi th出 emi dpoi nt
of thei nterval repl aci ngci ther 叫 or αb,whi l e kccpi ng thc root still trappcdbctween
amcti on val ues of opposi te si gn. The i t emt i veMmi que isexpressedas thefol l owi ng
aI gori thm(aset of procedures 白 紙 紅 erepeated).
Al gori tI J m!or Bi secti onMetl l od (AS. 2)
Step 1: Choose 叫 and叫 tostart . Let叫 〈 叫
Step2: Set α =叫 + ( 叫 ー 叫 )12
Step3: If 中(α.) = 0. 0- Converged Sol uti on- exi t
日 se If ( αb- αn) <l OE・04- tol erancemet - exit
El se Ifゆ (α.)* 中( 叫 )>0. 0; 曲 en叫 ← α
El seαb←α
go toStep2
We consi der Exampl e 5.1 whi ch is reproduced here
Mi ni mi ze f{α) =( α_ 1)2(α- 2) ( α- 3)
Subj ect to: 0 Sα 云 4
ThcFOCi s
中仰(何仲 α
S関 5_え . 3. m: This 地 孔 AB code runs the bi secti on met hod on 臥 制 pl e 5. 1. It
iIl ustrates thetrappi ngof 山 emul l rnumby thet wo val ues on ei ther si deOf themot.
百 le陀 町 宮 17 i terati ons for a tol erance of 1. 0 E・04 whi ch termi nated thc program.
Compa削 hi s wi th5 itcrati 叩 sfor theNewt on- Raphson 1削 hod. for a bettcr sol uti on.
U 巴 l ower number of i飽 ratJ ons c佃 be油 凶 to 恥 quaJ ity of ir由 ' nnati on山 edto
upd脱 出 e d悶 伊 vari abl e changcτnc gradi ent-bascd i nformati on uscd in thc
Newton- RaPI EBon tEChmque ls mor e sophi sti cated. though出 cgradi ent computati on is
an addi ti onaJ work l oad.
T K probl emdescri bedby Equati on ( 53) cmbe handl ed di rectl yby thEbi secti on
脱 出 od山 町 (α.) is 恥 same箇中 (α). Ina wa 町Y. t 出h附e
o叩pp 卯o削 utザY tωo 陀 附 rev1 判 1路凶附 s討i比t pri or knowl cdge. parti 叫 arl y wi th resp附 ω defi ni ng and
剛 sl ati nganal gori 伽 i ntoI UD出ngcode. There is u 刷 su拘 ast凶ron 昭gc ∞orre 附 s叩po 叩 nd 伽l
betwecn 自 estcp-by-stcpd ω cri pti onof 出 e numeri caJ techni que (t山heal gori t白hr町同E
白山e c ∞od 也e 白伽aωt wi l l i mpl ement 出 e 蜘 ps ω effect a sol uti on (for exampl e.
:22tit-B;日 EP;521111:ZZ220: よ22LtJbreば
fol l owl ngt wo methods areused extensi vel y f oronedi mensi onal mi 凶 mi zati on. They
handl c themi ni mi zati onprobl emdirectly.
5.2.4 Pol ynoml al Appr oxl mat l on
The me白 od is si mpl c inconcept. I nsteadof mi ni mi zi ng a difficult functi onof one
vari abl e, mi ni mi ze a pol ynomi al 出 at approxi mates thefuncti on. Thc opti mal vaI ueof
曲 e vari abl 巴 白 紙 mi ni mi zes the pol ynomi al is thcn consi dcred to approxi matc the
opti mal vaJ uc of 白 evari abl efor t1J e ori gi naI functi on. It is rarefor t1J e dcgree of t1J e
approxi mati ng pol ynomi aJ to excecd threc. A quadrati c approxi mati on is standard
叩 less 曲 e曲 irddegree is w釘 ranted. It is c1ear that scri ous errors inapproxi mati onare
expectedif 出 epol ynomi al is tosi mul atethebehavi or of 出 eori gi nal functi onovcr a
l釘 .ge range of vaI ues of 山 vari abl e. Mathemati cal t1J eorems exist 白紙 justify a
quadrati c representati onof t1J e functi on, wi tha prescri beddegree of error, wi tIJ ina
smaJ l nei ghborhood of t1J e mi ni mum. What 山 is ensures is 出 at t1J e pol ynomi al
approxi mati on gets better as 白 e mi ni mumis bei ng approach叫 . Exampl e 5.1.
repf吋 ucedfor conveni epce, is used for iI1ustrati onof theprocedure.
Exampl e5. 1
Mi ni mi ze f(α. ) =( α_ 1)2(α- 2)(α ー 3) (5.1)
Subj cct to: 0 S α S 4 (5.2)
A quadrati c pol ynomi aJ P(α) is used for the approxi mati on. This pol ynomi al is
P(α) =bo+bρ +b
Two el cments n田 d to 恥 understood pri or to t1J e fol1owi ng di scussi on. 百 le first
conccms 白 eeval uati onof t1J e pol ynomi 叫. and t1J e second concerns 出 e i ncl usi onof
exp陀 ssi on (5.2) in 出 e di scussi on. Thc pol ynomi aJ is compl etel y defi ned if the
coeffi ci ents b
• b. , and b
arc known. To determi ne恥 m. t1J reedatapoi nts [(α1' /.),
(α .2' !z). (α.3,13)] arc generatedf romEquati on (5.1). Thi s sets up a I i ncar systemofthree
equati ons in 尚 昆 e unknowns by 問 qui ri ng 白紙t 1J e vaJ ucs of t1J e functi on and the
pol ynomi aJ must be the same at t1J e t1J rce poi nts. The sol uti on to t1J is systemof
equati ons is 山 val ues of t1J c coeffi ci ents. 百 1巴 consi derati on of expressi on (5.2)
depends on t1J e typeof one- di mensi onaJ probl cmbei ngsol ved. If theone-di mcnsi onaI
probl emis a genui ne si ngl e-vari abl edesi gnprobl em,出 enexpressi on(5.2) needs to
be present. If 出 e one- di mensi onaI probl em is a subprobl em 台。 m 山 e
muJ ti di mensi onal opti mi zati onprobl em, t1J en expressi on(5.2) is not avai l abl e. Inthat
cωe a scanni ng proccdure is used todefi n巴 α1,的, andα3・
Scannl ng Procedure: 百 lepr民 間 is startedf rom恥 l ower I i mi t f orα. A vaJ ue
of zero for 出 is vaJ ue叩 bej usti fi edsi nceit refers tovaJ ues 鉱 山 current iteration.
A constant i ntcrval for α,6. αi s al soi denti fi ed. For a wel1-scal edprobl emt1J is vaI ue
is usual1y 1. Starti ngat t1J e l ower I i mi t,曲。 i ntcrvaI is doubl ed until tlJree poi nts are

, .



detenni ned such 出 at the mi ni mumis bracketed between them. Wi 白 respect to
Exampl e 5. 1 , thescanni ngprocedure generat回 thefol l owi ngval ues:
α¥=0; /¥=6 α = 0; j{O) = 6;
同 =1; ん =0 α = 1; j {l ) = 0;
this cannot be円 ωt hemi ni mumis n01yet 回 pped α = 2; j (2) = 0;


/2= 18
A process suchas that illustrat巴 d is essenti al sSit is bothi ndi fferent to theprobl em
bei ngsol vedand can be programmed伺 sily. The i mportant requi rement of any such
proc凶 s is to ensu白 山 at the mi ni mumlies between 出 e Ii mi ts establ i shed by the
procedure. 百 lis procedure is devel oped as MATI . ABm- fi) e Upper Bound_1Var . m.
The set of I i near equati ons to estabIi sh the coeffi ci ents of the pol ynomi al in
Equati on(5.9) is
6 = b
+ b¥(O) + b
(0) (5. 108)
円 =4; α = 4; j(4) = 18;
(5. l Ob) 0::: b
+ b¥(l) + b
4 3.5 3
0.5 。
(5. l Oc) 18= b
+ b¥(4) + b
pol ynomi al appro必ma1ion.
∞de devel opment is termed a bl ock structuredapproach- I n this way- l arg巴 code
devel opment is possi bl eusmgsmal l erpi eces ofexi sti ngcode.
Rgure5. 3
Usi ng MATI . AB出 esoh瓜 onto白 eequati ons is
ho=6. 0; b¥=- 9; b
百 ICapproxi mate sol uti ontoEquati on(5.1) is 1.5. Fi gu陀 5.3dωcri besthepol ynomi al
approxi mati onprocedure. lt is cl ear 出 at theapproxi mati on l eaves much tobe desi red.
On theother hand, if thedatnfor thepol ynomi al coeffi ci ents were around a smal l er
regi onnear α=2. 6, then出巴 results woul d be mor e i mpressi ve. Note出 at ra山 町 出 m
usc a hi gher-order pol ynomi al , a better set of datais p児 島 町 吋 .
The scanni ngprocedure and出 epol ynomi al approxi mati on techni ques wi l l fi nd
si gni fi cant use expl ori ngmul ti di mensi onal opti mi zati onprobl ems. The s位置 tegy for
devcl opi ngMATLAB codc f romnow on must i ncl ude出 epossi bi l i tyof code reuse. 百 le
same code segments can be used inmany di 仔 erent numeri cal techni ques. Such generi c
upper Bound1Var . m: Thi s codcsegment i mpl ements 山 e detcnni nati on of 山 e
upper boundoi l hE凶 onof one vari abl e. The i nput tothefuncti onis the附 ncof t hc
器問: 思議議部先制抗告。部
c o t Z詑 Z3 ; 4 2 ; ; ; 2 2 d E mu s t h a M山 Bm- fi)e,in白 esame di rectory
tZTftU3231:;:;Z?5512-お お む twhi ch thc functi on
P(α. }=6- 9α+3α2
一一 =- 9+6α=0; α: =α ・=1. 5
0 α r
and thepol ynomi al is
The mi ni mumfor thepol ynomi al is thesol uti onto
cod巴 segment i mpl ements 白 e pol ynomi al
functi on of one vari abl 巴・ Thi s functi on uscs
upper Bound_1Var ( ' EXam
l e5_1' ,0,1,10)
Pol yAppr ox_1 Var. m: 羽 lis
approxi mati on method for a
Upper Bound_1 Var . mto detenni ne the range of 出 e variable. The i nput to 白 e
functi on is the name of the functi on; the order (2 or 3) of the approxi mati on;
l owbound- t he start val uc of the scan passed to Upper Bound_1Var . m;
intvlstepー 恥 scanni ng interval passed to Upper Bound_1 Var. m; in町 ials一 郎
number of scanni ng steps passed to Upper Bound_1Var . m. The output of 出 e
programis a vector of two values. The first el ement of thevector is thel ocati onof the
mi ni mumof theapproxi mati ngpol ynomi al , andthesecondis thefuncti onval ueat
this location.
The functi on陀 ferencedby出 ecode must be a MA TI . AB m・file,inthesame di rectory
( Examp抱 5_1. m). 百 1e i nput for Exampl e5_1 is the val ue at whi ch the functi on
needs tobecomputed. and its output is theval ueof thefunction.
Usagc: Val ue Pol yAppr ox_1Var ( ' Exampl e5_1' ,2,0,1,10)(5.13)
The two programs us巴 出 eswi tch/case and the/eval statements fromMATLAB. The
code alsoilI us町 ates callingandretumi ngf romother 釦 nctions.
5.2.5 Gol den Secti on Met hod
Th巴 methodof gol densecti onis thccrcamof thefami l yof interval 問 duci ngmethods
(for exampl e. 出 ebi secti onmethod). It reduces 出 einterval by thc same fractionwi th
eachiteration. The intervals 釘 ederi vedf romthegol densecl i onratio. 1. 61803. Thi s
ratiohas si gni fi cance泊 a巴 sthetics as wel l as mathemati cs [3,4]. The method issi mpl e
toi mpl ement . It is indifferent to出 eshapeand conti nui ty properti es of 血 efuncti on
bei ngmi ni mi zed. Most i mportant , Lhe number of iterations to achi evea presαi bed
tol erancecanbc establ i shedbeforeLhe iterations start.
Al gori thmf or Gol den Secti ol l Met hod (AS. 3)
Step1. Choose αl owα 叩,
τ = 0. 38197(fromGol den Rati o) ,
e = tol erance= (L¥α)Onal /(αup_αl ow)
N = number of iterations = -2. 078lne
Sωp2. α1 =( 1 -τ)α l ow+τα 叩 ;fl =/ (α1)
向 =τ ri
+ (l ー τ)α ,U
;f2=爪 句 )
τnc poi nts areeq凶 distant frombounds
Step3. If ( i < N)
I f/I >/ 2
α10" ←α. ; α1←α2;fi ←12
~   τ α , I o w + (1-τ)α 叫ん=/ ( 向)
i ←i + 1
GoTo Step3
I f h>/ .
Il ' J
αup←α/:2;向← α1;12←A
α1= (1- 't)a!ow+τα ゆ ;/1=/ (α. )
; r i + 1
GoTo Step3
s i 出 iiZ昔話器鴻! 日
onl ycal ふlateふ evalues. The al go仙 mis i mpl emented ina different file.
Exampl eS. l
Mi ni mi ze / ( α) =( α_ 1)2(α- 2)(α- 3)
Gol den Secti onMet hod
St叩 1. (1.10" = o. o;l ow= 6;α u
=4; !"' =18
L¥<Xj!nol = 0.001; E = (0.00114); N = 17
Step2. α. = 1. 5279;/. = 0.1937; 向 =2.4721; 12=ー0. 5401
F1gure5・4 i1l ustrates th巴 l ayout of i nformati onat this stage.It canbCVEri nd 山 at
α. . α2arel ocatedsymm巴町 icallywi thmspect toouter bounds dO嶋田 ,"p.
Step3. i = 1
/ 1>12
( V= 1. 5279; l ow= 0.1937; 叫 =2.4721;/. = - 0.5401
α2 = 3.0557;12= 0. 2486


i =2
Step3. conti nued. . .
a. up = 3.0557; 戸 =0. 2486; α2= 2.4721; 五=- 0.5401
α1= 2.1115;12=ー0. 1224
i =3
Go To Step3 etc
IneachitcrationLhereisonl y one制 ofcal 叩lationtoeva山a胸中 evari abl eand伽
val ueofthe functi on. Therest of thEComputauonts masSI gnsngextsti ngi nformatIon.
The differcn回 凶 臨 ou町 bomdsi sal sochangi ngwi theachkmuon- hExmpl eal
Start: L¥α=4. 0
Iteration1: L¥α = 4.0- 1. 5279= 2.4721
vector of four pairs of vari abl eand functi onval ues after the final iteration. Wi th a
smal l tol erance剖 yone of l hesepairs provi des 出巴 mi ni muml ocati onof thefunction.
It is recommended山 at the secondpair be accessedfor this pu中 ose. 百 1e functi on
referenced by the code must be a MATLAB m-fi l e, in the same di 悶 ctory
(Exampl e5_1. m). The i npul for Exampl e5_1. mis theval ueal whi chthefuncti on
needs tobe computed, andits output is theval ueof thefuncti on.
U阻 .ge:
Val ue Gol dSect i on_1Var (' Exampl eS_1' ,0. 001,0,1,10) (5.14)
There areno newcommands. Thc organi zi nganddi spl ayi ngof i nformati oninthe
Command wi ndowis worthnoti ng. If this code is part of a larger program, 出 叩 the
pri nti ngof i nformati oninthecommand wi ndowneeds tobe swi tchedoff. It c叩 be
加 medon for debuggi ng.
Compar / son wl th Pol ynom/ al Approxl mat l on: The two popul ar methods for
mi ni ouzi ng a functi on of one vari abl e are pol ynomi al approxi mati on or gol den
section. The al gori l hms for pol ynooual approxi mati onand由巳 gol densectionhave
significant differences. The former is a one-shot approach and is accompani ed by
significant error intheesti mati onof themi ni mum, whi ch will i mprove as thedatafor
approxi mati onget better. I mpliedinthei mpl ementati onof thepol ynomi al approach
is theconti nui tyof 白 efuncti on. The gol densecti onmethod, on theother hand, is 佃
iterativetechni que. The number of iterations depends on出 etol eranc巴 expectedin血 e
final 隠 sult and isknown prior to出巴 start of thei terati ons-asignifi 伺 nt i mprovement
relative to theNewl on- Raphson method where山巴 number of iterations cannot be
predi cteda priori. The i mpl ementati onissi mpl eandtheresults i mpressi veasit isable
t ohomei non曲巴 oun回 1um. It is indifferent tothenatureof thefunction.
There is no reasonwhy thetwo cannot be combi ned. τbe gol densecti oncanbe
usedtoestablishfour datapoi nts wi tha reasonabl etol erance(i nsteadof a l owvalue)
anda cubi c pol ynooual canbe fit 10identifytheapproxi mate locationof mi ni mum.
百 1巴 one-di mensi onal subprobl emin a mul ti vari abl e opti mi zati on probl emis
empl oyed for 白 edetermi nati onof the stepsi zeafter thc searchdi recti onhas been
identified. It is easier 10undcrstandthesel erms by recogni zi ngthegeneri c al go仙 m
for unconstrai n吋 opti ouzati on. In order to focus the di scussi on, rather than 白 e
general obj ccti vefuncti onrepresentati on, a specific one, Exampl e 5.2, is i ntroduced.
f(xl ・X
' X
) = (xl ー ら )2+ { ~ - x
)2+ 3(x
- 1)2
Exampl e5. 2
Mi ni mi ze
' t
' J

4 3
Gol den5ectlon- startingvaluas.
dα= 3. 0557- 15279 = 1. 5278
Flgur由 5.4


The final iterationis reproducedby runni ngthefol l owi ngcode:
i terati on17
2. 6395
-0. 6197
2. 6406
-0. 6197
2. 6402
- 0. 6197
2. 6399
- 0.6197
百 lel ol eranceα2一 αI inthefinal iterationis about 0.001as expecl ed.
Gol dSecti on_1Var. m: The codc lr叩 slates 白 eal gori thmfor the gol den s田 tion
m白 od( A5.3) in川弘 TLABcode. The i nput 10thefuncti onis出 ename of thcfuncti on
俳 t nCtname) whose mi ni mumisbei ngsought; 曲 etol crance(tol) of theapproxi mati on;
the 平蹴 val ue (仰l ow 叫bo 側 und) of 白恥e s 釘c叩 p仰a鎚 蹴e叫d tωo Up 仰p仰er市Bo 叫 und_1Va 町 r口r. m 宵 m舵 官 n仕1; 出肱

s鈴ca朗nmn略gi加nt飽erv叫 ( u ωt仰 1) paお55鉛e伐吋dtωoUp 仰per Bou 叩md_1Va 町 r口' .m;t白hem 聞 u 叩m帥E
蹴 p戸sρ仰仰11巾 旬 p蹴 叫 to Upper Bound_1 V町 .m. The output of 恥 programis ;
A c山 富 orygl ance at Exampl e 5.2i ndi cates 曲 atthesol uti oni satxl = l ; x2=I ; X3=1;
and themi ni mumval ueoffi s O. Threevari abl es arechosen because we ar芭 notgoi ng
touse any graphi cs for illustration(i t is essenti al tofol l owthevector al gebra).
Generl c Al gorl t hm(A5. 4): The generi c al gori thmis an iterativeone and is al so
refer聡 dt oωasearchal gori thm. as thei terati ons tak巴 pl aceby movi ng al ong a search
direction. These searchdi recti ons can be determi ned inseveral ways (Chapter 6). The
al gori thmwi thout any convergencel stoppi ngcriteriacanbe expressedas
Step 1. Choose X
Step2. For each iterationi
Determi ne searchdi recti onSJ vector
Step3. Cal cul atesX; =αj Sj
Note: sX; is nowa functi onof thescal ar clj as Sj is known f romStep2
clj is cal l edthestepsizeas it establ i shes thel engthof s Xj
Xj φ 1 =X/ +s X
的 is determi ned by Mi ni mi zi ng F(Xi +l)
百I i s is referredtoas one- di m巴 nsi onal stepsizecomputati on
i ←i + 1; 00 To Step2
Appl i cal i on0/ Generi c A駒 市 hm( A5. 4) ωEXI l I Tlpl e5.2
… ~   } 印 刷









w M

且 e




ι ﹄



















f ( X1) = f(α1) = (0- 0)2+ 2(0- 6α1)2+ 3(6α1一 1)2= 72αl +3( 6αJ - 1)2
Mi ni mi zi ng j{α1) , 出 c val ue of α1 = 0. 1. Thi s is a one-di mensi onal
opti mi zati onprobl em.
Whi l e thesol uti onabove was obtai nedanal yti cal l ybecause it was a si mpl e l i near
exp陀 ssi on,出 epol ynomi al approxi mati onor thegol den secti onmet hod coul d have
been used inst巴 ad.
Xl =l L │…
and soon.
h 山 is secti on addi ti onal exampl es of si ngl e-vari abl e opti mi zati on probl ems arc
expl ored. The avai l abl ecode for thepol ynomi al approxi mati on and gol densecti on
shoul dhel psol veany and all si ngl e-vari abl eopti mi zati onprobl ems. This is l argel y
left tothereader toexpl oi t . Inthis secti ononl y extensi ons, modi fi cati ons, or creati ve
appIi cati ons of the one-di mensi onal opti mi zati on probl emar芭 consi dered. First,
Exampl e 5.2is revi si tedtomodi f y thegol den secti onmet hod so出 at it canbe used to
四 l cul ate由巳 S旬 psi zeinmul ti di mensi onal probl ems. Exampl e 5.3is a sol uti ontothe
Bl assi us probl em. 百 lis is an exampl e of the exact sol uti on to 白 e Navi er・Stokes
珂 uati on for fl owover a flat platc. Exampl e 5.4is an exami nati on of thei nequal i ty
constrai nt in Exampl c 5.l a by the gol den secti on met hod so that equal i ty and
i nequal i tyconstrai nts can be handl ed.
5.4.1 Exampl e 5. 2- l I l ustratl on of Gener al Gol den Sect l on Met hod
τ'he exampl e f rom血 eprevi ous secti onis vi si tedagai ntomodi f y 出 egol densecti on
method to d巴 tcrmi ne 白 e stepsi ze for mul ti di mensi onal probl em. 百 is book is al so
about getti ngMATLAB towor k for you. Ina scnse出 ssubsecti onis l argel yanexerci se
indevel opi ngthecodc白 紙 will seeconsi derabl euse inthenext chapter. An明 ui val ent
devel opment wi th respect to 出 e pol ynomi al approxi mati on method is left as an
exerci sefor thestudent.
τ'he gol dcn secti on met hod requi res the establ i shment of the upper bound.
τ'h巴 refore,t wo functi ons wil1 need to be changed to handl e dcsi gn vectors and the
search di recti on vectors. The code is avai l abl e ぉ Upper Bound_nVar . m加 d
Gol dSect l on_nVar . m. The modi fi cati ons are nOl very chal l engi ng and can be
i nferredby compari ng theusage of 山 et wo functi ons.
For thesi ngl evari abl e:
Usage: Upper Bound_1Var (' Exampl e5_1' ,0,1,10) (5.12)
For many vari abl es:
U腿 ge: Upper Bound_nVar ( ' Exampl e5_2' ,x ,s ,O,l , 工 0) (5.17)
x: cur r ent posi t i on vect or or desi gn vect or
s: pr escr i bed sear ch di r ect i on
Of course, Exampl e5_2. mretums the val u巴 of 出 e functi on correspondi ng to a
vector of desi gnvari abl es. 百 lis i mpl i es 白 紙 thefuncti oncall toExampl e5_2. mcan
onl y take pl ac巴 after the desi gn vari abl es areeval uated f rom出 e val ue of stepsi ze
appl i edal ong山 eprescri bedsearchdi 陀 cti onf romthecurrent l ocati onof 出 edesi gn.
The usage of Gol dSecti on_nVar. mmust accommodat e functi on (5.17) and 出 e
need todeal wi tha desi gnvector. Compari ng wi ththesi ngl e-vari abl ecase
U蝿 ge:
Val ue Gol dSect i on_1Var ( ' Exampl e5_1' ,0. 001,0,1,10)
U皿 ge:
Val ue Gol dSect i on_nVar ( ' E xampl e5_2' ,0. 001,x ,s ,0,1,10)
Vi si t 血 巴 codetos巴巴 thed巴 凶 15.
Compar / son w/ th Sectl on 5. 3: The code Gol dSect l on_nVar . mwas runf rom
山 eCommand wi ndowusi ngthefol1owi ng listing:
>>x [0 0 0 ];
>>S [ 0 0 6] ;
>> Gol dSect i on_nVar ( ' Exampl e5_2' ,0. 001,x ,s ,0,1,10)
After 11i terati ons thefinal Ji ne intheCommand wi ndowwas
ans CI
0. 1000 1. 2000
The above is i nterpretedas:
o o 0. 5973
α1= 0. 1000; f (α1) = 1. 2∞ x
=0; A2=0. 0;
whi ch matches Secti on5.3and Equati on(5.16).
= 0. 5973 (5.19)
5. 4. 2 Exampl e 5. 3 -Two- Pol nt Boundar y Val ue Pr obl em
A rea1exampl e for one-vari abl eopti mi zati oncan be f ound in出 enumeri cal sol uti on
of thel ami nar f l owover a flat pl ate[5]. 百 leprobl em, usual l y attribut巳d to Bl assi us,
represents an exampl e of theexact sol uti onof theformi dabl e Navi er-Stokes cquati on.
百 le mathemati cal f oml Ul ati on, al l owi ng for si mi l ari ty sol uti ons, is exp陀 ssed by a
thi rd-order nonl i near ordi nary di fferenti a1 equati on wi th boundary condi ti ons
sp配 ified at t wo poi 脳 - a two-poi nt boundary val ue pr obl emσPBVP) . What
folI ows is theessenti a1mathemati cal descri pti on. 百 leinte悶 tedreader can fol l owthe
detai l s inmost books on fluidmechani cs, inc1udi ng出巴 suggestedrefe問 nce.
Mathematl cal Formul atl on
ff" + 2f " = 0
x=O; f (O)= 0; f ( O) =O
x=∞ f (回 }=1
The sol uti on, nondi mensi onal vel oci tyintheboundary l ayer, is obtai nedas
ず =f (x)
(5. 21b)
The sol uti on to Equati ons (5.20) to (5.21) is l argel y through speci al iterative
techni qucs. These techni ques use a numeri cal i ntegrati on procedure l i ~ 巴 出  
Runge- Kutta met hod to i ntegrate thesystemby guessi ngand adj usti ngthemi ssi ng
boundary condi ti ons at theiniti a1poi nt such白 紙 thefinal poi nt boundary condi ti ons
arerea1ized. This is due to出 efact that thei ntegrati onprocedu陀 susual l ysol veinitial
val ueprobl ems onl y. Inthe Bl assi us probl em,出 is woul d i mpl y the mi ssi ng initial
val ue can be regardedぉ adesi gnvari abl e.
( ' ( 0) =α
The obj ecti vefuncti onthereforewi l l be
Mi ni mi ze: ザ( ∞) 一 1)2 (5.24)
I mpl i edintheformul ati onis伽 l f (∞) isobtai nedby i ntegrati ngthedi 他 rential 叫 uati ons
(5.20) (山 ωe can be consi dered as differential cons釘'ain臼) . 百 lere is one othcr
consi derati onto takeinto account. 百 lei ntegrati on methods requirethe戸 ubl emto bc
expressed in state spa目 f orm,whi ch r珂 凶 問 叩 n山 - ordぽ differenti a1珂 uati on 10be
expre鎚 edas a systemof n白百 l -ordcr 叫 uati ons. 百 leconversi onis fairlystandardand is
done throughu溜 l Sformati onand i nu叫 uci ngaddi ti onal v組 曲 les. For this exampl e:
y ~ =f =Y2
Y; =f' =Y3
Y; =f " =ず' / 2
The opti mi zati oncan be formul atedぉ: Fi nd α
Mi ni mi ze: [Y2(∞)ー l f
usi ngtheRunge- Kutta met hod (or 叩 y other method) toi ntegral e

3 v
















Fr ompubl i shedwor k [5J theval ueofα ・=0.33206. I nsl eadof 出 e白nal poi nt at ∞ it
is consi dered 10be 5.0. Leavi ng白 is val ueas a vari abl ecandefi nea correspondi ng
two-vari abl eopti mi zati onprobl em. In山 ei mpl ementati ontofol l ow,出 epower 叩 d
conveni ence of MA1L AB ar芭 readi l yapparent
Exampl e5_3. m: Thi s m-fi l e is lI sed to i mpl ement expressi on (5.25). It calls the
MATLAB built-in functi on ode45 10 use the Runge・Kutta 4/5 order method. The
functi onode45 requi res thestateequati ons tobe avai l abl einanother m-fi l e. It a1so
E叫 ui res 出 edefi ni ti onof thei nterval of i ntegrati onas wel l as theinitial condi ti on. The
fol l owi nglinef romthecod巴 dcscri bes theusage (曲 ecode is welI comment ed too)
[t y ]
ode45( ' Ex5 3 st at e' ,t i nt val ,bci ni t ) ;
t is thei ndependenl vari abl eand y thedependent vector.
Ex5_3_st at e. m: The stateequati ons in(5.26) areavai l abl ein出 is file.
st st [ y (2) ー0. 5*y( 1) *y( 3) ] ' ; , y( 3)
Refer tol hecode for details. Gol dSect l on_1 Var . mis used f romthecommand line
for 白 is probl emas
U岨 ge:
Gol dSect i on_1Var ( ' Exampl e5_3' ,0. 001,0,0. 1,10)
Start i terati on:
st ar t
al phal ( l ow)
1. 0000
al pha( 工)
0. 1528
0. 1904
al pha( 2)
0. 2472
0. 0378
The fi nal i terati on:
i t er at i on 12
al phal ( l ow)
0. 3359
0. 0000
al pha( l )
0. 3364
0. 0000
al pha( 2)
0. 3367
0. 0000
al pha( up)
0. 4000
0. 0161
al pha( up)
0. 3371
0. 0000
The resul t expected i s α= 0.33206. However, the tol erance speci fi edwas onl y
0. 001. The initial boundi ng i nterval was onl y 0.4. The gol den secti on has
produced an i mpressi ve sol uti on to 山 e TPBVP. Al so note the conveni ence of
modul ar pl ' Ogrummi ng. No changes we陀 made to Upper Bound_1Var . mor
Gol dSect l on_1 Var . mtorunthis exarnpl e.
5. 4. 3 Exampl e 5. 4 - Root Fi ndl ng wi th Gol den Sect l on
The appl i cati onof thegol den secti onmet hod totheroot fi ndi ngprobl emis i mportant
toaccommodat e cqual i tyand i nequal i tyconstrai nts. Consi der Exampl e 5.4:
g(α):0.75α ,2 _ 1. 5α ー 1=0
0 $ α$ 4
明 is is 白 ei nequal i ty cons回 int of Exarnpl e 5.l aexcept 伽 t it is transformedtoan
equal i ty constmi nt . In血 is f ormtheαnecessary tomake it i ntoanacti veconstrai nl
can be establ i shed. The si mpl est way toaccompl i sh this is toconvert theprobl emto
a mi ni mi zati on probl emfor whi 抽 出 e numeri cal techni ques have al ready bcen
cstabl i shed. Squari ngEquati on(5.27) ensurcs that themi ni mumval uewoul d be zero
(if it exists). Therefore thesol uti on10Equati on(5.27) is thesame as
M出 mi ze: 月α): [0.75a
- 1. 5α- 1] 2
whi ch can be handl edby Gol dSect i on_1Var . m. While山 is was qui ck and pai nl ess
therear官 some attendant probl ems usi ng this approach. 目隠し the nonl i n回 rity has
i ncreased. This is usual l y not encouraged in numeri cal i nvcsti gati ons. Second, the
number of sol uti ons has sati sfyi ngF OC i ncreased. Fi gure 5.5rep陀 sents thefuncti ons
inEquati ons (5.27) and (5.28).
' 司
Sol utl on Usl ngGol dSect i on_1Var . m: The fol l owi ng di scussi on is based on
theexperi ence of attempti ngtorunthc gol densecti onmet hod wi ththel ower bound
at the val ue of 0, 白 e case wi th 出 e othcr exampl es. The問 is a probl emsi nce the
sol uti oncannot be found. The probl emapp回 rs tobe intheca1cul ati onof theupper
bound. lnmany of thenumeri cal techni ques thereis an assumpti on白 at theobj ecti ve
functi onbei ngdea1t wi this 山首 modal ,出 at is, has a si ngl emi ni mumintheregi onof
i aterest. For this to happen. it is assumed 白 at 血 e functi on bei ng mi ni mi zed wi1l
al ways decreasef romthel ower bound. Take a l ook at 山 efuncti onfi nFi gure5.5. It
is i ncreasi ngat thc va1ue of zero. Upper Bound_1Var . massumes that 白 einiti a1
i nterva1is tool argeand conti nues tobi sect it tofi nda 釦 ncti onva1ue白 紙 is less 曲 an
theone at thel owcr bound. Thi s dri ves thei ntervaI tozeroand thefuncti onis exi ted
wi ththel ower bound and wi th no sol uti on.
Remedy 1: The easi est way out of thedi ffi cul tyis tochangc theval ueof thel ower
bound and tryagai n. It is useful toundcrstandthat 山巴 probl emis wi ththecal cul ati on

' a




sol utl on
0.5 2.5 3 3.5
Fl gure5.5 Exampl 自 5.4.
of the upper bound (bracketi ng the mi ni mum) ra山 er than wi th 出巴 gol den secti on
method. Changi ng the l ower bound to 1 and rerunni ng th巴 gol densecti on me出 od
provi des a successful 陀 sult whi ch matches 出 egraphi cal sol uti on. Whi l e success was
real i zed. such a pr田 edureneeds user i ntcrventi onωwel l as some knowl edge about
th巴 probl cm- i nthis case therewas 出 egraphi cal sol uti on_ These arenot acceptabl e
qual i ti ωfor an automati c proc低 lure.
Remedy 2: Si oce the posi ti ve sl ope at the l ower bouod is i ndi cati ve of the
probl em. it shoul d be possi bl etosense this posi ti ve si gn 01 the sl ope and move the
l ower bound to the poi nt where the sl ope is negati ve. Such a poi nt can be
di scovered by thesam巴 scanni ngprocedure. Thc sl opecomputati on is numeri cal
and is based on f orward first di fference. Oncc mor c出 is change inthel ower bound
is onl y needed for the uppcr bound cal cul ati on. Upper Bound_m1Var . mis the
modi f i ed upper bound cal cul ati on whi ch rel ocates the   o w ~ r bound to the poi nt
where the sl ope is negati ve. The usage and i nformati on f romrunni ng the
Gol dSect l on_1 Var . mi ncorporati ng the chaoged upper bound cal cul ati oo are
shown bel ow. The samc usage wi th thc ol d Upper Bound_1Var . mdoes oot
produce any useful rcsul ts.
U岨 ge: Gol dSect i on_1Var ( ' Exampl e5_4' ,0. 001,0, . 1,20)
st ar t
al phal ( l ow)
al pha( l )
al pha( 2)
al pha( up)
。 0. 9931
1. 6069
2. 6000
1. 0000
3. 0624
2. 1720
0. 0289
i t er at i on 16
al phal ( l ow)
al pha( l )
al pha( 2)
al pha( up)
2. 5266
2. 5270
2. 5273
2. 5278
0. 0000
0. 0000
0. 0000
0. 0000
1. Burden, R. L., andFaires, J. D. , Numeri cal Anal ysi s. 4thed. , PWS_KENT Publishing
Company, Boston, 1989.
2. Hosleller, G・H. . sami na- M. s. . and MontaI OU- p. D. ,Anal yti cal ,Nwneri cd,仰 d
computati onal Methodshrsci ence and Engi neeri ng,prmti cc-Hal l -Ensl ewoodCliffs.
NJ . 1991.
3. Vー帥 1酬 ・ G. N.. Nu抑 制 Opl i l 仰 11011 Tecl mi q附 for Engi neeri ng Desi gl l ,
McGraw-Hi l l , NewYork. 1984.
4. Huntl cy. H. E., The DlvIlle Proporl i on: A Sl udy in Muthemal i r : al Beauty. Dover
Publ i cauons, NewYork. 1970.
5. Schl i chung. H. , Boundaη-Layer Theory, McGr aw- Hi U. NcwYo氏 1979.
5.1 Extend Exampl e 5.1toi ncl udc t wo i nequal i ty constrai nts for whi ch a sol uti on
exi sts-Di spl ay白 fωu 鵬 l (
5.2 Extend Exampl e 5 丘引.1 toincI ude t wo i 加n叫 ua叫alityconstrai nts for whi ch t 由herl巴 1凶sno
sol uti on. Di spl ay thefuncti ongraphi ca11y.
5.3 Reprogram白 ecode Sec5_2_2. mωobt ai n出 eval ues of f , 中 , and ct' through
MA'τ 1. ABfuncti onprograms 回 dappl y toExampl e 5. 1.
5・4 間 22J;;二13記 念 品 222:l:::Z347tユ設計 t
5.5 I denti fy the val ue of αwher e 山 e constr泊 nt (5.3) b巴comes acti ve usi ng 由巳
Ncwton- Raphsoo techni que.
5.6 I mpl emcnt a bi secti on procedure for the rni ni mi zati on of 山 e functi onof onc
vari abl e. Appl y it tosol veExampl e 5. 1.
5.7 For thepol ynorni a1apprOl dmati on i mpl ementati on, cxtend出 c code todi spl ay
出 eori gi nal func凶on加 d出 eapproxi mati ngpol ynomi al.
5.8 For 出 c gol densecti oni mpl cmentati on, extendthecodc todi spl aytheori gi nal
functi onand thc anal external bounds on thevari abl e.
Veri ちr 出 at 白 ebounds on thevari abl earedecreasi ngby曲 esame ratio. Why is
this better 白 anthebi secti onmethod?
5. 10 How woul d you set up thegol den secti onmet hod todet巴rmi nethezeroof 出巴
functi oni nsteadof themi ni mum.
5. 11 Combi ne 出 e gol den secti on 叩 d 由 e pol ynomi al approxi mati on to fi nd the
mmi mumof 出 efuncti on.
5. 12 I mprov巴 theaccuracy of thesol uti ontotheBI 蹴 ius叫 蜘 1over that presented
inthetext .
5. 13 Obtai n the sol uti on to
approxi mati on method.
the Bl assi us 巴quatl On usmg the pol ynomi al
Thi s chapter iIl us住 ates many numeri ca1 techni ques for muJ ti vari abl eunconstrai ned
opti mi zati on. Whi l e unconstrai ned op出 血 ati on is not a common occurrenc巴 in
engi neeri ngdesi gn, neverthel ess thenumeri ca1techni ques incI uded heredemonstrate
i nteres駐 屯 i deas and al socapturesome of theearl yi ntensi vewor k in出 eareaof desi gn
opti mi 四 tion[1]. They aI so provi de themeans tosol veconstrai nedprobl ems after they
have been transformedi ntoan unconstrai nedone (i ndi rect m巴出 odsー Chapter7).
Thi s book is uni que for thepresentati onof nongradi ent techni ques (Secti on6.2)
whi ch can be empower ed by the ubi qui tous avai l abi l i ty of incr芭di bl e desktop
computi ng power. Unti l very recentl y. appl i cati on of desi gn opti mi zati onrequi red
mai nf rame computers that rel i edon l argeprograms wri tteninFORTRAN. Today' s
desktopcan runprograms 白 atreq凶 reonl y Ii mi ted programmi ng skills. The desi refor
gl obaI opti mi zati on has brought i nto focus numeri caI techni ques 也 at are l argel y
heuri sti c, have l i mi tedneed for sophi sti catedgradi ent-basedmethods, requi rel i mi ted
programmi ng ski Hs. and requi re l i mi ted programmi ng resources. Usual l y such
techni ques requi re 白 紙 the sol uti on space be scanned usi ng a l arge number of
iterations. Thesemethods can i deal l ybe run on personaI desktops for unl i mi tedti me.
The pressぼ 巴 of l i mi tedcomputi ng resources is no l onger a si gni fi cant constrai nt on
thesetechni ques. Whi l e they may compromi se on el egance and sophi sti cati on, it is
i mportant to recogni ze 出巴 opportuni ty to engage desktop resources for greater
useful ness.
The uncons町 ai ned opti mi zati on probl emrequi res onl y the obj ecti ve functi on. The
book has chosen toemphasi ze anaccompanyi ng set of si deconstrai nts torestrict 出巴
sol uti on to 祖 acceptabl e ぬ i伊 叩 acel隠 gi on. Most numeri cal 附 加 i qu白 in 出 S
chaptEr agpo白 血 e 副?∞ns回 ints in I叩 ng out 恥 method. It i 包s u 削 s釦u剖 d拘I勿y 白恥
附 P仰on附s討ib 凶i日Ji町ty0 ぱft恥h 恥1砲ed 由e叩 町 tωove町r 的 t 恥h恥e討副d也cco 匂叩n附l I I nts as part of his expl orati on
of 出巴 opti mum. 百 is checki ngof 白 csi de∞ns甘 副 nt canbe easilyprogrammed inthe
code. Thus, theprobl emfor 出 is chapter canbedefi nedas
Mi ni mi ze f (X); 医] "
Subj ectto: xf $; XI $; xr; i = 1,2, • • • , n
6.1.1 Examp' e 6.1
; ; 2 3 1 2 : : z z ; 2 2 2 Z2 ; 2 J Ze m巴rpossi bl e押 hicaJ descri pti or
Mi ni mi ze
f ( X) =f (xl ・X2) = 3 + (XI - 1. 5x2l
+ (X2_ 2)2
Subj ect to:
OSX( S5;
Fi gure 6.1 provi des thecontours of 出 eprobl emand出巴 e)lutioncanbe l ocatedat
fJ ぷi ZL 2; ar v al uei s f =3 百 叩 蜘 nふ adrati c吋 hesol uti on
6. 1. 2 Necessar y and Suffl cl ent Condl tl ons
2; 江 trzz;::2jr2zτs忠之江 ;tt:!sZ立;:?足立
condi ti ons) are白 創 出 egradl ents must vani shat thesol uti on
21=2( Xt - 15ら ) =0
ま叩糾 1句 ) + 2(X2- 2) = 0
Equ叫ons (6.5) c叩 beωved to obtai n xj = 3・xi = 2. This matches 出 e graph凶
器2i l j i tM223品官官民法 iC別 :ttぉ; s z
問=ドニ l



0.5 2.5
5 3 1.5 2 3.5 4 4. 5
Fl guro6.1 Exampl e6.1.
The Hessi an is positive definite consi deri ng the deten凶 nants alone. Sec6_1_1. m
provi des血 E∞defor thesymbol i c calculationandalsodi spl ays白巴 g百phi 回 l d白 cription
ωt he probl em. The resultsof 白 ecalculationarcdi spl ayedinthecommand wi ndow. 百 le
satisfactionof theSOC identifies 血 脱 出 巴 sol uti onfor thedesi gnis a mi ni mum.
6. 1. 3 El ement s of a Numerl cal Technl que
τ'he el ements of a typi 回 1num師 団 1techni quc or al gori thm, canbc associ atedwi ththe
generi c a匂ori 伽 n in位。 duced duri ng 血 e di scussi on of the rel evance of the
one-di mensi ona1op白 ni zati onfor mul ti vari abl eprobl ems (Chapt町 5). 唄 us al gori thmis
iterativeandisalsoreferredtoasa searchalgoritlvn , astheiterationstakepl aceby movi ng
al onga searchdirectionor a searchvectorduri nganiteration. Thωes国 民 hdirections can
bedetem山 .edinmany ways. The different 臨 hni ques 白 at arep隠 sentedinthis chapter
pri mari l ydiffer 泊 howthcsearchdir民 tioni sωtabl i shed. 百 le問 mai ni ngel ements of the
algori 血 m,ex伺 pt for 血巴 convergenccJ stoppi ngcriteria, arethesame for al most all of
themethods. The aJ gori thmcanbe exprl 凶 scdas fol l ows.
IFileslObe downl ondedrromIhcwebsi 胞 nreindicaledbyboldracesonsserir Iype.
General Al gori t hm伺 aη
Step1. Ch∞seX
Stcp2. For each i terati oni
Determi ne searchdi recti onvector 8j
(lt woul d be ni ceif theobj ecti vedecreasedal ongthis di recti on)
Stcp3. Cal cul atesX, =αsi
Note: aXj is nowa functi onof thescaI ar <<j as S, is known f romStep2
的 is cal1ed出 estepsi zeas i tωtabl i shes 出 el engthof AXj
αj is d巴termi nedby Mi ni mi zi ngf (丸 山 whcreX' +I = Xj +企X1
Here, the convergcnce criteria ( FOC/ SOC), and thestoppi ngcritcria(I s
desi gnchangi ng? Exceedi ng i terati oncount? etc.) must be checked.
i ←i + 1;
OoTo Step2
ln this chapter, this al gori thmi c structure will be uscd for aI l of the methods. τ 'he
one-di mensi onal opti mi zati onwil1be i mpl emented usi ngthegol den secti onmetbod.
Exampl e 6.1 wil¥be used todcvel op thc di ffercnt numeri cal methods.
τ'hesemethods areaJ so caJ l edzero-order methods. τ 'he order refcrs to山 eordcr of thc
deri vati ve of theobj ccti ve functi on, necded to establ i sh thesearchdi recti onduri ng
any iteration. Zero order si gni fi es that no deri vati ves 制 官 used [2] , whi ch in t um
i mpl i es 白 紙 onl y functi on val ues are used to 巴S泊 bl i sh tbe search vector. There is
another i mportant si gni fi cance that accompani es tbe mi ssi ng deri vati ve
computati on- the F OC shoul d not be appl i ed in these methods. If wc were
computi ng deri vati vcs we mi ght as wel l inc¥ude this i nformati onfor establ i shi ngthe
searchvector. Onl y thechanges in白 eobj ecti vc functi onor thedesi gnvari abl es can
provi d巴 convergence四 d / or stoppi ngcriteria.
羽 田 etcchni ques arei ncl udedinthis secti on. The first is a heuri sti c one basedon
randomsearchdi recti ons. The sccond one cycl es througha searchdi rccti onbasedon
each varl abl c. 百 le last is Powcl l ' s met hod whi ch has tbc property of quadrati c
6. 2. 1 RandomWal k
The searchdi recti onduri ngeach i terati onisa r andomdi recti on. Randomnumbers and
sets of randomnumbers areusual 勿 avai1abl ethroughsoftware. MATLAB i ncl udes the
ab凶 ty to generate severaJ typcs of r andomnumbers and matri ces. Most
computer-generatedr andomnumbers ar唱 cal l edpseudorandomnumbers as they cycl e
after a suffj ci ent number of t hemhav巴 been created. The one吋 di mensi onal stcpsi ze
computati on is donc by thc Gol dSect l on_nVar . m向 ncti on(Chapter 5).

A句 orl thm: RandomWal k (A6. 2) 、
Step1. Choose X

N(number of iteratlons)
Set i = 1
Step2. For eachiterationI
= Randomvector
Step3. Xj +
= Xj +α,Sj
α'j is determi nedby mi ni mi zi ngj{Xi+I)
i ←ー i + 1
lf i < N 00 To Step2
el sc SωP (i ωrati ons exceeded)
EZ2222zteお szz:;12rz::;; むれは也事 i
圃 l;説話器忠告部諜誌記 f悲: ; 訪問
si gnbased on a test of randomnumber.
RandomWal k・m: Tl mr andomwal k al gori thmt A62) i s i mpi ementcd in this
OP6mi adon- TMcodeof the randomwal k functi on al so i ncl udes thE fol l owi ng
~ 唱
, ~ ・  
出 econtour of thc obj ecti veis drawn.
• For t wo- vari abl eprobl ems tn
• For t wo回 vari abl Eprobl ems thesuccessful i terati ons Me trackedon thecontour
d theiteration" number identified.
. 2; : u16v m油 附 d…“ …削 仲 間 10的 C叩 刊 記 dfor
theone-di mensi onal stepsi zecomputati on.
. The mul ti vari abl eup戸 rboundcal cul ati on(devel opedinchaptcr5)i s used for
. ;口?コ江::; E :泣:誌広: 2:zr1;z;;ニ11i In1山 here 出伽0…即 凶 帥 叫 川 h d 吋di 曲 枇 iI1泊附 ω r 肉 悶 e回c
.:エ::二 i : :エ;:ニ e L制 i gnvari abl es and山 cti ons for all of
thc i tcrati ons arc pri nted.
Use of previ ous code iII us町 ates code reuseand modul ar programmi ng. Once more
r伺 ders ar宅 strongl yencouraged 10visit thecode tounderstandth巴 位 ansl ati onof the
aJ gori lJ l mand subsequent appl i cati on.
U掴 ge:

RandomWal k( ' Exampl e6_1' , [ 0. 5 0. 5] ,20,0. 001,0,1,20) (6.7)   ~

I nput: (They areth巴 i tcms encl osedwi thi ntheoutermost parenthesi s inexpressi on
(6.7).) a
賞 ' Exampl e6_1・:
[0. 50.5]
0. 001
百 lCfileExampl e6_1. mwhere lJl e obj ecti veis descri bed
starti ngdcsi gnvecl or (dvarO)
number of i terati ons of themet hod (niter)
tol erancefor 出 egol den secti onproced町 e(tol)
initial stepsi zefor scanni ngupper bound (I owbound)
sl epi nterval for scanni ng(intvI)
number of scanni ngsteps (n肘 aIs)
A vector of n + 1 vaJ ues of desi gnvari abl es at thelast iterationand thevaJ ue of the
obj ecti vc functi on.
Executi ng functi on (6.7) in lJl e Command wi ndowproduces the fol l owi ng
i nformati on(出 ecomment s areadded later):
- The desi gn vect or and f unct i on dur i ng t he i t er at i ons










0. 5000
0. 5000
1. 0440
1. 0440
1. 0440
1. 0440
0. 9756
0. 9756
2. 1093
2. 0481
2. 0350
2. 0350
2. 0350
2. 0350
2. 0350
2. 0350
5. 3125
5. 3125
4. 6481
4. 6481
4. 6481
4. 6481
4. 6349
4. 6349
3. 0663
3. 0025
3. 0020
3. 0020
3. 0020
3. 0020
3. 0020
3. 0020
告 st ar t
者 di d not move
も decr eased f unct i on
告 di d not move
告 di d not move
告 di d not move
告 decr eased f unct i on
告 et c
3. 0804
3. 0804
3. 0804
3. 0804
3. 0804
2. 0350
2. 0350
2. 0350
2. 0350
3. 0020
3. 0020
3. 0020
3. 0020 も 20 i t er at i ons
2. 0350 3. 0020
Compare 自 問 vaI uωt o 山 e actual 叫 uti on of xi =3,xi =2,j* =3. T出s is
remark且hl efor a method出 at has no senseof preferreddi recti on. If theexecuti on15
resated wi th Menucal i nput,m ent i mnewsol uti onwi l l beobtai ned. Fi gure62tracks
th; 伽 Iges of the d叫 nvari 枇 son theco脚 l f pl ot. The movem巴 nt of ilie血 si gn
vari abl es are pai red wi th the corzespondi ng i terati on number . Ther andomwal k
functi on can bEExECUted for several di ffErent starti ng sets of desi gn vari abl es.I t
shoul dnot di sappoi nt. The Gol dSect i on_nVar . mis 山 esarneone as inChapl er 5
except 白 紙 iliepri nti nghas been suppressed.
Whi l e executi ng 出 e RandomWal k. mthe pri nti ng of 山 i terati on 山 mb ?
cmates a mess as you approach thesol uti on. You can swi tchit of f by comment t qgme
旬以( ー・ )statement- Thgeodei nRandomwal k- mal soseI ves as a templ ateforother
al gori i l i ms. lt is worth expl ori ngindetai l.
Fl gure6.2 Randomwalk: Exampl e6.1.

6. 2. 2 Pattern Sear ch
The PattemSearchmet hod is a mi nor modi fi cati on10l he Utlivariatemet hod wi tha
maJ or I mpact . As the PattemSearch uses the Uni vari ate met hod the latter is not
di scussedseparatel y. IntheUni vari atemet hod, a1so known悩 theCycl i c Coordi nal e
Descenl method, each desi gn vari abl e (consi dered a coordi na包) provi des a search
di recti on. Thi s is a1so refe汀 edto as a coordi natedi recti on, 四 d is easi l y expressed
throught 恥h恥1聡eu叩 n 山1
n削 umb邑釘r o ぱf v副aria油叫blesωs 山
d也e鈴側s討ig 伊nvari abl es) i 氏teratio 叩 ns onef or 巴悶ac凶hof t白hen searchdi recti ons. It can be shown bv
appIi cati on伽 t for probl ems wi thconsi derabl enonl i nea均, theUni vadatemet130J
tends toget l ock叫 i ntoaz討i gz 却 agp 仰at悦t胞E叩emof s叩ma 】 m叫a
由巴 s叩ol u山Jt 山110 叩 n.
τl l ePat旬 mSearchprc問 dureattempts todi srupt thiszi gzagbebavi or by executi ngone
addi ti onal iterationfor eachcycl e. Ineachcycl e, at 白 eend of 11Uni vari atedi recti onsー the
n+l s ea陀 h di recti on is assembl 伐 1as a I 伽II 加n即 脚 1隠巴悶即 ar c ∞伽 o ω叩 m】 11蜘b
d恥伽町附c伽 “on悶s 聞削d曲恥eopt i mum叫 u凶 eof t 山h恥le巴 蹴 p阿 討 悶 ze fo 叫 at direction. 正one-di m叩 ional
opti mal stepsi zeis 白 en∞mputed佃 dthenext cycl eof iterationbegIl1s.
A勿 ori thm: Pattern Sear ch伺 6.3)
Step 1. Choose X
, N
(number of cycI es)
j (1) =j ( X,) ; X
(1) = X,
E, ・ E2: tol erancefor stoppi ngcriteria
Setj = I (initializecycl ecount)
Step2. For eachcycl ej
For i =1,11
S; =会(uni vari ate山 p)
X; +, =X; +αβ ,
α/ is determi ned by mi ni mi zi ngj (X; +,)
(storeval ues ofα.i and S/)
cnd of For l oop
号 =玄 α: ; S; aX, 肺 , - X, (Pattcm取 肘
Xj =Xn+, + CちiSj
X心・+1) ←Xs J "心・+1) =j (再) (stor巴 cycl eval ues)
Step3. 6j =1rU+ 1) - 1rω A X = X,.u+I ) - X

If l.1jl S E" stop
I f AX
.1xS   stop
If j =N", stop
X , ← ~
j ←j +l
GoToSt ep 2
ThePatternSearchmethodhasaf cwaddi ti onal stoppi ngcri tcri a. Thcscarebasedon
howmuch thef uncti on i sdecreasi ngfor each cycl eand how much change istaki ng
pl acei nthcdesi gn var i abl ei tsel f. For thel atter thi sIl1f ormati on i sbasedon thel ength
of thechangei n thedesi gn vector for thecycl e. Onceagai n theapproach i ssi mpl eand
the i mpl ementati on is di rect. Pr ogr ammi ng such a techni que wi l l not bc too
forbi ddi ng.
Pat t ernSearch. m: To conlI nue出 eexposure 10MATu. B programmi ng出 i smethod
is i mpl emented very di fferent 1y f romtheother al gori thms that have bcen transl ated
intocode sofi訂 . It is a stand- a1one program. Therc areprompts for 山 er i nputs wi 山
appropri ate defaul t val ues in cas巴 the user deci des not to enter a val ue. The
one-di mensi onal stepsi ze computati on is i mpl emented by 血 e mul ti vari abl e gol den
secti onmcthod. The i nteresti ngfeatu問 sareas fol1ows.
. It is a freestandi ngprogram.
・ 百 lefilecontaIl1i ngtheobj ecti vefuncti onis sel ectedthrougha list box.
• Scveral programcontrol par宙 neters are entered f romthe keyboard after a
sui tabl eprompt i ndi cati ngdefllult val ues.
• Dcfaul t val ues arc initializedif theuser chooscs not 10entcr theval ues.
• Thc desi gnvcctor is trackedand pri ntcdaftcr thc programis stopped.
• Si nce theupper bound cal cul ati ons arc based on posi ti vc val ucs of stepsi ze, thc
search di recti on is 陀 vcrs吋 if 由 巳 問 t umed stepsi ze is 出 e same as the l owcr
• Iterati oncounts arerecorded.
• For two-vari abl eprobl emtheobj ecti vefuncti oncontours arepl otted.
. The first ni ne or Icss i terati ons aretracedon出巴 contourpl ot . Thc Uni vari ateand
thepattemsteps areco)or codcd.
U掴 ge: Pat t er nSear ch (6.8)
The prob)emis Exampl e 6.1.百 lefi]eis Exampl e 6_1. m.
Out put ( MATl .AB Command wi ndow): Wi t h 山 e defaul t va)ues and the starti ng
vector [43] thefol l owi ngarewri ttentotheCommand wi ndow.
>> Pat t er nSear ch
The f unct i on f or whi ch t he mi ni mum i s sought must be a
MATwa f unct i on M - Fi l e. Gi ven a vect or dependent var i abl e
i t must r et ur n a scal ar val ue. Thi s i s t he f unct i on t o be
MI NI MI ZED. Pl ease sel ect f unct i on name i n t he di al og box
and hi t r et ur n:
噌 司
The f unct i on you have chosen i s : : Exampl e6_1
maxi mumnumber of cycl es [ 1000] :
conver gence t ol er ance f or di f f er ence i n f [ l e・ 8] :
conver gence t ol er ance on change i n desi gn x [ l e- 8] :
I nput t he st ar t i ng desi gn vect or . Thi s i s mandat or v as
t her e i s no def aul t vect or set up. The l engt h of シour
vect or i ndi cat es t he number of unknowns. Pl ease ent er i t
now and hi t r et ur n:
[4 3]
The i ni t i al desi gn vect or [ 4. 00 3. 00]
Conver gence i n f : 3. 999E嗣 009 r eached i n 10 i t er at i ons
Number of usef ul cal l s t o t he Gol den Sect i on Sear ch
Met hod: 31
Tot al number of cal l s t o t he Gol den Sect i on Sear ch
Met hod: 40
The val ues f or x and f ar e
4. 0000
4. 4992
4. 4992
4. 3868
4. 1432
4. 1432
3. 8229
3. 3338
3. 3338
3. 1750
3. 1980
3. 1980
3. 1944
3. 1473
3. 1473
3. 0346
2. 9958
2. 9958
2. 9971
2. 9972
2. 9972
2. 9972
2. 9977
2. 9977
2. 9999
2. 9998
3. 0000
3. 0000
2. 6924
2. 7616
2. 7616
2. 5287
2. 2224
2. 2224
2. 1543
2. 1321
2. 1321
2. 0916
2. 0980
2. 0980
2. 0682
1. 9969
1. 9969
1. 9981
1. 9981
1. 9981
1. 9986
1. 9988
1. 9988
1. 9990
1. 9999
1. 9999
4. 2500
4. 0000
3. 6916
3. 6398
3. 5801
3. 4022
3. 2890
3. 0494
3. 0343
3. 0180
3. 0175
3. 0121
3. 0118
3. 0096
3. 0067
3. 0016
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
0. 4996
0. 3081
0. 2254
0. 2446
0. 2339
1. 3148
0. 4896
0. 0681
0. 3248
0. 0234
0. 0405
0. 1579
0. 0471
0. 0298
2. 3916
0. 0391
0. 0013
0. 0322
0. 0001
0. 0005
0. 4563
0. 0005
0. 0002
4. 9305
0. 0002
2. 9998
2. 9998
2. 9998
2. 9998
2. 9998
1. 9999
1. 9999
1. 9999
1. 9999
1. 9999
0. 0000
0. 3398
0. 0000
0. 0000
0. 2020
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
To veri fy 白 紙 出 eprogramis executi ngthea1gori thm, sc佃 thedesi gnvecl ors 岡 山 E
first l Wo col umns. The uni vari al esteps can b巴 recogni zedwh巴 none component of the
vec10r changes whi l e 由巳 other mai ntai ns its val ue. The pattern di recti ons can be
;1. recogni zedwhen bothcomponents change.
~   The number of i terati ons is a si gni fi cant amount . One reasonis that thetol erances
aremuch smal l er. It can be notedthat 由巳 di fferencebel ween l wo i l erati ons ch叩 ges
I i nearl y. Such methods are expec1ed to run for a l arge number of iterations. The
,1 : programcon佐 01par羽 田 ters control theacc町 冨 cy of 出 eresul ts as wel l as 由 enumber
of iterations. The sol uti onis mor e accurat巴 thantheRandomWa1k me出 od.
Fi gure 6.3 iII ustral es 出 e a1gori thmgraphi cal l y. The uni vari ate and the paUern
di recti ons arequi tedi sti ngui shabl e. When executed on your machi ne th邸 巳 steps are
expl i ci tl y col or coded. Readers areencouraged 10go over thecode. and modi f y i1to
0.5 1.5 3
Rgure 6.3 Pal l emsearch: Exampl e6.1.
refl 配 t th巴 ir preference. The number of l i nes is around 250. We at琶 湖 rti ngtoget i nto
senous programmmg.
6. 2. 3 Powel l ' s Me'肋 od
lf there were onl y one zero-order met hod 白 紙 must be progranuned, the
overwhel mi ng choi ce woul d be Powel1's met hod [3]. The pri nci pal reason for 曲 e
deci si on woul d be that it has the property of ql l adrati c convergence, namel y, for a
quadrati c probl emwi thn vari abl es convergenc巴 wi l l be achi evedinless thanor equal
to n Powel1 cycI es. A quadrati c probl emis an uncons住温 i ned mi ni mi zati on of a
functi on that is expressed as a ql.ωdrati c pol ynomi al - a pol ynomi al wi th no term
havi ng a degree greater than two. Exampl e 6.1 is an exampl e of a quadrati c
pol ynomi al in t wo vari abl es. Engi neeri ng desi gn opti mi zati on probl ems are rョrel y
descri bedby a quadrati c pol ynomi al . 百世 sdoes not i mpl y that you cannot use Powel1' s
method. What this means is that the sol uti on shoul d not be expected to converge
quadrati cal l y. For nonquadrati c probl ems, as the sol uti on is approached i terati vel y,
出 eobj ecti vecan be approxi mated very wel l by a quadrati c functi on. It is at 山 is stage
出 at thequadr副 cconvergenc巴 propertyis reaJ i zedinthecomputati ons.
The actual aJ gori thm( A6.4) is a si mpl e (the wor d is not bei ng used Ii ghtl y)
modi fi cati ontothePattemSearch al gori thm. l neach cycJ e (a仇 er 出 efirst) , i nstead
of usi ng uni vari ate di recti ons in the first n i terati ons. the search di recti ons are
obtai ned f romthe previ ous cycJ e. The newsearch di recti ons areobtai ned by Iψ
shi fti ngthedi recti ons of the previ ous cycJ e. In出 is way a hi story of the previ ous
searchdi recti ons is used toestabl i sh the next . Wi t h this change. Powel l ' s met hod
shoul d converge at most in three cycJ es for Exampl c 6.1 rather than the several
listed intheprevi ous secti on.
A匂 ori thm: Powel l ' s Met hod仰 6. 4)
Step 1. Choose X" N
( number of cycl es)
Jt l) = j{X
); Xc{l ) = X,
E, ・ E2: tol erancefor stoppi ngcriteria
Setj = 1 (initializePowel l cycl e count)
For i =1, n
S; (uni vari atestep)
Step2. For each . cycl ej
For i =l . n
If j 注 2,S; ←S;+,
X;+, =X; +αI SI
α; is detenni ned by mi ni mi zi ngJtXi+')
end of For l oop
Sj =S; +, =I α ,;S; = X
+, -XJ ( pattemstep)


Xf =Xn+, +α戸j
XcU+1) ←Xf ; f cU+恥 f (Xj) (s附 cycI e叫 附
Step3. !:if=f cU+ 1)一五 ,U); d1{ = XcU + 1) - Xc(j)
lf ldfl S E" stop
lf d1{T dX S E2. stop
If j=N
• sωp
X,← X

j +l
GoToSt ep2
APPMeat t on of poweI I ' s Met hod: The transl ati onof theal gori thmi ntoMATLAB
COdei sl ef t as m exerci sefor thestudent. Step-by-stepappl i caGon t oExampl e 6.1 is
shown here. Exampl e 6.1 is res旬 飽 das
Exampl e6. 1
Mi ni mi ze f(X) = f (x" X2) = 3 + (x, -1.
5 X
2)2+ (x2ーが (6.3)

X, 仲 E, =
S, =[ 6] ; S2
j =l
Step2. j = 1
X2 4 3 1 +αl 国= ? ? l
2. 吋 =1/4; X2=防 l
問 句 砕 ユ
4 4 4 6 1 5 ; X 4 z j ; 沢 民 ) =4. 5576
S) = 0. 25[6] 仰 輔 自   ぷ
X4=[ 03251ベぷ i 5]
i = 2:
i =3:
Inlight of theprevi ous definition. thesewoul d bereferredtoasfi rst. order m巴出 ods.
The searchdi recti ons will beconstructedusi ngthegradi ent of theobj ecti vefuncti on.
Si nce gradi ents are bei ng computed. 出 e Kuhn- Tucker condi ti ons (FOC) for
unconst凶 nedprobl ems. Vf = O. canbeusedtocheck for convergence. The SOC are
hardl yever applied. One of thereasons is 血 at it woul d i nvol vethecomputati onof an
n X 11 second deri vati ve matri x whi ch is consi d巴red compuぬ tionally expensi ve.
particularlyif 出 eeval uati onof the0何回 tivefuncti on問 qui resa call toa finiteel ement
method for generati ngrequi r叫 i nformal i on. Another reason for not cal cul ati ngthe
Hessi anis 血 at theexi sl enceof 白 esecondderi vati veina real desi gnprobl emis not
certaineven thoughit is computati onal l y possi bl eor feasible. For probl ems that can
bedescri bedby symbol i c calculations. MA11. ABshoul dbeabl etohandl ecomputati on
of secondderi vati v巴 at thepossi bl esol uti onandi岱 ei genval ues.
Wi thout SOC thesemethods requi reuser' s vi gi l ancetoensure白 紙 thesol uti on
obtai nedis a mi ni mumrather 山 ana maxi mumor a saddl epoi nt. A si mpl 巴 way 10
verify 出 is is to perturb the obj 巴ctive functi on through pe目 urbati on in th巴 desi gn
vari abl es at thesol uti onand verifyit is a local mi ni mum. Thi s bri ngs up ani mportant
prope町 of theseme血 ods- t heyon1y f i nd l ocal opti mums. Usual l y this will be c10se
to th巴 desi gn where the iterations are begun. Before concl udi ng the desi gn
expl orati on. it is necess釘 Y toexecute the method f romseveral starti ngpoi nts to
di scover if other mi ni mums 巴 xist and select the best one by head to head
compari son. The bul k of exi sti ng unconstrai ned and constrai ned opti mi zati on
methods bel ongto出 is category.
Four mcthods arepresented. The first is thc Steepest Descent method. Whi l 巴 this
method is not usedinpractice. it provi des anexcel l ent exampl e for understandi ngthc
al gori thmi c pri nci pl es for 臨 gradi ent-basedtechni ques. The secondis theConj ugate
Gradi ent techni quewhi ch is a classical workhorse pa凶 cularlyini ndus町 usage. The
thirdand fourthbel ongtothecategoryof Vari abl e Metri c methods. or Quasi - Newt on
me出 odsωt heyarealsocalled. These methods havc beenpop叫 釘 for some ti me. 回 d
will possi bl ystaythat way for a l ongti metocome.
Thc gcneral probl emand specific exampl e ar志 向 producedfor conveni ence
f ( X
) =4.4843
α・=- 0.14111'¥' Y 川 .
  吋 1¥. 2 = 11. 01041; f ( X
) =4.4145
α; =0.4235;
Step3. l.df l = 1.353>EI' conti nue
dXl = 1.8992> Ez. conti nue
} = 2: One cycl eover
GoToSt ep2
Step2: }=2
xJ 0. 855り
J =11 1 c' 7n 1 :
11.l 570 l '

二111iZ] ; S 島←叶吋 2 戸 イ2=[ 0 イ=[ 0 ら0 品2沼2 払5J 叫 山 =0ω 附 叩 ω 4 伺仰側 0的 防 3 お 叫 5
i =3:_
v _10. 95671 r 0.10ω11 r'l l
吋 - U. 1966 J 伊 α 以 = /2/; f ( X
) = 3
おご抗出土: : ニ i ; ; 32! emd恥 C叫 utati ons in出 i…ercise蹴 stopped
山 i t E : 2 7忠 ; : J r : at J ; ; : : : : woul dit 叫 here? You蹴 巴 …agedto
- ηl e method onl y took two cycl es toconverge (as expected)・
・ T heextent of compu凶 onis 恥 sameas thePattemS即 chb川 lenumberof
iterations is significantlysmal 1er.
・2: ; ; 212S22;お 訟 は : r s econdml ei S 叩 tive. H 恥 E叩 町 町 n即 附 c印e
• The stepsi ze for the last itcration is 20ぉ . The supporti ng al gori thmfor
; ; : : : !1gand gol densecuonmust be abl etO凶 tp the mi ni muma 剖t 伽 elar
• Whi 】 討i l eno 削oti l h 加u 凶I犯sl回m即 t鈴凶 ε吋dh恥m町er凧E久.th 白恥 h恥el sa制x対i ml 剛 3机加削 u 刷 叩 1m 町n m 叩Il川u 叩um】be 凶?引r 印'0 ぱf c句y戸c1eωs tωoc∞onv刊er:屯ge伺 n 町1にc印efl白or日
z詑::抗r払L :忠:立::z2ぷ1沈均衣な;f: ユ: f 曲 目 町 g poi nt . Y…
j (X); [X]"
i =I . 2. . . . . n
j (X) = f(XI . X2) = 3 + (XI - 1.5x2)2+ ( X2 - 2)2
OSXI S5: o S X 2 S 5
Mi ni mi ze
Subj ectω
Subj ect to:
Mi ni mi ze
訪協 2jz:32ifi:5132ti: 誌記; 容認
matri x. lf sea帥 di recti ons S . 品j a佃n吋d号 町 叩 昭 , 除 di廿悶ec“伽o α∞n 眠E
c ∞on.町リ肝卯uga舵cya訂rc
Steepest Descent Met hod
This method provi des a natural evol uti on for the gradi ent based techni ques [4].
The gradi ent of a functi onat a poi nt is thedi recti onof themost rapi di ncreasein
SJ[ U] Sj =0
Veri 今 if it is truc for thcexampl e.
theval ue ofthefuncti onat tllat poi nt. The descentdi r巴cti oncan be obtai ned reversi nl!
thegradi ent (or mul ti pl yi ngit by -1). The next stepwoul d be toregardtl1e descent
vecl or as a search di recti on, after all we are aUempt i ng 10 decrease tl1e functi on
Out put (wri ttento白 eCommand wi ndow): A coupl e of i terati ons at 出 estart and at
司 自 EEnd(out of 20i terati ons)Me copi ed b-l ow.
through successi ve i terati ons. Thi s seri es of steps gi ve riseto白 eSteepest Descent
al gori thm j
A匂 ' ori thm: Stl僧 P白 t Descent 伺 6. 勾
Step J. Choose XJ , N (number ofi terati ons)
!.(l ) = j {X1); X. (l ) = X
(storeval ues)
EJ , E2' EJ: (tol erancefor stoppi ngcriteria)
Set i = 1 (i niti a1i zei terati oncounter)
Step2. Sj = - Vf (X/ )(出 is is comput ed inStep3)
X;+l   α
的 is determi ned by mi ni mi zi ngf (X; +I)
Xs(i + 1) ←X件 1; 五(i + 1) = f ( X
) (storeva1ues)
Step3. aJ =!. (i +l ) ー ぶ り a X= Xs(i + 1) - Xs(i)
Ifl aJ l 陶 (functi onnot changi ng)
If a X
a X E2; stop(desi gnnot changi ng)
[fi + 1 = N; stop
If Vf( X; +l l Vf (X
) E3; converged
i ←i +1
GoToSt ep2
St eepest Descent. m: Thi s is an m-fi l e白 a1executω a1gori thmA6. 5. It uses 白 e
gol den secti onand tl1e upper bound scanni ngPI 1凹 ess. 百 leた aωres ofthi s programare
as fol l ows.
• For t wo vari abl es it wiJI drawtl1e contour pl ot .
• For t wo vari abl es the desi gn vector changes can be seen graphi cal l y in sl ow
moti on wi tl1steps indi fferent col or.
• The desi gn vari abl es,出 efuncti on va1ue, and 出e SqUar巴 of tl1e l engtl1 of 曲 巴
gradi ent vector(ca11edKTval ue) at each i terati onaredi spl ayedintl1e Command
wi ndowat compl eti on of tl1e number of iterations.
• The gradi en1of tl1e functi onis numeri cal l y comput叫 usi ngfirst f orward fi sI te
di fference. The gradi cnt computati on is tl1erefo陀 automati c.
• Fr oma programmi ng perspecti ve, basi c stri nghandl i ngtochange linecol or is
i ntrocl uced.
Usage: St eepest Descent ( ' Exampl e6_1' , ( 0. 5 0. 5] ,
20,0. 0001,0,1,20) (6.10)

0. 5000
0. 5000
5. 3125 5. 2969
0. 5996
0. 9483 4. 7832 2. 8411
1. 0701
0. 8429 4. 3766 3.1379
2. 6901
1. 8659 3. 0298 0. 0507
2. 7455
1. 8502 3. 0233 0. 0462
2. 7585
1. 8960 3. 0181 0. 0316
Fi gure 6.4rep陀 sents thegraphi cal rnoti onof thedesi gn vari abl es for 20 i tErati ons.
The graphi cal pi ctureand thechanges inthedesi gnarEnot i mPEESSi VE・TheSteepest
Descent mEt hod iSWOEf ul l y MadequatE Compazed toPowel l -s met hod even i fthelatter
is a zero-order metl1od. MO[ 1回 ver,白 is concl usi on is dr awn wi th rωpect lo an回 sy
probl em(quadrati c) Exampl e 6. 1. 百 is perf ormance j usti fi es tl1e l ack of seri ous
interest intheStecpest Dcscent method.
円 gure6.4 Sl e邑 pest desωnt : Exampl e 6.1.
r J ぷ; なお?ね 12: ロ:JI 出:241詰誌な 2EZ誌な
巴xpected to overcom巴 this pattem. In thecase of the Uni vari ate m仙 odthis was
achEeyed through patFrn Search met hod in the zero-order f ami l y. An i terati on
breaki 時 0円 of the zi gzag pa旧 m (or preventi ng getti ng l ocked i nto oInEP、i5
:芯:芯: : r r o: 九ばliz::: 尽忠:立ロ 22:口eme 叫 e仙白 o山d “Su 叫ch 川 】
6. 3. 2 ConJ ugat e Gradi ent (Fl etcher- Reeves) Met hod
TheCo吋 ugateGradi ent method, ori gi nalI y dueω F1etcher and Reeves [5] , is a smaI l
間宮 e i器z器部i i ; 号帯?:諮;諮詳 ; 5鵠諜鵠 5謎誌器25g:;:混2訳;qi詫号告:?告:1i詰z能i5;鴇詰諮;
c ∞onv, 叩erge 叩 nc巴 b ecau sethe s earchd l tu1o E n 3s aMr巴 cω01 ベゆIj ψj ugate w 川 it出h陀 s叩p民 削ttωotheHessi an
:ご工幻:?xi : E: 広:包二誌ば:ごナ lu 山u凶“ωo 叩 n. A qu聞 帥 la
A匂 'or/l的 m: ConJ uga俗 Gradfj開 t伺 a句
Step1. Choose X" N (number of i terati on直)
!s(l ) =j ( X,); X. (l) = X,(storeval ues)
E, ・ E2・E3: (tol erancefor stoppi ngcriteria)
Sel i = I (i nilializeiterationcounter)
Step2. If i =I ,Sj =- Vj ( X
El ses= 可 ' (xj f Vj (X
Vj ( Xj_l) T Vj ( Xi_')
Si =- Vj (丸 ) + sSj _,
X' +I =X/ +αβt
αI is determi nedby mi ni mi zi ngj (X
X.(i + 1) ←X; +,; 五 (i +1) =j ( X川 ) % (sω陀 V山 巴 s)
S句 3. !J.j =f s(i +l ) ーが1) ; 企X=X. ( i +1ト X. (。
Ifl !J.j l s 凶E町Iμ; 岬 (仇f仇向u町 m附削Jn肱刷刷 n恥削刷 E陀削削附 c ω叫ti 山i onn刷E
I旺f !J. X' !J. X: $E2必 stωop (desi gnnot changi ng)
I f i +I =N; 5tωop
I f Vj (Xi +,f Vj ( X件,) :$E3; converg巴d
i +1
GoTo Step2
cふ; : : P22J t : : 2J Ul z ; i t t r t t ; 1 2 3 t z r : r n ;ぷ;。;:;
computati on--s. and th巴 succes5i ve adj ustment of 恥 search di recti on
i ncorporati ngthis val ue. represents therati oof thesquare of thecurrent gradi ent
vector tothesquare of theprevi ous gradi ent vector. The first thi ngtonOl i ceis 山 at
a degree of robustness is bui l t i ntothe met hod by carryi ngi nformati onf romthe
previ ous iteration. Thi s is Ii ke mai ntai ni nga hi story of themet hod al bei t for j ust
one i terati on. Si nce the F OC is based on the l ength of the gradi ent approachi ng
zero at thesol uti on. 出 is P訂 ti cul ar f onn of i ncorporati onof is i ngeni ous. If the
previ ous i l erati onis cl osetothesol uti on. thens is l argeand出巴 previ ous i terati on
pl ays a si gni fi cant rol e in the current i terati on. On the other hand, if s is l arge.
suggesti ng the sol uti on is still far away, then the current val ue of the gradi ent
determi nes thenewsearchdi recti on.
Appl l catl on 0 ' ConJ ugat e Gradl ent Met hod: This method is i mportant to
warrant worki ng through the iterations. These caI cul ati ons, si mi l ar to Powel l ' s
m巴thod,canbe done usi nga hand caI cul ator. usi nga sp陀 adsh回 t,or usi ngMATLAB
itsel f. The author recommends thecaI cul ator as 白 eprocess is sl owenough todevel op
an i ntui ti on worki ng wi th the pal temof numb巴rs. In the fol l owi ng・and unl i ke
Powel l ' s m巴thod,on1y the rel evant caI cu1ati ons are recorded. The reader shoul d
fol l owaI oog veri fyi ngthenumbers arei odeedcorrect.
Step 1. X1 =1 o:sl; f (X,) = 5. 3125
1 0.5 1 • _. ___ . . . . . ___. __ 10. 59961
Step2. S, = α = 0. 1993; f(αi ) = 4. 7831; X2 =1 O: 94841
t = 2 │ - 1 . 6 4 5 9 1
Step2: s = 0. 5351 町 民  
11. 91351 牛 131
2 = I O: 83831; α ,i = 1.2544; f(叫 ) =3. 0; X3 =121
Con) ugat eGradl ent . m: This m-fi l ewill executetheConj ugate Gradi ent method.
For two吋 ari abl e probl ems there will be a contour pl ot over whi ch 出 e iterative
progress inthedesi 伊 spaceis recorded. The property of qua合 8tiCconvergence can
be observed ぉ the sol uti on is obtai ned after t wo iterations. This is defi ni tel y
i mpressi ve compared wi th 出巴 St田 pest Descent m巴出 od. K田 p in mi nd 出 at the
modi fi cati ontotheaI gori thmis mi nor. As 出 evari ous codes areruninorder. some
changes inthefi gurecanbe observed. Thesearelarg巴 lydue totheuseof some string
processi ng functi ons. Y ou are encouraged to see how白 ey are i mpl emented. The
author is certai nthereare出 i ngs you woul d do di fferentl y. Do not hesitat巴 to甘 y,as
白 紙 is theon1y way tol earntouse MATLAB eff, 巴 ctively.
U蝿 ge: Conj ugat eGr adi ent ( ‘ Exampl e6_1' , [ 0. 5 0. 5] , (6.11)
20,0. 0001,0,1,20) ;
Out put ( Command wi ndow)
The probl em: Exampl e6_1
The desi gnvector, functi onval ueand K T val ue duri ngthei terati ons
2. 9995
0. 9483
1. 9982
4. 7832
3. 0000
0. 2490
2. 8411
0. 0001
Keep in mi nd the sol uti on is dependent on the tol erance for the one-di mensi onaI
stepsi ze searchand that thederi vati ves arecomput ed nurneri cal l y. The executi onof
白 ecode shoul ddi spl ay Fi gure 6.5inthefi gurewi ndow.
6. 3. 3 Davl don- Fl et cher- Powel l Met hod
The Davi don-Fl etcher-Powel l ( DFP) [6] met hod bel ongs to 山 e fami l y of Vari abl e
Metri c Met hods ( VMM) . I t was nrst i ntroducedby Davi don and seVEral yEMB later
was devel oped in its current f ormby Fl etcher 剖 d PowelI. B巴i ng aware of the
Conj ugate Gradi ent met hod, thesemet hods woul d not merI t incI usi oninthis book if
they di dnot have theproperty of qu. adrati c convergence, whi ch they do. General l y
2 3.5 4 4.5
Fl gure6.5 Coni ugategradl ent method: Exampl e 6.1.

this far凶 lyof met hods go beyond that. As thesol uti onis approached, they behave l i ke
Newton' s met hod (Secti on6.4). The quadrati c convergence of Newton' s met hod is
qui l ei mpressi ve- i t wi l l l ocate山 sol uti ontoa quadrati c probl eminolleiteration.
Thi s Newt on- I i ke behavi or of theV MM, as thesol uti onis approached, has gi ven t hem
anot hernam巴 termedQuasi -Newtol l or Newton-l i ke methods.
The DFP met hod is presentedhere for hi stori cal i nterest 血 d because it is a little
easi er tounderstandthan由巴 others. I nfact , Huang (7) documents a generi c procedure
f romwhi ch most of 出 epopul ar methods can be obtai ned, and f romwhi ch you coul d
al soderi veyour own.
The Conj ugateGradi ent method' s i rnprovement over theSteepesl Descent met hod was
possi bl e because of the i ncl usi on of the hi story 針。 m the previ ous iteration. ln the
quasi -Newton methods th巴 hi story f romaIl previ ous iterations is avai l abl e. Thi s
i nform岨 onis col l ectedinann X n m縮 ixcal l ed血 emetri c. The metri c is updatedwi 出
eacbiteration釦 dis used toestabl i shthes回 詑 hdi recti on. An initial choi cefor themetri c
is al so 明 u田 d. It must be a り 刷 tetバ'c posi ti ve deftnite matri x. For the m抽 od to
converge, themetlI c must hol don toits posi ti vedefi ni tepropertythroughtheiterations.
h 出 eDFP method, themetri c approaches 由巳 i nverseof theHessi anat Ul esol uti on.
A句 ori thm: Davi don- F , 制 ' cher- Powel l (A6.刀
Step 1. Choos巴 x.. [A1l (initial me甘 ic},N
Eh e2' eJ: (tol erancefor stoppi ngcri 町 ia)
Set i = 1 (i nitializ巴 i terati oncounter)
Step2. 8,=ー [Aj ]Vf (Xj)
= X
+α ,su AX=αj S,
αi is determi ned by mi ni mi zi ngf (Xi +l)
Step3. If Vf(Xi +I )TVf ( X
) 三 e3; converged
If ! f( X
) ー f(Xj )1::;; el; stop (functi onnot changi ng)
  ・ (desi gnnot changi ng)
If i + 1 = N, stop (i terati onI i mi l )
El s巴
Y = Vf (Xj +l) - Vf ( Xj}
z=[A; ]Y

V' Z
[ Ai+Jl = [ A, ] + [B] + [ q
i ←i + 1
GoTo Step 2
Intheabove白 ematri ces areencl osed by square brackets. The initial choi ce of the
metri c is a posi ti vedefi ni tematri x. The i denti tymatri x is a safechoi ce.
DFP. m: The DFP al gori thm(A6. 7) is coded in山 is m-fi l e. Si mi l ar 10theprevi ous
programs, for two vari abl es 山 edesi gnchanges aretrackedon a background contour
pl ot , Fi gure 6.6 . The defaul t initial metri c is Ihe i denti ty matri x generatedusi nga
MATLAB built-infuncti on.
Usage: DFP( ' Exampl e6_1' , [ 0. 5 0. 51 ,4,0. 0001,0,1,20) ; (6.12)
Output: The output f romthe above i nvocati on (copi ed f romthe Command
wi ndow) can be obtai nedby removi ng thc scmi -col onafter thestatement . The vari abl e
defi ni ti ons match thosedefi nedintheal gori thm. For Exampl e 6.1



4 4.5
FJ gure6.6 DFP Mel hod: Exampl e 6.1.
i t er at i on number : 1
0. 4990 2. 2467
del x =
0. 0996
0. 4483
- 1. 1458
2. 6153
- 1. 1458
2. 6153
0. 0094 0. 0422
0. 0422 0. 1899
開 0. 1610 0. 3676
0. 3676 - 0. 8390
0. 8483 0. 4097
0. 4097 0. 3509
i t er at i on number : 2
1. 2443 0. 5446
The pr obl em: Exampl e6_1
The desi gn vect or , f unct i on val ue and KT val ue
dur i ng t he i t eど at i ons
0. 5000 0. 5000 5. 3125 5. 2969
0. 5996 0. 9483 4. 7832 2. 8411
2. 9996 1. 9987 3. 0000 0. 0000
6. 3. 4 Broydon- Fl et cher- Gol df artト ShannoMet hod
If you were to programonl y one gradi ent-bascd m巴 thod. then thc Broydon-
Fl et cher - Gol df ar b- Shanno( BFGS) 181met hod woul d be the one. I t is a
quasi - Ncwton met hod and currentl y is the most popul ar of th巴 Vari abl e Mctri c
methods. It enj oys l he propcrl y of quudrati c convcrgencc and has robustness by
carryi ngf orward i nformati onf romI13Eprevi ous i terati ons. Thedi f f erencEbEtween
由巳 DFPand BFGS is theway themel ri c is updal ed. The f ormer converges tothe
i nverseof theHessi an. whi l e thelatter converges totheHessi an itself. Ina sense
山 e BFGS is mor c di rect. The BFGS has repl aced the Conj ugate Gradi enl
techni ques as a workhorse insol vi ngnonl i near equati ons.


For convergence the metri c must be posi ti ve defi ni te. An initial choi ce of
posi ti vedefi nIl巴 matri x for themetri c is usualJ y suffi ci enl l oensure this property
for quadrati c probl ems. The i denti ty matri x is usual l y a defaul t choi ce.
Al gori thm: Br o附 加 ・ F倒 的 er- Gol df arb- Shanno( BFGS) Met hod (A6. 却
Step 1. Choωe X" [Atl (i nitial metri c) , N
E" E2・E3: (tol erancefor stoppi ngcriteria)
Sct i = 1 (initializeiterationcounter)
Step2. The searchdi recti onis obtai nedas a sol uti onto
[A;]S, = - Vf ( X, )
川 ・ C l , S, ; dJ ( =α ,S,
αi is detenni nedby Mi n旬並 zi ngf (X川 )
Step3. If Vf ( X川 )TVf ( X川) 三 E3・converged
If I r( Xi+l) ー f ( X,)1 EI ・sωp (functi onnot chungi ng)
If !1X
dJ ( E2; stop (desi gnnot changi ng)
If i + 1 = N, stop(iterationlimit)
El se
Y = Vf ( X
) - Vf ( X,)
[B) =ー τ 一一
y' dJ (
Vf ( X, ) Vf ( X,?
[ C ] = 中
Vf(X;)' Sj
[Aj +d= [A;] + [B) + [c]
i f-ー i +1
GoTo Step2
Appl i c沼 tl on0 ' BFGS Met hod: The met hod is appl i cd to Exampl e 6. 1. Onl y
cssenti al computati ons areshown.
cpI XI =l s l ; 川 1) = 5.31お ; 凶 1 Vf ( X1) 七二 l
叫 耽 叶 協  
s t e p 3 Y =1 2 J r i ; m=l t ml


i 立 Eぷふ c l =│ 2 m- 3例 2 1
- 3. 0429 6. 5095
削 S 2 4 2 2 1 ; 町 =1お 44; X3=[ H 胸 =3. 0; 帆 ) =[ g]
The transl ati on of the BFGS al gori l hmi nto codc is left as an exerci se for the
student. It is r ecommcnded that Ul ecal cul ati onof thesearchdi recti oninStcp2 be
accompl i shed as a sol uti ontoa set of l i near equati ons (as listed) raUl er thani nverti ng
the matri x A. One other thi ng to note for Exampl e 6.1 is that if the pol ynomi al
approxi mati on was used for the one-di mensi onul process, the cal cul ati on of α-
woul d have becn exuct (why?).
S巴cond-order methods for unconstrai nedopti mi zati onarenot uscdb巴caus巴巴 val ual i on
of the Hessi an matri x duri ngeach iteration the Hcssi an matri x is consi dered to be
computati onal l y expensi ve. A second-order met hod wi th 白 e property of quadrati c
convergence is very i mpressi ve. An n-vari abl eprobl emcan convergc inone iteration.
As menti oned before, for real desi gn probl ems, where deci si ons arerequi redto be
made on Ul e di screte nature of some of 出 e vari abl es, 出 巴 exi stence of the first
deri vati ve, not to menti on second deri vati ves, is questi onabl e. Moreovcr, Ul e
quasi - Newton methods of 由巳 last t wo secti ons are abl e to effccti vel y functi on as
second-order methods as 出 ey approach 批 sol uti on- and they do not need the
esti mati onof secondderivatives. One second-order met hod is prl 巴 sentedherefor the
sake of compl eten巴ss.
I ndeed, there is onl y one basi c second-order techni que for unconstrai ned
opti mi zati on. It is based on the extensi on of the Newt on- Raphson techni que to
mul ti vari abl e probl cm. Many di ffcrent cxtensi ons are avai l abl e, but in this text , a
(i terati onI i mi t) stop If i + 1 = N,
El se
i ←i + 1
GoToSt巴p 2
Appl i cati on of Modi f i ed Newt on Met hod: Al gori thm(A6. 9) is used to sol ve
Exampl e 6.1. Onl y essenti al computati ons arei ncl uded:
f ( X
) =5.3125







s, =1 1 : u, = 1. 0: X? =1 1 一¥ ¥ ; α1=1. 0; Xヮー¥; ¥
1 1. 5000 l ' U ot - J . v,   - 1 2 1
Asexpect ed onl y a si ngl ei terati onwasnecessary toobtai n thesol uti on. Transl ati on
of theal gori thmi nt oMATLAB code isonce agai n left as an exerci se.
converged f ( X
) = 3.0;
Three addi ti onal exampl es arepresentedin出 is secti on. Wi t h many of thenumeri cal
techni ques al ready pr ogr ammed, obtai ni ng the sol uti on is a matter of j udi ci ous
appl i cati onof thenumeri caI procedures. Thi s secti onal sopresents some creati veuse
of the methods and expl ores nonquadrati c probl ems. 百 le frrsl exampl e is 出 e
Rosenbrock probl em[9]. 百l I s exampl e was among those created to cha11eng巴 白 e
numeri cal techni ques for unconstrai nedmi ni mi zati on. It is al sosometi mes referr芭 dto
as thebananafuncti on. I f you have a copy of th巴 Opti mi <.ati onTool box f romMATLAB,
you wi1l see 山 is as part of 出巴 tool box demo. You wiJ1 al so note 山 at the tool box
contai ns many of 出巴 techni ques devel oped inthis secti on. The second exampl e is a
sol uti ontoa nonl i near two-poi nt boundary va¥ue probl emthat isdue 10由 eNavi er-Stokes
叫 uati ons descri bi ng f l owdue to a spi nni ng disk. 百 1巴 last is an unusua¥ data fi 凶 ng
exampl e usi ngBezi er curves. Once agai ntheOptirnizationTool box presents its versi on
of thedatafittingexampl e when runni ngoptdemo at theMA TLABprompt
Exampl e 6. 2- Rosenbr ock Pr obl em
f(xl ・X2) = 100(x2- xT)2+ (1 - XI)2
-2::;;xl ::;;5;
S t e p 1 4 3 1 ;
抑 協 izl
6. 5. 1
The Rosenbrock probl emis
Mi ni mi ze
Subj ect to:




di rect extensi onrecast in出 egeneral al gori thmi c s釘 uct町 巴 (A6. 1) is presented. Once
agai nthegeneral probl emand speci fi c exampl e areas fol1ows.
(6. 1) f(X); [X]n
xl 三 Xi 三 xj t
Mi ni mi ze
(6.2) i =1,2,... ,11 Subj ect to:
f (X) = f(XI' X2) = 3 + (XI - 1.5x2)2+ (X2- 2)2
O::;;XI:豆 5; 0::;;x2::;;5
Mi ni rni ze
Subj ect to:
Modi f l ed Newt on' s Met hod: The Newt on- Raphson met hod, used for a si ngl e
vari abl e, sol ves the probl em中(X) = O. Thi s equati on can 問 present 白 e F OC for an
unconstrai ned probl emin one vari abl e (Chapter 5). The i terati ve change in 由 e
vari abl eis comput ed through
dx=ー単王 L=ーと主主
ゆ'(X) f"(x)
where f is th巴 si ngl e-vari abl e obj ecl i ve functi on. The mul ti vari abl e extensi on lo
computi ng a si mi l ar change in出 evari abl evecl or is
The ori gi naI Newt on- Raphson is not known for its robustness or stability and
Equati on (6.14) shares thesame di sadvantage. To control thedesi gnvari abl echanges
and tobri ngit under 由 eschem巴 of a1gori thm( A6.l) thel eft-handsi deis defi ned邸
the search di recti on v巴 ctor S f ol l owed by a standard one- di mensi onaI stepsi ze
computati on. Thi s is terrnedthemodi fi edNewt on method. The compl ete al gori thmis
as fol l ows.
ー 問 ー
Al gori 的 m: Modl f l ed Newt on Me的 od{A6. 9)
Step 1. Choose XJ , N
E" E2' E3: (tol erancefor stoppi ngcriteria)
Set i = 1 (initiali7..ei terati oncount巴r)
Step2. The searchdi recti onis obtai nedas a sol uti onto
[H(Xj )]Si = - Vf ( X
); [町 is 出 eHessi an
Xi叶 =Xi +α,/ Si ; AX=α .,SI
α" i is deterrni nedby Mi ni mi zi ngJ(X
Step 3. If Vf ( X川 ) TVf ( Xi+I) 三 E3' converged
If I r( X仲 1) ー f(Xi )1::;; EI' stop (functi onnot changi ng)
I f AX
s X::;;E2・ Sl Op (desi gnnot changi ng)
The si deconstrai nts areused for drawi ng contours. The sol uti ontothis probl emis
x ; = 1. 0; x ; = 1. 0; I = 0.0
Thc probl emis notori ous for 犯 qui ri nga l argenumber of i l erati ons for converg叩 ces.
Changes in desi gn are smal l as the sol uti onis bei ng approached. Here the Random
Wal k and Conj ugate Gradi enl methods areused toexami ne thesol uti on. Both these
methods are changed 10 drawappropri al e conl our levels. To avoi d c1utter, di sabl e
pri nti nglexl i nformati onon批 l i nes representi ngdesi gnchanges. Al so, si nc沼 山 ese
methods are run for 200 i terati ons, the f mal pri nti ng 10 the Command wi ndowis
avoi ded. The trac巴 of thedesi gnvari abl es on thecontour pl ot is provi ded as theoutpul
f roml heprogram, al ongwl thIh巴 linal val ues. As thefi gurc is bei ngdrawn, it is c1ear
that theprogress tothesol uti onis sl ow.
Sol utl on Usi ng RandomWal k Met hod: Fi gure 6.7 presenl s the resull of
executi ngRandomWal k. m for thls exampl e. Th巴 programi nvocati onand output f rom
曲 eCommand wi ndowis
-1 。 2 3
X 唱
Fl gure6.7 Randomwal k method: Exampl e 6.2.
4 5
>> RandomWal k( ' Exampl e6_2' , [3 2] ,200,l . Oe- 08,0,l ,20)
The pr obl em: Exampl e6_2
2. 0000e+002 1. 0000e+000 1. 0000e+000 1. 2327e- 007
Si nce th巴re are no stoppi ng criteria bui 1t i nto the met hod (was suggested as an
exerci se) , themet hod ranfor 200 i l erati ons (first val ue). 明 tefinal val uefor thedesi gn
V削 or and山 eobj ecti vefuncti onarevery i mpressi ve:
r andomwal k : x ; = 1. 0; x ; = 1.0; I = 1. 23e- 07
Sol uti on Usl ng Co呼 ugat eGradl ent Met hod: The i nvocati onand output f rom
theCommand wi ndowis
》∞nj ugat eGr adi ent ( ' Exampl e6_2' ,口 2) ,200,1. Oe四 08,0,1,20)
The pr obl em: E xampl e6_2
No. of i t er at i ons: 200
9. 6B07e- 001 9. 3700e- 001 1. 0222e- 003
The sol uti onis
conj ugate gradi ent : x; = 0. 96807; x; = 0. 937; r = 1. 0222e - 03
Thi s appears no mat ch for theRandomWal k met hod for 出 is i nvesti gati on. Fi gure 6.8
tracks 由 edesi gnchanges on thecontour pl ot.
6. 5. 2 Exampl e 6. 3ーThr ee- Dl mensi onal Fl ownear a Rot at i ng Di sk
Thi s exampl e represenl s another of Iheexact sol uti ons to出 c Navi er-Stokes equati ons
f romfl ui dmechani cs. The nonl i near two-poi nt boundary val ue probl emdescri bes a
vi scous f l owaround a flat di sk出 at rotates about 四 axi sperpendi cul 釘 toitspl ane wi 出
a constant angul ar vel oci ty. After sui tabl e 児 defmi ti on [10] the mathemati cal
descri pti on of the probl emis the fol l owi ngset of coupl ed nonl i near equati ons wi th
boundary condi ti ons at t wo poi nl s:
2F+H' =0
+ F' H - G
- F " = 0
(6. 16b)
2FG+ HG' - G" = 0
. . .

2 3
Fl gure6.8 Conj ugategradl ent method: Exampl e 6.2.
4 5
F(O) = 0; G(O) = 1; H(O) = 0
(6. l 7a)
F(∞) =0; G( 回 ) =0
(6. 17b)
Equati ons (6.16) areconvertedωstate spa回 f onn(s白 Chapter 5, Exampl e 5.3). The
desi gn vari abl es arE the mi ssi ng boundary condi 6orts on F' (O) ,G' ( O) and the final
val ueof thei ndependent vari abl e(inlieuof ∞). The obj ecti vefuncti onis 出 esquar吋
crror inthei ntegratedvaJ ue al thefinal condi ti on.
The Optl ml zati on Pr obl em
Mi ni mi ze f (xl' x
): YI (X
)2+ Y3(x
)2 (6. 18)
where thc slalc vari ab1es areobl ai nedf romthesol uti onof thc fol l owi nginitial val ue
probl em:
YI (O) O
Yi +Y尚一え
~     ,
Y3 ー Y4
Y4 2YI Y3+ Y,v' 5 Y4(O)
Y; -2YI Y5(O)

Thc statcequati ons (6.19) 釘 .c i ntcgral edusi ngMATLAB functi onode45 .
Sol utl on to the Pr obl em: The three-vari abl e probl emis sol ved usi ng the DFP
method. Exampl e6_3. mr芭 t ums the obj ecti ve functi on. It calls the functi on
Ex6_3_st at e. mwhi ch rcturns the system巴quati ons in state fonn. The call to the
DFP met hod wi th白 einitial desi gnvector of [1 - 1 8J wi th20 i terati ons of theDFP
met hod, and a tol erancefor thegol den secti onof 0. 0001 is shown bel ow. AJ so inthe
call is thei nput tol heupper bound cal cul ati on.
Usage: DFP (' E xampl e6_3' , [1 ・ 18] ,20,0. 0001,0,1,20) (6.20)
The final val ue for thedesi gnafter 20 i terati ons was
x;(ゆ ) = 0.5101; .t;(t.tか) = - 0. 6157; xj (併 ゆ =10. 7627; j = 5万 08e心 06 (6.21)
111e tol erance for the gol den secti on was onl y 0. 0001. h is possi bl c l hc obj ecti vc
cannot decrcasc unlil a smal l 巴 r tol eranceis speci fi ed.
The publ i shedval ues for thedesi gnvari abl es ar巴
x ; = 0.5101; 弓( 物) =- 0. 6159 (6.22)
whi ch is remarkabl y cl ose to I he val ues obtai ned by the DFP method. The vaJ ue of
xj is ILslLallynot speci fi edbUI is about 7. Si nce it is a substi tutefor i nfi ni ty, theJ arger
number shoul d not be a probJ em.
Exampl e 6.3 is not trivial probl em. The i ntegrati on is hi ghl y sensi ti ve to the
initial val ues. A reasonabl e starti ng poi nt is essenti al to prevent the i ntegrati on
f romgenerati ng NaN's (nol a number). The second desi gn vari abl e musl have a
ncgati ve si gn. The exampl 巴 i l l ustrates the appl i cati on of standard opti mi zati on
techni que to soJ ve a nonl i near di fferenti al system. A si mi l ar appJ i cati on can be
made to probl ems in sysl emdynami cs and opti ma1 control . It is essenti al to
understand白 紙 theal gori thms outl i nedhere arenumeri cal too1s 出 at transcendany
parti cuJ ar di sci pl i ne.
There arenumeri caJ techni ques 山 at address two-poi nt non1i near boundary val ue
probl ems mor e effi ci entl y. However, computi ng resources, especi al l yon a PC, arenot
a concern. Therefore, t11eproccdure adopted in由 is cxampl e is an acceptab1eapproach
for this cl ass of probl ems. Even the RandomWal k met hod is i mpressi ve in 出 is
6. 5. 3 Exampl e 6. 4 - Fi ttl ng Bezi er Par amet r l c Cur ves
The exampl e i ntroduces a nontradi ti 叩 al and an unusual curve fiuillg procedure.
Gi ven a set of datapoi nts a Bezi er curve 01 user-chosenorder is fil 10 thedata. Bezi er
curves are parametri c curves 自 制 御 官 a speci al case of uni f ormB-spl i nes. Bezi er
parametri zati on is based on the Bemstei n basi s functi ons. Any poi nt P on a
I wo-di mensi onal Bezi er curve (any parametri c curve) is aCl ual l y obtai nedぉ P(x(v) ,
y(v)) , where 0 S v S 1. 百 leactual rel ati ons are
P 尺(か例v 吟)=[ドx y 升) =エB 却Jんn肌即.1川(い v
JII.;(v) や)ゾ (1 -ν)". 1
whereB; 紅 E白 everti ces of 出 epol ygon that determi n巴 sthecurve. B; represent a pai r
of val ues in two-di mensi onal space. The order of thecurve is lI - t h巴 hi ghest power
in l he basi s functi ons. The acl ual computati ons of the poi nl Son I heCUl veω' e ellsier
usi ngmatri x al gebmand arewel l expl ai n巴dand document ed inRef erence 11. Fi gure
6. 9 a cubi c B巴 zi er curve. The fol l owi ng are some 01 ' l he most useful
prop凶 ies of thecurve as obs巴 rvedinthefi gure:
• The curve is compl etel y defi nedby thepol ygon obtai ncdby j oi ni ngtheverti ces
・τ'hedegree of 由 epol ynomi 必 defi ni ngthecurve is one less 出 anthenumber of
verti 白 s of thepol ygon.
. The firsl and last poi nts of thecurve arecoi nci dcnt wi ththefirst and last vertex.
Thc remai ni ng vertex poi nts do not l ypi cal l y I i e on thc curvc.
. The sl opcs al theends of thecurvc have the same di recti onas thesi des of the
pol ygon
. The curve is contai nedwi thi ntheconvex hul l of 山 epol ygon.
Inthis ex創 npl etheconveni ent matri x representati ons for thc curve arc used. They are
not di scussedhere, however. but areavai l abl c inthecodc. For 由 isexampl e. thedcsi gn
vari abl es arethei nsi depol ygon verti ces. For n = 5, thcrearc 6 ve凶 cω. The first and
last vcrtex areknown f romthedatapoi nts. Thi s l eaves 4 verti ces or 8 d巴 si gnvari abl es
f orn = 5.
Exampl e6_ 4. m: 百 lis m- fiI c is a stand-al one programto run thc cxampl c in this
sccti on. Typi ng Exampl e 6_4 al the prompt shoul d start thc program. Actual l y the

8 0 5

ー "
/ ,
a ノ '
, 、 / ,
,、 / i
f ¥ / ,
、 , . / ' 町
. 、 今 占
1 、 J 〆,.,
, 、 - - -
ー, 同 一 ,
, 、 ,
1 、
, 〆 、 ,
1 / " 、 ,
1 . / 、 l '
1 . / 、 .
1 / 、 J
/ 、 ,
, / 了
.' / C!I 1 /
, ノ ー 四 一 -
,/ 占 E3
' " ふ
B唱 φ
0.2 0.4
Fl gure6.9
x data
Acubl c B自 zl 日 rcurve.
exampl e is generi c as most deci si ons a陀 basedon白 etext file that contai ns thedata
tobe fi ued. 11calls on m叩 y m-fi l es. The Bezi er caIcul ati ons arefairlymodul ar. Thi s
programis not 凶 vi al. A mor e i mportant 隠 asonfor its i ncl usi onis 白 at,if understood,
it si gnal s a reasonabl e maturi ty wi th MATLAB programmi ng. Mor e i mportanl l y, it
exempl i fi es 出 eway toexpl oi l the i mmense power of bui l t-i n MATLAB suppo目. The
author encourages theuser toexert somc effort traci ngtheprogress of thecal cul ati ons
血 rough白 ecode. A 101of programcontrol is based on出 euser' s choi ce concemi ng
the order of the Bezi er curve. Al so the initial val ues for the desi gn vari abl es are
generated automati cal か" based on thedata10 be fitted. The functi oncal l i ngsequence
used in出 is exampl e can be bettcr understood by thefol l owi ngstructure:
Exampl e6_5_3. m
CaUs ui getfi l c topi ck up山 e(* .txt) 日Ic 山 at contai ns t wo- col umn data白 at wi l l be
fit (Ex6_5_3. txt)
τ'hrows up an i nputdi al ogbox toobtai ntheorder of fit f romtheuser (0 = 5)
Cal l s coef f . mwi ththeorder i nformati on
Cal l s combl nat l on. m- cal cul ati on01 ' combi nati on for Bcrnstei nbasi s
Cal l s Factori al . m- caIcul ates factori al of ani nteger
Cal l s Curve_f i t. m whi ch handl es theopti mi 祖 tion
CalI s DFP
CalI s Bez一Sq_Er r . m一 cal cul ati onof 対 functi on
Cal l s gri悶' sdf伽un. m- cal cul ati onof deri va低叫tiv巴 s
Cal l s Gol d_Sect i on一nVa 創 r ι川' . m 官首官 - 1 D stepsi ze
Cal l s Upper Bound_nVar . m- bracket themi ni mum
Si nce there are ei ght desi gn vari abl es, a11 of the i nformati on except for iteration
number. convergence criteria, and 白 巴 obj 巴cti ve functi on is suppressed f romthe
Command wi ndowtoprevent scrol l i ngand loss of comprehensi on.
百 lesi gni fi cant el ements in出 is codear芭 as fol l ows:
• Pi cks up白 edatafilethrougba filesel ecti onbox.
• Uses an i nput di a10g box toquery theuser on theorder of thec町 ve.
• Theord巴r determi nes thesizeof thecoeffi ci ent matri x A and it is automati ca11y
generat凶 .
• The number of desi gn vari abl es is automati cal l y determi ned. Once agai nit is
basedon曲巴 order of thecurve.
・ The initial val ues for thedesi gnareautomati cal l ygeneratedbasedon theda胸
tobe fi t .
・TheDFP method is usedtofi ndtheopti mum.
• The obj ecti vefuncti onis computed inthegeneri c fiI e Bezi ecSq_Er r . m. The
dataspeci fi c to出 e巴xampl earepassedthrough出 eMAT L AB gl obal statement .
・The ori gi nal dataset and白 巴 Bezi er curve fittingthedataar, 巴 di spl ayedat the
・Theobj 配 ti ve functi onand th巴 convergencedatafor eachiterationarepri nted
totheCommand wi ndow.
ObJ ectl ve Functl on: Th巴 obj ecti ve functi onis the1開 st squarederror over all of
thedatapoi nts between出 eactual dataand thefittedBezi er curve. It is expr' 巴ssedas
Mi ni mi ze:
I I Da佃
六X): エIy, 帥 -YBT
1<=1'" . J
The fol l owi ng i nformati on is copi ed f romthe Command wi ndow (the file is
Ex6_5_3. txt and出 eorder is 5):
持 Bxampl e6_4
i t er at i on number : 1
gr adi ent l engt h squar ed:
obj ect i ve f unct i on val ue:
i t er at i on number : 2
gr adi ent l engt h squar ed:
5. 0001
0. 5302
5. 6890
obj ect i ve f uncヒi on val ue: 0. 2497
i t er at i on number : 3
gr adi ent l engt h squar ed: 0. 0546
obj ect i ve f unct i on val ue: 0. 1830
i t er at i on number : 4
gr adi ent l engt h squar ed: 0. 1322
obj ect i ve f unct i on val ue: 0. 1605
i t er at i on number : 5
gr adi ent l engt h squar ed: 0. 0573
obj ect i ve f unct i on val ue: 0. 1467
i t er at i on number : 6
gr adi ent l engt h squar ed: 0. 0374
obj ect i ve f unct i on val ue: 0. 1437
i t er at i on number : 7
gr adi ent l engt h squar ed: 0. 0071
obj ect i ve f unct i on val ue: 0. 1433
i t er at i on number : 日
gr adi ent l engt h squar ed: 0. 2793
obj ect i ve f unct i on val ue: 0. 1415
i t er at i on number : 9
gr adi ent l engt h squar ed: 0. 4337
obj ect i ve f uncヒi on val ue: 0. 1401
Fr omthedataabove it appears that thee町 or is around 0. 14and is difficult torαi uce
further. It al soappears thereducti onintheobj ecti veis accompani ed by thei ncrease
inthegradi ent. after a certai nnumber of is movi ng away f romsati sfyi ng
theFOC. The best val uefor theF OC is about 0. 007and starts toworsen wi thfurther
iteratI ons. It is qui tepossi bl e白 紙 numeri cal i naccuraci es may be a probl emas there
are 巴xtensi ve computati ons i nvol ved in each iteration. It is al so l i kel y 白 at 白 e
numeri ca1gradi ent computati on may need tobe refi ned. Currentl y出 egr叫 ients 釘 e
computed usi ngforwardfinitedi fferencewi tha stepof O. OO1. These areparameters
tltat n田 d to b巴 巴 xpl ored in opti mi zati on as there is no uni versal set of val ues
for all cl asses of probl ems.
Fi gure 6. 10illustrates 出 eori gi na1data叩 dtheBezi er curve a白 er 出 elast iteration.
The pol ygons arenot shown as our effort is togen巴 ratethec町 ve. τhefit is defi ni tel y
acceptabl eand thereis probabl y little白 eBezi er curve can do about a coupl eof poi nts
that arenot very smoothl y l ocated. It is probabl y thesepoi nts that do not a110wthe
obj ecti vefunctI on todecreasefurther wi thout i ncreasi ngth巴 gradi ent . This concl udes
another i nnovati ve use of unconstrai ned opti mi zati on. 百 le author has used these
curves toopti mal l y desi gnairfoils for l ow-speedflight .
' "





Not e on F OC for Unconst rai ned Mi ni mi zati on: 明 白 exampl e has alerted us
tothequesti onof FOC Vf = O. If anal yti cal computati onof thederi vati veis possi bl e
for a probl em. thenFOC has the likeliltoodof bei ng met. ln desi gn probl ems like
Exampl e 6. 4 , where theel ements of thegradi ent vector have thef orm


h 。

唱。‘・ 4
可) o. e..._ 0
σ- e ...旬。
三 三 土 9
5. Fl etcher, R., and Reevcs, R. M. , Functi on Mi ni mi zati onby Conj ugal e Gradi ents. The
Comput er J oumal . Vol . 7, pp. 149- 180, 1964.
6. Davi don, W. C. , Vari abl e Metri c Methods for Mi ni mi zal i on, U.S. Al omi c Energy
Commi ssi on Research and Devel opment Reporl No. ANL - 5990, Argonne Nati onal
Laboratory, 1959.
7. Huang, H. Y. , Uni fi ed Appr oachωQua命凶caJ l y Convergent A1gori thms for Functi on
Mi n 回 国 .tion , J oumal of Op肋 u wtI on刀 ' aoryand Appl i cati o/ls, Vol. 5, pp. 405- 423, 1970.
8. Vanderp1aats, G. N. , Nl /l neri cal Opti mi zati o/l Techni ques f or Engi neeri ng Desi gn,
McGr aw- Hil1, New York, 1984.
9. Hock , W. , andSchi ttkowski , K., Test Exampl es f or Non Li near Programmi l!g Codes,
LectureNotes InEconomi c and Mathemati cal Systems, 187, Spri nger-Verl ag, Berl i n,
10. Schl i chti ng, H. , BoundaηI Layer Theory, McGr aw- HilI, New York, 1979.
11. Rogers, G. F. , andAdams, J. A. , Mathemati cal El ementsf or Comput er Graphi cs, 2nd ed. ,
McGr aw- HilI, New York, 1990.



- :




a 守




Appl y theRandomWal k met hod toExampl e 5.2.
Sol ve usi ngtheRandomWal k method:
Mi ni mi ze !(X. ,x2) = x1- 2 み 2 x 1 ~   2 x . +4
Appl y thePattemSearch met hod toExampl 巴 5.2.
Sol ve usi ngthePattemSearch m巴白 od:
Mi ni mi ze !(.x.,x2) = x1- 2x1X2 + xi +.q-2x. ... 4
Modi 均 , the Patもe mSearch so that programcontrol p紅 白 neters for contour
pl otti ng, gol den secti on, and upper bound cal cul ati on can be set by 由 巳 国 er.
I ncl ude曲巴 prompt and al l owfor defaul t val ues incase theuser deci des not to
takeadvantage of it .
Transl atethePowel1met hod i ntoworki ng MATLAB code. Veri fy thesol uti onto
Exampl es 6.1 and 5.2. Start f Tomseveral poi nts and veri fy that the number of
cycl es toconverge is 出 esame.
Veri fy白 紙 曲 esearchdi recti ons areconj ugate wi th児 spect totheHessi an matri x
for Exarnpl es 6.1 加 d5. 2.
Sol ve usi ng Powel l ' s m巴山 od 叩 d 白 at veri fy any t wo search di recti ons are
co吋ugatewi 白 respect totheHessi an matri x at thesol uti on:
Mi ni mi ze !(.x.,x2) = x1- 2x1X2 + xi +.q-2x. + 4
Sol ve Exampl e 5. 2usi ngtheSteepest Descent, Conj ugate Gradi ent, DFP, and
BFGS me曲 ods.
6. 10 Sol v巴 usi ng the Steepest Descent , Conj ugate Gradi ent, DF P, and BFGS
methods 自己 probl em
附 凶 凶 ze f(x. OX2) = x1- 2x1X2 + xi +.q-2x. + 4
6. 6
6. 8
6. 9
6. 7
6. 3
6. 4
6. 2
and whi ch are best comput ed numeri cal l y, some speci fi cati ons areneeded regardi ng
deri vati ve computati ons if they aretobe used for enforci ngthenecessary condi ti ons.
x ・data(readf romflle)
Orlglnal dataand Bezl er curve.
怖い i )
1. Fl etcher. R叫 Practi cal Metl l Odsf or Opti mi wti on, Vo1.1, Wi l ey, New York, 1980.
2. Brent, R. P. , Al gori thms f or Mi l l i mi zati on wi thout Deri vati ves, 加 問 ti ce-Hal l ,
Eng1ewood Cliffs, NJ , 1973.
3. PowelJ. M. J. D. , NOl l l i near Opti mi Wl i on, Academi c Press, New York. 1981.
4. Fox, R. L.. Opti mi zati ol l Methodsf or El!gi neeri ngDesi gn. Addi son-Wesl ey. Readi ng,
MA. 1971.
Fl gure6. 10

6. 11 Veri fy theval ues in S巴cti on 6.3.3 usi nga cal cul ator. Veri fy if thematri x A at
出 esol uti onis 山 ei nverseof theHessi an.
6. 12 Veri fy if the matri x A at the sol uti on is the i nverse of the Hessi an for DFP
me出 odfor
Mi ni mi ze f(XI ,x2) =x1- 2xtx2+xJ x ~ - 2x1 + 4
6. 13 Devel op 山 e BFGS met hod i nto MATLAB code and veri fy the caJ cul ati ons in
Secti on6.3.4.
6. 14 Devel op the Modi f i ed Newt on mct hod i nto MATLAB code and appl y it to
Exampl e 5.2.
6. 15 How wi l l you i ncorporate白 esi deconstrai nts i ntothecod巴 for aJ l of thevari ous
methods inthis secti on? I mpl emcnt t heminthc numeri cal procedures.
6. 16 Sol ve Exampl es 6.2, 6.3. and 6.4by one other m巴thodof thesecti on.
6. 17 I denti fyand sol vea systemdynami cs probl emusi ngany met hod ofthi s secti on.
6. 18 I denti fyand sol vc your own curve fit probl cm.
百 lis chapter expl ores aJ gori thms/ methods that handl e 出 e gcneral opti mi zati on
probl em. Bot h equ叫 ity and i nequaJ ity constrai nts are i ncl uded. For an engi neeri ng
probl 巴 mthis wi l l i nvol ve bri ngi ngseveral nonl i near rel ati ons i nto thedesi gn space.
Thi s is certai n to i即 応 ase the degree of di ffi cul ty in obtai ni ng 出 c sol uti on. For the
desi gner 山 ere is an addi ti onal burden of bei ngmor e attenti veto thedesi gnchanges
and thecorrespondi ngnumbers tocoax thesol uti onif themathemati cal defi ni ti onof
theprobl emis parl i cul arl ysevere. InaJ l of l heseprobl ems l herearet wo outcomes that
出巴 al gori thms seek to accomp¥ i sh. 百 le first is to ensure 曲 at the desi gn is feasi bl e
(satis自由aJ l constrai nts) and thesecond伽 t it is opti mal (satisfies the Kuhn吊 l cker
condi ti ons). Whilethefocus is on determi ni ng曲 esol uti on, inti mes of di ffi cul ty, it is
essenti al 10remember Ihatfeasi bi l i 砂 is mor e i mportant 出 at opti mal i ty. Al so unstated
in 出巴 rest of 白 echapter is 山 at 出 eopti mal sol uti on must be feasi bl e. Two di sti nct
approaches wi11 be used to handl e the constrai ned opti mi zati on probl em. The first
approach is termed出 ei ndi rect approach and sol ves 山巴 probl emby transformi ng it
i nto an unconstrai ned probl em. The second approach is 10 handl e 出 e cons凶 ints
wi thoul 回 nsf ormati on- thedi rect approach.
The i ndi rect approach is an expressi onof i ncremental devel opment of thesubj ect
totakeadvantage of 出 ecurrent methods. For ex創 npl e,it l everages theDFP met hod
to handl e constrai ned opti mal probl ems. Two i ndi rect methods are presented. th巴
Exteri or Penal ty Functi on (EPF) met hod, and 出巴 Augment ed Lagrange Mul ti pl i er
( ALM) method. The di rect approach handl es theconstrai nts and theobj ecti ve together
wi 山 outany凶 nsformati on. Four methods蹴 p陀 sented. 百 lemethods in出 i sbookare
Sequenti aJ Li near Programmi ng (SLP) , Sequenti al Quadrati c Progr ammi ng ( SQP),
General i zedReduced Gradi ent Met hod ( GRG) , and Sequenti aJ Gradi ent Restorati on
Al gori thm( SGRA) .
The standardformat of thenonl i near programmi ng probl em(NLP) isreproducedhe陀
for conveni ence:
Mi ni mi ze f (x" X2, • • • • X
) (7.1)
Subj 配 t to: hk(x" X2' . . . • X
) = O. k = 1,2, • • • .1 (7.2)
gj (X" X2. ・・. . X
) 0, j = 1. 2, • • • . m
i =I . 2. . . . . 1t
Invector notati on
Mi ni mi ze f(X). [Xl "
Subj ect to: [b(X)]f = 0
[g(X)] ", S 0
X1nwSX: ; ; X"1'
For this chapter thefol l owi ngi ndi ces arercserved: It. number of variablω; 1. number
of equal i tyconstrai nl s; m. number of i nequal i ty constraints. Many of Iheal gori thms
and methods present a reasonabl e level of difficulty. A two-vari abl e prob1em(for
graphi cal i1l ustration). wi th a single equal i ty and i nequal i ty constrai nt is used to
illustrate the vari ous algori 出 ms. Al1 of the functi ons are nonl i near a1though the
constrai nt functi ons have si mpl egeometri c shapes.
7.1.1 Pr obl emFor muJ at l on- Exampl e 7.1
Exampl e 7.1 is a si mpl e mathemati cal formul ati on. It does not 陀 present any
engi neeri ngprob1em. It is cons凶 ctedtohave a sol uti onof xj = 1叩 dxi = 1 for 恥
∞nstrai nedprob1emand xj = 0. 8520and xi = 0. 8520for theuncons回 i nedprob1em.
The two constrai nt functi ons have si mpl e non1i neari ty. The叫 uality constrai nt is a
ci 四 lewhi l ethei nequal i tyconstrai nt is ane¥li pse.
Mi ni mi ze f (x" X2): x1-     + xr + XI - 2x1 + 4 (7.9)
Subj ecl l o: h(X"X2): xr +xi - 2=0 (7. ¥Oa)
g(x" X2): 0.25xT + 0. 75 - 1 S 0 (7. l Ob)
O:5xl :55; 0:5x2:55 (7. ¥Oc)
3 4
F1gure7.1 Graphi c剖 solu1ionfor Exampl e7.1.
Fi gure7.1 di spl ays thegraphi cal sol uti on to出巴 probl em. It is cl ear f romthe figure
that thesol uti onis at xT= 1. xi = 1 (why?). The val ueof thefuncti onis 3. A si mpl e
a即 roachtothesolutionof theprobl emis 10u印 刷 uati on(7.l Oa) 10el i mi nateone of 曲 e
variables 針。 m山 eprobl em由 erebyreduci ngit toa single・variableconstrai nedprobl em.
百 lis is not done he陀 but it is a good exercise. FI g7 _l . m
will createthefigure.
7. 1. 2 Necessar y Condl t i ons
引 le necessary condi ti on-or the Kuhn- Tucker (KT) condi ti ons for 由 e
probl em- was devcl oped in Chapter 4. It is based on the mi ni mi zati on of 出 E
Lagrangi anfuncti on(円. The condi l i ons are
Mi ni mi ze
F(xl .. 'X
' AI ... Af' sI ... s
) = f (xp

) +λI hl +…+λ 内 +
Pぱ 1+・ ・ ・ +s", g", (7.11)
1Ft!田 10be downl oadedfrom白 ewebsi 旬 areindicaledbyboldfacesans serif Iype.

(7. l7c)
Equati on s巴t (7.17) is sol ved symbol i cal l y in MATLAB. Thcre are t wo sol uti ons tha!
sati sfy thesi dc cons回 ints (7.l 0c)
xi +xi - 2=0
subj 配 tto山 econs汀 aints. 百 le陀 aren + 1 + m unknowns intheLagrangi an. The same
number of equati ons is requI I 吋 tosol ve出巴 probl em. Theseareprovi ded by theF OC
or Kuhn- Tucker condi ti ons. Here' l equati ons areobtai nedas
j = 1. 2. . . . . n
dF dr • dh. . • dh, • 0 d81. . 11 d8m

= +λ 」 +ー ・ + L - 4 1」 +ー ・ +P,J S7=0;
axi a x a I a x t t a x i t axi dxt
x; = 1. 4142;
x; = 1. 0676; λ ・=- 0. 1217
λ・=0 X; =0;
x ; = 0. 9275;
80th of these vi ol ate 出 e i nequal i l y constrai nt. I n Fi gure 7.1. the first sol uti on
defi ni tel y do凶 but l hesecond is a liltledi ffi cul t 10see. Zoomi ng shoul d hel p. Case a
docs not provi de a sol uti on.
1 equati ons areobtai neddi rectl y t1lfOugh由 eequal i tyconstrai nl s
k = 1. 2. . . . . 1
hi x,・X2.... . xn)=O;
百 lis establ i shes four equati ons as g =O Case b:
8j <0 then
m cquati ons areappl i cdthroughthe2
sj8j =O→if 鳥=0
(7. 18a) 4xt 内 , λ , +0.5s X
(7. 18b) - 2xf +2x,x
+2入X2+1. 5
島 >0 l hen if 8j =0
(7.18c) xi - 2 =0
The K T condi ti ons appl y onl y if thepoi nts areregul ar (seeExampl e 4.3).
(7. 18d)
Equati ons (7.18) 脱 sol v叫 symbol i cal l y and l he onl y sol uti on sati sfyi ng Ihe si de
constr剖 nts ,s
0. 25 万 - 1 =0
s " =O
whi ch was expected(for thedesi gnvari abl cs not l hemul ti pl i crs). The val uc of s" = 0
is not expecl edas lhis is anacti veconstrai nt. The K T condi ti ons can be consi dered10
be weakl y sal i sfi edas s is not strict1y posi ti ve. As inChapter 6. wherever possi bl e.
出 eK T condi ti ons areused tocheck for convergence.
i.." =- 0.5000; 弓 =1. 0; xi = 1. 0;

Appl l cati on 0' the K T Condi ti ons: 百 eLagrangi an
F( x"

• λ - 2xT x
+xT+x, -2x, + 4 +
λ   ヰ 戸   xi + 0. 75 - 1)
(7. 16b)
Threeequati ons areset up through
E; =44- 4xt X2+2XI +xi - 2+2hi +05PXl =0
ァ = ー 2xT+2x,x
+2 i.. x・2+ 1. 5 s X
= 0
El ement s of a Numer i cal Technl que
There is a di sti ncI di 仔 erence in the numeri cal techni ques wi th i ncorporati on of the.
constrai nl S. even if l hei deaof usi nga searchdi recti onvecl or is still useful . The maj or
di fferences inthea1gori thms of theprevi ous chapter was山巴 waythesearchdi recti ons
were establ i shed. To understand the necessary changes. the general approach for
unconstrai nedopti mi zati onas seenintheprevi ous chapter was:
h =xi - 2 = 。
Wi t h this i nformati onEquati ons (7.16) become
The fourl hequati onis deve10ped through
s =O Case a:
General Al gorl t hm伺 6. 1)
Step 1. Choose X
(7. 17b)
4x1 - 4 X, x

十 λ ,
- 2xT + 2x, x
+ 2λ x
Step 2. For each i terati oni
Detenni ne searchdi recti onvector Sj
(It woul d be ni ceif theobj ecti vedecreasedal ongthis di recti on)
Step3. Cal cul ateA Xj =α ,Sj
Note: A Xj is now a functi onof thescal ar ( Xj as Sj is known f romStep2
( Xj is cal l ed白 estepsi zeas it establ i shes 白 el engthof AXj
a; is deterrni nedby Mi ni mi zi ngf(X; +I )' where
Xj +1= X, + A Xj
As theabove steps 釘 erevi ewed, inStep 2 a di l emma n民 ds tobe resol ved: Shoul d
the searchdi recti onSj decrease出 eobj ecti ve functi on0' shoul dit attempt tosati sfy
theconstrai nts? (It is assumed出 at it cannot do both though very oftenit wi l l.) Si nce
feasi bi l i ty is mor e i mportant 出 an opti mal i ty, it is 回 si er to i denti fy the latter as a
si gni fi cant i nfl uenceindeterrni ni ngthedi recti on. Thi s is mor e di ffi cuIt thanit sounds
as each constrai nt can have its own favori te di recti on at the current poi nt . Such
occasi ons call for a trade-off and toseeif it is possi bl etonegoti atea decrease inthe
obj ecti vefuncti onas wel l . It is cl ear that Step 2 wi l l i nvol ve i nvestment of effort and
some sophi sti cati on.
Step3 is not far behi nd inthe児 qui 回 ment f orsp田 ial handl i ng. Once thedi recti on
is establ i shed (Step2) , what ki ndof stepsi ze wi l l be acceptabl e1Several resul ts are
possi bl c. First, thc obj ccti ve functi oncan b巴 decreased al ong the di recti on. Second,
current acti ve constrai nts (i ncl udi ng 由巳 equal i ty constrai nts) can become i nacti ve,
vi ol ated, or can still p陀 serve白 eacti vc state. Thi rd, current i nacti veconstrai nts can
undergo a si mi l ar change 01 " state. Fourth, probabl y most i mportant, current vi ol ated
cons釘 aints can become acti veor i nacti ve. It is di ffi cul t toencapsul atethis di scussi on
in a generi c al gori thm. Somc al gori thms combi ne Steps 2 and 3 i nto a si ngl e step.
Some di vi dethei terati oni ntoa feasi bi l i ty componenl and an opti mal i ty component .
Inessence thegeneri c a1gori thmfor 白 is chapter, whi ch is not parti cul arl y凶 eful ,is:
Generl c Al gorl t hm(A7. 1)
Step 1. Ch∞seX
Step2. For each i terati oni
Detenni ne searchdi recti onvector Sj
Step3. Cal cul ateAX; =αj Sj
Xj+1 = Xj + A Xj
These mc出 ods were devel oped to take advantage of codes 白 at sol veunconstrai ned
opti mi zati on probl ems. They are al so referred to as Sequenti al Unconstrai ned
Mi ni mi zati on Techni ques ( SUMT) (1). The i deabehi nd山 eapproach is torepeatedl y
call the unconst,ai 1/ ed opti mi zati o1/ al gori thmusi ng the sol uti on of the previ ous

iteration. 百 le uncons住 ai ned al gori thmitself executes many i terati ons. Thi s woul d
r叫 ui 陀 a robust unconstrai ned mi ni mi zer to handl e a l arge cl ass of probems. The
BFGS met hod is robust and i mpressi ve over a l argecl ass of probl ems.
A preprocessi ng l ask i nvol ves tr叩 f ormi ng the constrai ned probl emi nto an
unconstrai ned probl em. Thi s is l argel y accompl i shed by augmenti ng the obj ecti ve
functi on wi th addi ti onal functi ons 白 紙 refl ect 白 e vi ol ati on of the∞nstrai nts very
si gni fi cantl y. These 釦 ncti ons are referred to as 由 e penal , η f附 cti ons. 百 le印 Wお
si gni fi cant acti vi ty in 山 is 紅 白 at one ti me whi ch l ed to enti re farnilies of di fferent
types of penal l y functi on methods [2). The firsl of these was the Ex1eri or Penal ty
Functi on (EPF) method. The first versi on of ANSYS [3] 白 紙 i ncorpora1ed
opti mi zati onin its finiteel emenl programrel i edon the EPF. The EPF (presentedin
白 e next secti on) had several shortcomi ngs. To address those 出 e I nteri or Penal ty
Functi on (I PF) methods were devel oped l eadi ng 10 the Vari abl e Penal ty Functi on
( VPF) methods. I n 白 is text . , onl y the EPF is addressed l argel y due to academi c
i nterest . I nvi ewof 白 eexcel l ent perf ormance of 出巴 di rect methods, these methods
wi l l probabl y not be uscd today for conti nuous probl ems. They arc once agai n
i mportant in gl obal opti mi zati orl techni ques for constrai ned probl ems. The second
me白 odp民 sentedinthis secti on, l heAugment ed Lagrange Met hod ( ALM) , is l hebesl
of the SUMT. lts exceedi ngl y si mpl e i mpl ementati on, its qual i ty of sol uti on, and its
ability to generate i nfonnati on on the Lagrange mul ti p1i ers al l owit to seri ousl y
chal l enge由巳 di rect techni ques.
7. 2. 1 Exterl or Penal t y Funct l on ( EP町 Met hod
The transformati on of thc opti mi zati on probl em(7. 1)ー (7.4) 10 an unconstrai ned
probl emis made possi bl e through a penal ty functi on fonnul ati on. The transf onned
unconstrai nedprobl emis:
Mi ni mi ze F( X, '11' 'g) = f (X) + p(X, '11, ',) (7.19)
(7.4) X: :S; X/ :S; ヰ i = 1, 2, • • • , 1/
where P( X, 'h, 'g) is thepenal ty functi on. '11 and 'garepenal ty constants (al socal l ed
mul t与Jlie,s.)
The penal ty functi onis expressedぉ
p( X, ' h' ',,) = ' h1 L. "k(X)21+ '81ヱ( m阻 {O・め (X) })21 (7.20)
1"'1 l t1
InEquati on(7.20) , if 白 eequal i tycons回 ints arenot zero, their val ue gets squared and
mul ti pl i edby thepenal ty mul ti pl i er and thengets added to白 eobj ecti vefuncti on. If
th巴 i nequal i ty constrai nt is invi ol ati on, it toogets squared and added 10出巴 obj ecti ve
functi onafter bei ngampl i fi cdby出巴 penal ty mul ti pl i ers. Ina sense, if theconstrai nts
areno1satisfied, then由 eyarepen必i zed,hence thefuncti on' s n創 ne. It canbe shown白 紙
thc 四 l sfonncdun∞ns凶 n吋 probl emsol ves 出 eoriginal ∞m凶 nedprobl emas the
multipliers r", rg approach ∞. In order for P to remai n finile at 山 ωe val ues of 山 e
multipliers (1即 dcdfor a validsolution) ,出 econstraints must be satistied. Incomputer
i mpl ementati on 血 is limit is 陀 pl aced by a large val ue ins飽 ad of ∞. Another facet of
∞mputer i mpl ementati onof this method is白 紙 alargeval ueof themultipliers at the白百 t
ilerationisbound tocreatenumeri cal diffi 印 lties. 百 l esem叫 tiplier百 arestartedwi thsmal1
val ues 田 du凶 a凶 gωme凶 回 lly明 白 回 chiteration. 百 eunconslraI n吋 飽 chni que,for
ex胴 体 DFP,wi1¥ sol ve Equati on (7.19) for a known val ue of 恥 multiplicrs. The
solution剛 山 ncdfromtheDFP canbe consi der凶 as a functi onof thcmultiplier andcan
be thought of as
X. = X.(r"・r
) (7.21)
The SUMT iteration i nvol ves updati ngthemul ti pl i ers and出 einitial desi gnvector
and cal l i ng白 eunconstrai nedmi ni mi zer agai n. In出巴 al gori thmit is assumed that 山 e
DFP method is used (al though any method f romChapter 6 canbe used, BFOS is
Al gorl t hm: Exterl or Penal t y Funct i on ( EP, 丹 Met hod伺 又 2)
Step 1. Choose X
, N. (no. of SUMT iterations) , N
(no. of DFP iterations)
E;'S(for convergence andstoppi ng)
, r! (initial penal tymul ti pl i ers)
Ch. c/I (scal i ngval uefor mul ti pl i ers)
q = 1 ( SUMT iterationcounter)
Step2. Cal l DFP tomi ni mi ze F( X
, r%,
OutpUl: xq.
Step3. Convergence for EPF
If hk = 0, for k = 1, 2. . . . , 1;
If gj :5O. for j = 1. 2. . . . • m;
If all si deconstrai nts aresatisfied
Then Converged, Stop
Sl oppi ngCriteria:
d. F = F
- F 0-" d X = X
* - X( q- I ) *
q • q-- I
If (d.Fl :5EI: Stop(functi onnot changi ng)
El seIf d. X
d. X:5EI: Stop(desi gnnot changi ng)
日 self q = N.: Sl Op( maxi mumiterations reached)
Conti nue
←   r: ←r1・Cg
00 toStep2
潤 1
The EPF is very sensi ti vetothestartingval ueof themul ti pl i ers and toth巴 scal i ng
factors as well. Di fferent probl ems respond favorabl y to di fferent val ues of the
mul ti pl i 巴 rs. It is recommended that theinitial val ues of themul ti pl i ers be chosen as
theratioof theobj ecti vefunclI on tothecorrespondi ngtennin山 penal tyfuncti ona1
theinitial desi gn. Thi s ensures 出 at boththeobj ecti vefuncti onand theconstrai nts are
equal l y i mportant in deteロni ni ng the changes in the desi gn for 白巴 succeedi ng
One reasonfor thetennExteri or Penal ty is that at theend of eachSUMT iteration
出巴 desi gn will be il 砕 asi bl e (until the sol uti on is obtai ned). Thi s i mpl i es that 出 c
method detenni nes desi gn val ues 白 紙 areapproachi ng白 efeasi bl eregi onf romthe
outsi de. Thi s is a seri ous drawback if themethod fails p陀 maturel y,as it will oftendo.
Thc i nformati ongeneratedsofar is val uel ess as thedesi gns werc never feasible. As
seenintheexampl e bel ow. theEPF severel yi ncreases出 enonl i neari tyof theprobl em
creati ng condi ti ons for the me出 od to fai l . Il is expected 白 紙 the i ncrease in
nonl i neari ty is bal anced by a c¥oser starting val ue for the desi gn, as each SUMT
iterationstarts c¥oser tothesol uti onthan出 cprevi ous one.
lnthefol l owi ngtheEPF is appl i edtoExarnpl e7.1througha series of cal cul ati ons
rather 出 an through the translationof 山 eal gori thminto MATl AB code. Thereare a
coupl eof changes wi threspect toAl gori thm(A7.1). To resol vethepenal ty functi on
wi threspect to 白 ei nequal i ty constrai nt , the constrai nt is assumed to al ways be in
vi ol ati on80出 at 出 ereturnf romtheml l xfuncti onis 白 econstrai nt functi onitsel f. Thi s
wi1l dri ve the i nequal i ty constrai nt to be acti ve whi ch we knowto be true for this
exampl e. Numeri cal i mpl ementati on as outl i ned in the al gori thmshoul d al l ow
detenni nati on of i nacti ve constraints. I nstead of numeri cal i mpl ementati on of the
unconstrai nedprobl em, ananalytical sol uti onis determi nedusi ngMATLAB symbol i c
computati on. Exampl e 7.1 is reproducedfor conveni ence.
Mi ni mi ze f(Xj oX2): +4 (7.9
Subj ectω: h(x(, X2): (7. l Oa)
g(x( , 沙 万 :50 (7.LOb)
o :5Xl :55; 0 :5X2:55 (7.l 0c)
Fi gure7.2is thccontour plot of thetransformedunconstrai ncdfuncti onfor val ues of
rh= 1 andら =1. The i ncreaseinnonl i neari tyis rcadi 1y appar宅 nt . Fi gure7.3is theplot
for rh= 5 and rg= 5. Handl i ng山 esefuncti ons numeri cal1y evenwi ththeBFOS wi l1
be a consi derabl echa11enge. Both thefi gures suggest s巴 veral poi nts 出 at satisfy first
order condi ti ons. Thei r cl oseness m池 田 it difficult for any numeri cal techni quetofi nd
theopti mum. l t iscl ear 白 紙 theEPF severel yi ncreases thenonl i neari tyofthe probl em.
Sec7_2_1-pl ol mcontai ns the code伽 t generates the pl ot . Si nce the code uses
symbol i c mani pul ati on. actual l y drawi ng tlle plot takes ti me. Eval uati ng the data
Dumeri cal l y will make a bi gdi fference.


ー ャ 7九..;..;..:
..・- - 暗唱 、 、
. . ー 曹 、 . ' "
3 4
Fl gure7.2 E刈 阿 or Penal tyFuncl l onMel hod: Exampl e 7.1 (n. = 1, rg= 1).
Appl l cati on 0 ' EPF A句 orl thm(A7. 2): The correspondi ng unconstrai ned func-
ti onfor 白 is probl emis

• 九   +4) +
日 + xi - 2)2+ ' 8(0お xf +O万三一 1)2 (7.22)
Sec7 _2_1_cal c. mis an m- fiI e that wi l l cal cul ate出 esoh瓜 onand theval ues of 白 e
functi on. for Exampl e 7.1. for a predetermi ned set of val ues of thepenal ty mul ti pl i ers.
It requi res t wo i nputs 台。 mtheuser at di fferent stages. The first i nput is theval ues for
白 emul ti pl i ers for whi ch出 esol uti onshoul d be obtai ned. 百 leI i st of sol uti on (there
areni ne for this probl em) is di spl ayed in由巳 Command wi ndow. The user fi nds 白 e
sol uti onthat satisfies thesi deconstrai nts (usual l y one) whi ch must be enteredut 白 e
prompt (note: it must be enteredas a vector). The programthenpri nts out 白 eval ues
of 山 e vari ous functi ons i nvol vedin白 e exampl e. The fol l owi ngis postedf romthe
Command wi ndowfor bothof thepenal ty mul ti pl i er百 set to25.
持 Sec7 2 1 cal c
ent er val ue f or r h [ def aul t 1] 25

$everc nonllnearltywlthmany locol ml nl mums
ー ー τ , ..
Fl gure7.3 Exterior Penal tyFuncti onMethod: Exampl e 7.1 (n. = 5. fg= 5).
ent er val ue f or r g [ def aul t 1] 25
- 1. 4125 - 0. 1015
- 0. 8692 1. 0980
- 0. 6263 - 1. 2107
- 0. 0278 - 1. 3266
- 0. 0124 1. 3267
- 0. 0090 - 0. 0000
0. 7893 - 1. 1267
0. 9775 1. 0165
1. 4184 - 0. 1318
I nput t he desi gn vect or chosen f or eval uat i on
[ 0. 9775 1. 01651
The desi gn vect or [ 0. 9775 1. 0165 1
obj ect i ve f unct i on: 2. 9810
equal i t y const r ai nt : ー 0. 0112
i nequal i t y const r ai nt : 0. 0138
Here ).. is themul ti pl i er vector ti edto the珂 ual i ty cons町 ai nts,p is themul ti pl i er
vector associ ated wi l h the i nequal i ty constrai nts, and rh and rs are the penal ty
mul ti pl i ers used si mi l ar to l he EPF method. F is sol ved as an unconstrai ned
functi on for predetermi ned val ues of λ p, r" , and rg. Theref ore, the sol uti on for
each SUMT i terati onis
In the above run, the equal i ty and the i nequal i ty ∞nstr包 nt ar芭 not sati sfi ed (箇
expected). Tabl e 7.1 documents theresul ts of theappl i cati on of 白 eEPF met hod to
Exampl e 7.1throughSec7 _2_1_cal c. m. 百 lescal i ngfactor is5 for bothmul ti pl i ers.
A gl ance al Tabl e 7.1cl earl y i1I ustrates thecharacteri sti cs of theEPF met hod . As the
val ues of themul ti pl i ers i nc陀 ase:
X. =X・( λp,r
, rg)
. The desi gnapproaches 出 eopti mal val ue.
• The constrai nt vi ol ati ons decrease.
• The sol uti onis bei ngapproached f romoul si dethefeasi bl eregi on.
At 山 eend of theSUMT ilerationl heval ues of themul ti pl i ers and p巴nal ty constants
areupdated. The latter areusual l ygeometri cal l y scal edbut unl i keEPF do not have to
be合 i vento∞ for convergence.
Al gorl thm: Augment ed Lsgr ange Mu附 ipller 伺 LM) Met hod伺 7.3)
Step1. Choose X
, N. (no. of SUMT i terati ons) ,
(no. of DFP 巴
E 匂j'S 叫(fi “or刊 c ∞o叩 nv 刊巴 r唱ge叩nc印eand stoppi ng)
r ぺ; . r4; (Oi n 凶副itia 凶a1pe叩na 叫a叫Ity ml 剛 II 削u 凶llti 匂i 切Pμli託e隠
Ch, c
(sca1ing val uefor mul ti pl i ers)
  , 阻 val uefor mul ti pl i er百)
ν,.JJI (initial mul ti pl i er vectors)
q = 1 ( SUMT iterationcounter)
Step2. Ca11 DFP tomi ni mi ze F ( Xq, )..q, p", rZ, r:)
Out put: xq.
Step3. Convergence for AL M
If h
= 0, for k = 1, 2. . . . , 1;
l f g
:三 0,for j = 1. 2, . • . , m;
( 百島 >Of or め =0)
( I f Vf +k λ' kVhk 却 め
If all si d巴 constrai nts aresatisfiω
Then Converged, Stop
Stoppi ng Cri teri a:
M' =F q- Fq- 1・aJ ( = xq* - X<q- I) ・
lf ( M' )2 S EJ: Stop (functi onnot changi ng)
日 seIf aJ (T d. XS EI : Stop (desi gnnot changi ng)
El seIf q = N.: Stop ( maxi mumi terati ons reached)
An i mportant note 日 間 ds to be mad巴. 百 le anal yti cal computati on usi ng symbol i c
computati on appear吋 伺 sy. The same cannot be sai dfor thenumeri cal computati on.
especi a1ly l ooki ngat Fi gure 7.3.
Augment ed Lagr ange Mul ti pl i er ( ALM) Met hod
Thi s is the most robust of 白 e pena1ty functi on methods. Mor e i mportantl y it a1so
provi des i nformati onon白 eLagrange mul ti pl i ers at 山 esol uti on. Thi s is achi evedby
not sol vi ngfor the mul ti pl i ers but merel y updati ngt hemduri ngsuccessi ve SUMT
i terati ons [2,4). It overcom巴 s many of the difficulties associ ated wi 曲 the pena1ty
functi onformul ati onwi thout any si gni fi cant overhend.
7. 2. 2
Tr ans如 rmat l onto the Unconst r al ned Pr obl em: The genera1 opti mi zati on
probl em(7.1)ー(7.4) is 凶 l sform吋 ωi n山 間 山 odof Lagrange mul ti pl i ers
Mi ni mi ze
府 川 r8): f ( X) や d X ) z dトト∞剖)+
(X) 剖)
Tabl e 7.1
i = 1. 2..... n
Exl eri or Penal l y Functi onMel hod: Exampl e 7.1
xls巧 S x, !
Conti nue
日 ← 日 +2rhh(X"O)
pq←pq+2 r
( max也 (xq*). -pqI 2r
r1.← rrCh; r: ←r f Cg
0. 0227
0. 0037
7. 5097e・004
ー0. 0687
-0. 0302
-0. 0112
-0. ∞27
-5. 9775e-004
2. 9810
1. 0391
1. 0165
1. 0044
0. 9464
0. 9942
1. 0
25. 0
1. 0



00 toStep2
The AL M algori 出 m(A7. 3) is avai l abl e as a stand-al oneprogramin ALM. m. 百 le
programis l ongwhi ch is not unexpected. Thesea1gori thrns arenot 凶 vial. To translate
al gori thm(A7. 3) intoworki ng code is a佐官 mendous accompl i shment , parti cul arl yif
yourexpo制 限 toMATI . AB occurredthrough出 isbook. The reader shoul dwal k through
the code line by line. 百 e organi zati on and structure are kept si mpl e and
strai ghtforwal' d. Conunenl s hav巴 been liberally used as an cncouragement to
ALM. m: The code ga由 ers i nput f romthe user through several prompts in the
Conunand wi ndow. The prompts and 由巳 final iteration ar芭 copi ed bel ow. The
si gni fi cant features of thecode are:
• The startingdesi gnvector is thefirst i nput to山己 progr創 n. The number of desi gn
vari abl es is automati cal l yextractedf romthis i nformati on.
・ 百 le l ower and upper bounds on 血 巴 desi gn v回 abl es (side cons紅白 nts) 釘 巴
obtai ned出 roughuser prompts (though曲 eyarenot usedin山 eprograrn).
• Number of equal i ty constrai nts and correspondi ng initial mul ti pl i ers (λ) ぽ 巴
• Number of i nequaJ ity 叩 nstraints and correspondi nginitial mul ti pl i ers ( 日) 脱
• Val ues at cachSUMT itcration加 古 pri ntcdintheCommand wi ndow. Wami ngs
generatedby MATLAB紅 巴 swi tchedoff.
• The DFP. mis used for uncons回 i ned opti mi zati on. Thi s programuses
Gol dSect“i on一nVa 町 r
• τ百oeprogramex却P ε氏ct匂sthefol l owi ng臼 e伐stωobe avai l abl ei 加n出 ep 伊a白止:
Obj ecti vefuncti on Of un. m
Equalityconstrai nts Hf un. m
Inequal i tyconstrai nts Gf un. m
Unconstrai nedfunctI on FALM. m
• The programuses gl obal statements tocommuni cate mul ti pl i er val ues.
・Theinitial penal tymul ti pl i ers arecomputcd automati cal l y.
• Several parameters arecoded into出 eprogram. especi aIl y thoseneeded for the
gol densecti onand the upper bound cal cul ati on. These canbe changed by出巴
us巴r. if necessary.
Appl i cati on to Exampl e 7.1: Thc fol l owi ng represents the initial prompts
secn in the Command wi ndowand the val ues for thc starti ng iterations. Other
i terati ons are si mi 1arly di spl ayed. Al so inc1uded bel owis a consol i datedpri nti ng
ofval ues for a11ofthe SUMT i terati ons at thetermi nati onofthe program. The initial
desi gnvector is [ 32 ]. Initial λ = 1 and   = 1.
持 AL M
1nput t he st ar t i ng desi gn vect or
Thi s i s mandat or y as t her e i s no def aul t vect or set up
The l engt h of your vect or i ndi cat es t he number of
unknowns (n)
Pl ease ent er i t now and hi t r et ur n f or exampl e [ 1 2
3 . . . ]
[ 3 2]
誼 t
The i ni t i al desi gn vect or :
3 2
1nput t he mi ni mumval ues f or t he desi gn vect or . .
These ar e i nput as a vect or . The def aul t val ues ar e
3 * st ar t val ue bel owt he st ar t val ue unl ess i t i s zero.
1n t hat case i t i s - 5:
[0 0]
The mi ni mumval ues f or t he desi gn var i abl es ar e 0 0
1nput t he maxi mumval ues f or t he desi gn vect or .
These ar e i nput as a vect or . The def aul t val ues ar e
3 * st ar t val ue above t he st ar t val ue.
1f st ar t val ue i s 0 t hen i t i s +5:
[5 51
The maxi mumval ues f or t he desi gn var i abl es ar e 5 5
Number of equal i t y const r ai nt s [0] 1
1ni t i al val ues f or l ambda 1
Number of i nequal i t y const r ai nt s [0] 1
1ni t i al val ues f or bet a 1
ALM i t er at i on number : 0
Desi gn Vect or (X) 3 2
Obj ect i ve f unct i on 64
Squar e Er r or i n const r ai nt s( h, g) 1. 2100e+002
1. 8063e+001
Lagr ange Mul t i pl i er s ( l ambda bet a) : 1 1
Penal t y Mul t i pl i er s ( rh r g) : 5. 2893e- 001 3. 5433e+000
( ot her i t er at i on omi t t ed)
X not changi ng O. OOOE+OOO r eached i n 5 i t er at i ons
The val ues f or x and f and 9 and h ar e
3. 0000e+000 2. OOOOe+OOO 6. 4000e+001 1. 8063e+001
1. 2100e+002
7. 9603e- 001 7. 1223e- 001 2. 9443e+000 0 7. 379ge- 001
9. 9937e- 001 1. 0092e+000 2. 9995e+000 1. 8223e- 004
2. 9434e- 004
9. 9607e- 001 1. 0013e+000 2. 9962e+000 0 2. 731ge- 005
9. 9940e- 001 1. 0002e+000 2. 9994e+000 2. 5872e- 011
6. 2772e- 007
9. 9940e- 001 1. 0002e+000 2. 9994e+000 2. 5872e- 011
6. 2772e- 007
The val ues f or l amda and bet a ar e
1. 0000e+000 1. 0000e+000
- 9. 0876e- 001 0
- 1. 6493e・ 001 9. 5663e- 001
9. 1192e- 002 9. 5616e- 001
ー 7. 6430e- 002 9. 9221e- 001
Th巴 programtook fi vei terati ons (lifthwas not necessary) tofi ndthesol uti onas:
x;=O. 卯 94; x ; = 1. 0002; r = 2. 9994; h. = 0. 0000; g. = 0. 0007 (7.24)
whi ch is very c10se totheactual sol uti on. The val ues for 山 emul ti pl i ers 釘 巴
A.' = - 0. 0763; s" = 0. 9922
Thi s satisfies the K T condi ti ons. However, in Secti on 7. 1. 2 the val ues of the
mul ti pl i ers, obtai nedusi ngsymbol i c computati on, arequi tedi fferent. It is di ffi cul t to
expl ai nthedi screpancy except tosay白 紙 thesymbol i c cal cul ati onfai l edtodi scover
this parti cul ar sol uti on whi ch is i ndeed possi bl e. 百 le val ues in (7.25) ar芭 mo問
acceptabl e山 anthose inSecti on7.1.2consi deri ng白 eK T condi ti ons.
TheAL Mme山 od,as demonstral ed above, is 釦 effecl i veand useful met hod insDi te
of bel ongi ng tothecl ass of i ndi rect me山 ods. It has several si gni fi cant advantages over
theother penal l y functi onformul ati ons [2]. Som巴 of t hemarelisledbel ow: ー
• The met hod is not sensi ti ve to the initial penal ty mul ti pl i ers or thei r scal i ng
・ T he met hod does not 陀 qui re 山 e penal ty mul ti pl i ers to be dri ven to extreme
val ues 10ωtabl i sh convergence.
・ The equal i ty and acti veconstrai nt can be sati sfi edprl田 isely.
. The starti ngdesi gn can be feasi bl eor i nfeasi bl e.
• The initial choi ces for th巴 mul ti pl i ers 田 nbe rel axed.
・Thereis onl y a modest i ncreaseinthecal cul ati ons.
• At convergence theLagrange mul ti pl i ers wi1l be dri ven10tlleir opt i mumval ues.
Thi s aJl ows veri fi cati on of K T condi ti ons through the si gn of the mul ti pl i ers
associ atedwi 白 血 eacti vei nequal i tycons位 ai nts.
:剖 1
The di rect met hods inc1ude both the obj ecti ve 叩 d 出 .e constr凶 nts to search for the
opti mal sol uti on. Whi l e白 emet hods do not i nvol ve conversi on toa di fferent c1ass of
probl ems (l i keSecti on7.2), most of t hemareb鎚 edon l i neari zati onof thefuncl i ons
about the currenl desi 伊 poi nl. Li neari zati on is based on expansi on of the functi on
about thecurrent vari abl eval ues usi ngtheTayl or seri ω(Chapter 4).
Unesrl zatl on: The Tayl or seri es for a l Wo.vari abl e expanded functi onj{x, y) ,
quadrati cal l y about iliecurrent poi nt (xp' Yp) is expressed as:
的 1創 刊 ) = f (xf" y/,) -1- [ ま │J 13│{J +
! I 割 削 +2 蒜 │ 叫 +劉 州 側 )
L _.. I (x,.JI,) ----" I (xpJ',) -" I (弘司ト J



I f 白 e di spl acements areorgani zed ω a col umn v郎 防 [ ax 8y]T. 山 expansi on in
Equati on(7.26) can be expressed ina condensed manner ω
f汽 ( 円 山 ) ド 吋 =ザ 吻 f
L片[ 釘 A 勾制y 刈]T[凹H(仇;Xn.Yn 凡 υIJ] 川]I 釘 刊 ( 7.27)
ι , . , . I sYI 、 ,
For It vari abl es. wi th X
thecurrent poi nt and 8 x thedi spl acement vector,
f 仲 必 ) = f ( X
) + Vflぽ p)T8 X+t d XT H叩 成 問 )
Equati on(7.28) c佃 bewri l teninterms of 白 edi fferenceinfuncti onval ues ぉ
ザ =f ( X
+尚 一 f ( X
) = Vf (Xp)T dX + t dX
町 X
)組 問 )
df =Of +o
where of = Vj T a X is tenned白巴 Ji rst variation. o2j is thesecond vari al i onand isgi ven
by白 es田 ondtenn intheabove叫 uati on. Inl i neari zati onof thef uncti onf about the
current val ueof desi gnX
' onl y the fi 附 vari ati onis used. Th巴 nei ghbori ng叫 ueof
thefuncti oncanbe expressedas
l ( X
) = f ( X
) + V f ( X
) T a X (7.30)
Al l of the functi ons 加 山 e probl emc加 be l i neari zed si mi l 釘 ly. It is essenti aI to
understand 山 di fference bet weenf (X) andl ( Xp)' Thi s is ilI ustrated inFi gure 7.4
usi ngtheobj ecti vefuncti onof Exnmpl e 7.1expanded about thecurrent desi gnx, = 3,
x2=2. Th巴 curved I i nesf ( X) are the contours of 山 eori gi naI functi on. The s位 Bi ght
l i nes J (X
) arethecontours of thel i neari zedfuncti on(l i nes of constant vaI ue of the
functi on) obtai nedthroughthefol l owi ng:
Ori gi naI 印刷 i on: f(x). X2): x1- 2 xr X2 + xi + , ~ - 2x
+ 4
Li neari zedfuncti on: ](!J. XI ' !J. X2) =ω+92!J. X I - 6!J. X2
wher e the coeffi ci ents in 白己 second expressi on i ncl udes the eval uati on of the
functi onand gradi ent at x, = 3, X2 = 2 and substi tutedinE伊 ati on(7.30). InFi gure
7. 4several contours areshown al though those of int巴rest areinthenei ghborhood
of 出 ecurrent desi gn. Any nonl i near functi oncan be expanded i nthesame manner.
If another poi nt was chosen. then山 esl opes 01 " Lhe lines woul d be di fferent .
The quadrati c expansi on of thefuncti oncan be obtai nedby usi ngEqu凶 on(7.28).
百 leexpanded curves will be nonl i near (quadr甜 c). 百 l eyappear ぉ el l i psωi nFi gw芭 7.5
where severaI contours of 曲 e 0凶ecti ve functi on of Exampl e 7. 1蹴 expanded
quadrati 回l I ya加 ut XI = 3, X2 = 2. It is i mportant to附 ogni zethat 山 econtour冨 woul dbe
di fferent if another poi nt we陀 chosenfor expansi on.
Four di rect methods ar芭 di scussed The first is Sequenti aI Li near Prograrnmi ng
(SLP) , where the sol uti on is obtai ned by successi vel y sol vi ng the correspondi ng
l i neari zed opti mi zati on probl em. The s民 ond,Sequenti aI Quadrati c Progr ammi ng
( SQP) , uses 山 equadrati c expansi on for 出 eobj ecti vefuncti on. Li ke出 e SUMT,出 e
current sol uti on provi des 山 e starti ng vaI ues for the next iteration. The third. 出 e
GeneraI i zed Reduced Gradi ent Met hod ( GRG) . devel ops a sophi sti cated s朗 氏 h
di recti on and fol l ows it wi th an el aborateone-di mensi onal process. The fourth. Lhe
Sequenti a1 Gradi ent Restorati on Al gori thm( SGRA), uses a two-cycl e approach
worki ng on feasi bi l i ty and opti mal i ty a1tematel y to fi nd the opti mum. 百 l ere are
severa1other methods due to many researchers but 山 ey di ffer f rom由 ose listed in
smal l details. To k田 p the l ength of this chapter reasonabl e. 出 ey have not been
incJ uded. Wi t h由 ecode叩 dother procedures avai l abl ef romthis secti on. it shoul d not
bedi f白cul t toprogramth巴 addi ti onal techni ques出 at areworth expl ori ngfor parti cul ar
cJ asses of probl ems.
Except for speci al cl asses of probl ems inthefol l owi ngt wo chapl ers. 白 is chapter
is the pri nci pal reason for the book. It provi des a resol uti on of the compl ete
opti mi zati onprobl emas posωl ated. For a l ongti me now therehave not been any new
or greatl yi mproved numeri cal techni quω. 百 lereis a lot of wor k on gl obal opti mi zati on.

- 1
F1gure 7.4 lII ustratlonolllnearizatlon.
n Current i nterest in tradi ti onal desi 伊 opti 出 回 ti on (白 is chap町), is di rectedtoward
actual desi gnappucadons and mi grati onf mmt hemai nf mme and UNI X Envi ronment
to 白 e P C (like 出 is book) ・ For compl eteness. the standard f onnat of the NL P is
reproduced herefor conveni ence:
Mi ni mi ze !(x1oX2. • • • • x,,)
(7. 1)
Subj ectω:
ht<X" X2. ・・・ ,X
) = o. k = 1, 2. . . . • 1
め ~ X I ・ X 2         ,ん) 孟 O. j =1,2. . . . ,m
Xl S: Xi S: Xr . i =1. 2. . . . ,n
Invector notati on
Mi ni mi ze f( X). 医] "
Subj 田 tto:

3ト J

- 2

, 十

- 1
2 3
FJgure7.5 lII ustration01quadratlc9xpanslon.
[g(X)]mS 0
Exampl e 7.1 will beusedtoilI ustratethevari ous al gori thms.
Mi ni mi ze f(x. , xi): x1- 2xi x2+xi +xl r z- 2xl +4 (7. 9)
Subj ectto: h(XI , x:z): xi + rz - 2 = 0 (7.10的
g(X..X2):   SO (7.10b)
OSXI ::;5; OSX2SS (7. lOc)
7.3.1 Sequentl al Li near Pr ogr amml ng (SLP)
IntheSLP [5] all of thefuncti ons areexpanded linearly. If X; isconsi dered山 ecurrent
desi gnv回 tor,出 en出 elinearizedopti mal probl emcanbeset up as
Mi ni mi ze: l(aX) = f(X;) + Vf(X; ) T a X (7.31)
Su尚氏 tlo: 月以aX) : "k(X;) + vhI (X;)a X = 0; k = 1,2, . . . ,1 (7.32)
あ(AX): め (X;) + V8J{X;)TAX S 0; j = 1,2.. .. ,m (7.33)
Ax!ow壬 ax; S ax)IP; i = 1,2, .. . ,n (7.34)
Equati ons (7.31)ー (7.34) represenl a Li near Programmi ng (LP) probl em(Chapter 3).
Al l of thefuncti ons inEquati ons (7.31)一(7.34) ,except for aX. have numeri cal val ues
after substi tuti ng a numeri cal vector for X;. Assumi ng an LP programcode is
avai l abl e, it 回 nbe cal l edrepeatedl yafter thedesi gnis updatedas
X" , =X,+AX
Inorder toi ncl udea searchdi recti onand stcpsi zecal cul ati on,出巴企 X inEquati ons
(7.31)ー(7.33) is consi dered as a sol uti on for S (a search di recti on at the currenl
desi gn). Stepsi ze computati on slra飽 gy is no l onger si mpl e. I f出巴 re are vi ol ated
constrai nts, a1pha attempts toreduce白 is violation. If 白 ecurrent sol uti onis feasible,
出 estepsi zewill attempt toreducethe白 ncti onwi thout causi ngtheconstraI nts tobe
excessi vel y violated. Such strategies arel argel yi mpl emented at thedi screti onof 由巳
al gori thmor code devel oper.
A匂 orl thm: Sequentl al Ll near Pr ogr amml ng ( SLP) (A又 4)
Step 1. Choose X' , N
(no. of iterations) ,
E;'S(for convergence andstoppi ng)
q = 1 (iterationcounl er)
Step2. Call LP toopti mi ze(7.31)一(7.34)
Output: S
Use a constr叫 nedα・ cal cul ati on(X; = xQ)
AX=α. . S
Xq+I =Xq+AX
Step3. Convergence for SLP
If h
= 0, for k = 1, 2, . . . , 1;
If gJ S 0, for j = 1. 2, . . . , m;
[f all si deconstrai nts aresatisfied
If KT condi ti ons satisfied
Then Converged, Stop
Stoppi ngCriteria:
AX=xq+l - x
AX S EI: Stop(desi gnnot changi ng)
If q = N.: SI Op( maxi mumiterations 児 ached)
Conti nue
00 toStep2
The SLP is appl i ed10Exampl e 7.1bel ow. InChapter 3 no explicit programfor sol vi ng
白 eLP programwas devel opedal thougha command- by- command wal k白 roughwas
i ntroduced. A programfor appl yi ng白 eSi mpl exme白 od,psl mpl ex. mis i ncl udedon
白 ewebsi te(wi thout any comment or di scussi on). Usi ng i nstructi ons f romChapter 3,
addi ng somc programcontrol , and organi zi ng some bookkeepi ng can provi de a
MATLAB programfor LP probl ems. Al tematel y, M ...百 . AB'Sl i nprogcanbe usedina
l oop to hel p lranslate 出 e SLP al gori thmto code. In 出 is sec“on 曲 e l i neari zed
subprobl cmin each iteration is graphi calI y presented. The area of interest can bc
zoomed and thesol uti onreadfromthepl ot . Thi s is treatedas dX (i nsteadof usi ngit
as S). The desi gnis updatedandanother pl ot canbe obtai nedby runni ngthem・file
for a newval ue of thedesi gn vector. Sec7 _3_1- p1ot. m assists in di spl ayi ng the
I i neari zedfuncti ons f romwhi ch sol uti oncanbe obtai ned. The equal i tyconstrai nt is
出 egr田 nline. 百 leredis i nequal i l y constr通 i nt 百 ledesi gn vector is 回 加 putto白 e
program. Th巴∞deuses symbol i c cal cul ati on10sol veand di spl aythesubprobl em.
Appl l catl on of SLP
sωP 1. X
= ( 32)
Step2. Li neari zedsubprobl em r . ,
Mi ni mi ze 1(dX) = 64 + [92 -6] I I
s X
釦 ゆ ct 肱 II(dX): 11+ [6 4] I仏 1=0
I !l X21
g(dX): 4.25+ [1. 5 311内 I so
I !l X21
=ー1. 335,&2 = ax
= - O.75.For
thenext iterationX2 = [ 1. 665, 1. 25]. Repeat Step2.
Step2. Li n巴ari zedsubprobl em
Mi ni mi 吋 ほ ) = 6. 7985+ [13凹 05 -1. 382] 1
I LU21
S雌 ω : 百( 必 ) : 23347+[ 333251l h│=0
s X
l ax. l -
g(d X 川 臼 9+附 25 1. 8751/的 I SO


-4 - 3 - 2 4 5 2 3 o
Fl gure7.6 Exampl e7.1 SLP ileration1.
Fr omFi gure7.7(zoomed) thesol uti onis ax
=ー 0. 54,ax2 =ー 0.23. For thenext
= [1.125, 1. 02]. Repeat Step2.
Step2. Li neari zedsubprobl em
Mi ni m回 1(.瓜 ) = 3. 206+ [2.3957 -0. 2363] Iふ│
l 品 "'21
釦 bject 10: 五( 広) : 0.3060 + [2.25 2.04] 1的 1=0
l いι "' 2 引l
E削削叫凶附附):川:氾0ω0 …6
Fr omFi gure7.8(z初oomed の) thesol uti oni 凶sdx 川( =一 0.1ロ2笥5,dx2 =ー 0.02. For thenext
iteration X
= [ 1. 0 1. 0 ]. Si nce this is the known sol uti on, further iterations 釘 芭
百 leSLP took threeiterations t. ofindthesolution. Thi s is very i mpressi ve. It is true
onl y for th. osecases where thenumber . of equal i typl us acti vei ncqual i tyconstrai nts is

l I i
>< 0
- 2
- 3
Fl gure7.7 Exampl e 7.1 SLP l I erallon2.
4 5
l he same as 出 e number of desi gn vari abl es- a ful l comp[ement. The mai n
di sadvantage of l hemel hod appe釘 si f 白 ereis no full compl ement. Inl hat case出巴 si de
cons位置 ints (7.34) arecritical to出 edetenni nati onof l hesol uti onas 山 eydevel op i nto
acti veconstrai nts. Inl hesesi tuati ons l he l i mi ts on l hesi deconstrai nts 創 e cal l edl he
move limits. ln回 seof acti vesi deconstrni nl s I hemove l i mi l s establ i shtheval ues of
l hedesi gnchanges. If l hedesi gnchanges areconsi der吋 as l hesearchdi 陀 cti on(s回
al gori l hm) ,山 enbo曲 vaJ ueand di recti onarea仔巴 cted. For l hesechanges tobe smal l
as l hesol uti onis approached,出 emove l i mi ts have tobe adj usted(l owered) wi thevery
i terati on. 百 lestrntegyfor 血 is adj ustrnent may i nfl uence白 esol uti on.
Exampl e 7.1 wi l hout the i nequal i ty constrai nt, but wi l h 白 e equal i ty constrai nt
provi des an illus位置 l i on of the move limits. Fi gu問 7.9 represents l he lin回 ri zati on
about l hepoi nt [ 3, 2 J . The move limits inl hefi gu陀 (box) represent l hesi deconstrai nt
l i mi ts
  三 - 2 S2
lt is cl ear f romFi gure 7.9 Ihat wi l houl Ihe move l i mi ts thc sol uti on is unbounded,
whi ch is not very hel pful . Usi ng l hesol uti onobtai nedinl hefi gure, l henext i terati on
き 自 1
封 同 。
- 3
- 4
- 4 - 3 -2 -1

2 3 4 5
Fl gure7.8 Exampl e 7.1 SLP l I eration3.
C組 beexecuted. Once agai nl hemove lirnits wi l l i nfl uence出 esol uti on. If themove
l i mi ts are left at l he same val ue, the sol uti on wi l l al ways be on l he squぉ e wi th no
possi bi l i tyof convergence. These lirnits musl be l owered wi th回 chiteration. Usual l y
they are done geometri cal l y l hrough a scal i ng faclor. 百 lis i mpl i es l heconvergence
wi l l depend on l hestral egy for changi ng l hemove l i mi ts, not an appeal i ngsi tuati on.
The SLP is not one of l hepopul ar mel hods. 111emove lirnits areavoi ded wi 由 l henext
me出 od.
7. 3. 2 Sequent l al Quadr at l c Pr ogr amml ng ( SQP)
The l ack of robustness in由 eSLPduet o出 eneed for move lirnits can be countel 吋 by
i ncl udi ngan el ement of nonl i n回 rity in白 eprobl em. The陀 arc several ways lhis can
be achi eved. One way is 10 requi re l hal 山巴 search di recti ons ( remember in acωa1
i mpl ementati on s X isS) inEquati ons (7.31 ) ー (7.34) be l i mi tedby theuni t ci rcl ewhi ch
can be descri bed白巴 constrai nl :
STSS: 1 (7.35)
z o
- 1
- 2
- 4 - 3 - 2 o
Fl gu問 7.9 lII ustratlonof move IImlts.
百】巴 stepsi zedetermi nati onshoul d then account for the actu叫 changein thedesi gn
vector. The probl em expressed by Eq岡山田 (7.31)一(7.35) is a quadmti c
programmi ng (QP) subprobl em. An i temti vesol uti onof this subprobl emc血 beone
f ormof 恥 SQPme出 od. Thi s 脱 出 odis popul ar and is usedwi del y. Whi l e theQP
subprogramis a si mpl e nonl i near probl emwi th l i near cons位置 ints. it is still a
mul ti vari abl c nonl i near probl em. Fortunatel y, QP probl ems appear in many
di sci pl i nes and thereareeffi ci ent procedures and al gori thms toaddress theseprobl erns
[6-10]. One of th巴se methods is an extensi on of the Si mpl ex met hod of Li near
Programmi ng.
百 lesol uti onto白 eQP is wel l documented. It woul d r l叫 ui remuch di scussi onand
thereforeis not i ntroducedhcre. Readers arereferredto thevari ous references. Thi s
is al so 白 e pri mary reason why the QP met hod is avai l abl e in the Opti mi zati on
Tool box inMATLAs. The formal methods for QP probl ems arebased on effi ci ency in
thevari ous stages of thei mpl ementati on. An i neffi ci ent but successful sol uti on10the
probl emcan be obtai nedby usi ngI heA L M of theprevi ous secti onto sol vetheQP
subprobl em. After all. computi ng resources arenot a constrai nt today. Inlieuof formal
di scussi on. an i ntui ti veunderstandi ngof 白 enumeri cal met hod is attemp低 d inthis
book. The transl ati onof SQP i ntocode is not trivial and themet hod isusual l ythebasi s
of commerci al software. For exampl e, SQP is the onl y al gori thmfor constrai ned
opt i mum白 紙 MATLAB i mpl ements (as of this wri ti ng). Havi ng access to the
Opti mi zati onTool box al l ows NL P tobe sol vedusi ngSQP. A detai l eddi scussi onof
thesol uti ontechni que is al soavai l abl e[11].
百 le QP subprobl emempl oyed in this subsecti on is based on expandi ng thc
obj ecti ve functi on quadrati cal l y about the current desi gn. 百 le constrai nts are
expanded l i nearl y as in SLP [8]. 百 is is cal l edsequenti al quadrati c programmi ng
Mi ni mω 1(d.X) =f'叫 ) + Vf ( Xi
d. X+伊 T V
fl附 d X 間 )
Subj 田 lto: 九( 以) : hk(Xj ) + VhI ( X,) ほ=0; k = 1. 2. . . . . 1 (7.32)
  gj ( Xj) + VgJ{Xj)Td. XS 0; j = 1. 2. . . . . m (7.33)
sx}owS sxj S i = 1. 2, .・.. n (7.34)
In Equati on (7.36) Vす ( X,) is theHessi an matri x. Inactual i mpl ementati on thereal
Hessi an wi l l not be used. I nsteada metri c 開 ] 白 紙 is upda飽 d wi 出 eachi terati onis
used. 百世 s is based on 出巴 success of the Vari abl e Me凶 c Met hods ( V恥仏. f ) of the
previ ous chapter. Several researchers [6.9] have shown出 鉱 山 巴 BFGSupdate for the
Hes8i an provi des an e仔 i ci ent i mpl ementati on of the SQP method. Thi s QP is wel l
posed and convex and shoul dyi el da sol uti on. Sol uti onto this subprobl emis at 由巳
heart of 山 e SQP method. A consi derabl e amount of research has been i nvested in
devel opi ng effi ci ent techni ques to handl e 出 is subprobl em. In a formal
i mpl ementati on of 恥 SQP 山 d . Xin Equati ons (7.36) 加 d (7.32) ー(7.34) musl be
回 pl acedby thesearchdi recti onS. The QP for S al 80modi fi es theconstrai nt equati ons
80that a feasi bl edi recti oncan be f ound wi 白 respectto白 ecurrent acti veconstrai nts.
Fi gure 7. 10 illus岡 鉱 S 白 e concemabout movi ng l i nearl y to a poi nt on an acti ve
con8佐 ai nt. XO, thecurrent desi gnis on anacti veconstrai nt . If thesearchdi recti onSI
fol l ows 出巴 l i neari zedfuncti onat XO (thetangent to the functi on at that poi nt). any
stepsi ze. al ong自己 E問 見 h di recti on. however smal l , will cause theconstrai nt to be
vi ol ated. Inorder todetermi ne a nei ghbori ngpoi nt that will sti¥l sati sfytheconstmi nts.
a search di recti on sl i ghtl y less th朗 自 e tangent is used. This i ntroduces a l ack of
consi stency as thedevi ati onf romtangency becomes a matter of i ndi vi dual experi ence
andpracti 田 . Experi ments have suggested白 at 90- 95%of thetangent isa useful fi gure
al thoughmaki ng it as cl oseto100%is recommended [2]. τbe searchdi recti onfi ndi ng
QP subprobl emis
Mi ni mi ze 7(S) = f ( X
) + Vf(Xj )T ST[H] S (7.37)
Subj ecl ω: 九(S): c MX
) + VhI ( X,) S=O; k = 1. 2. . . . ,1 (7.38)
あ(S): cgi<Xi) + V 8.メx,)'f S S 0; j = 1. 2ぃ.., m (7.39)
s}OWSSj SSr"; i =I . 2. . . . . n (7.40)
Rep伺 ci ngHessi an wi thBFGS Update: l n Eql鳩 山 n (7.37) , cal cul ati ng the
actual Hessi an is strongl y di scouraged. Si nce the BFGS m巴出 od converges to the
Hessi an, theBFGS metri c is consi deredtobe an excel1ent repl acement of theHessi 佃
[9]. Theme凶 cis based on theLagrangi an of 出 efuncti on[12]:
Cal cul ati ng the Mul fl ptters: The popul ari ty of the SQP m白血 od is that it
attempts to update 出 e Hessi an based on the Lagrangi an of the probl em.
Cal cul ati ng the mul ti pl i ers is necessary to veri fy the Kuhn- Tucker condi ti ons at
th巴 sol uti on. Cal cul ati ngthemul ti pl i ers, for thecurrent val ue of thedesi gn vector,
is necessary for addi ti onal operati ons, i ncl udi ngsetti ngup thedescent functi onin
Equati on (7.43) , as wel l as updati ng the metri c repl acement for the Hessi an
(fol l ows next). The cal cul ati onuses th巴 F OCbased on theLagrangi an. The val ues
of thegradi ents of thevari ous functi ons arecomput ed at thecurrent desi gn. The
mul ti pl i ers 紅 白 then obtai ned as a sol uti on to a set of l i near equati ons. The
sol uti ons toQP and SQP probl ems usual l y takeadvantage of th巴 activeconstrai nt
set. Thi s i ncl udes all equal i ty constrai nts and onl y those i nequal i ty constrai nts that
are acti ve. Thi s i mpl i es that SQP i terati ons start at a feasi bl e point. The
mul ti pl i ers correspondi ng to i nacti ve constrai nts are set to zero. The remai ni ng
mul ti pl i ers can be establ i shedconsi deri nga set of an appropri ate number of l i near
i ndependent equati ons f romEquati on (7.12):
Equati on (7.42) or (7.43) can be used toobtai n白 eval ue g stepsi zeα. The functi onin
Equati on (7.43) requi res an i mportant observati on. Shoul d theLagrange mul ti pl i ers
be cal cul atedfor each val ueof α? I magi neusi ngthegol den secti ontosol veEquati on
(7.43) wi th 血 e mul ti pl i ers cal cul ated 企o mthe sa出 facti on of 白 e Kubn-Tucker
condi ti ons for each newval ue of X. Several updati ngstrategi es for 由巳 mul ti pl i ers in
Equati on(7.43) can be f ound in由 elist巴dreferenc巴s. A si mpl e one is tohol dtheval ues
of themul ti pi ers constant 創 出 eprevi ous iterationval ue and change t hemat 白 eend
of theiteration.
。. 42)
i =1,2,.. . , n











A 句





ικ t


、 ,



e J F eJ f " dh, . oh, • n e}g 1. . n e}g
一 = 止 +λ 」 +ー ・ +L - +P I 」   ・ ・ ・   ~ n ー . = O ;
axi a x i f t a x i t a x i f l a x i i a a x i
xj =x;'

Mi ni mi ze:
Mi ni mi ze:
The si deconstrai nts arenot necessary総 出 is is a wel l -posedprobl em. 官 lefactor c is
the adj ustment for movi ng tangent to the constrai nt.百 le suggested val ue for c is
0. 9-0. 95. For thei nequal i ty constrai nt,出 eval ue of c is 1 if theconstrai nt is satisfied
(shoul dbe abl etomove tangent totheconstrai nt).
Stepsi ze Cal cul ati on: On回 出 e search di recti on has been determi ned, it is
to cal cul ate thestepsi ze. Si nce the methods of 出 s chapter have seen an
enormous i ncreas巴 inthenumber of cal cul ati ons, thestepsi zecal cul ati ons arebased
on s加l Ul taneousl y decl ' 回 si ng 血 e obj 田 tive as wel l as i mprovi ng 出巴 constrai nt
sati sfacti on. Inorder toaccompl i sh this goal a sui tabl efuncti onmust be devel oped and
山 estepsi zedetermi nati onbased on mi ni mi zi ng this unconstrai nedfuncti on. Several
authors refer toit as thedescent fllnction. Th巴 descri pti onof this functi onvari es f rom
researcher to researcher. Some defi ne白 is as theLagrangi anfuncti oninwhi cb case
曲巴 Lagrange mul ti pl i ers must be determi ned. Many others i ncl ude onl y the
constrai nts wi th a penal ty mul ti pl i er and adopt 山巴 EPF f ormfor 白巴 functi on. Still
others i ncl udeonl y themaxi mumvi ol atedconstrai nt. Two f orms of thefuncti onaz芭
¥i stedbel ow:
4 3
Movl nglinear toanactiveconstraint.
Fl gure7. 10
><" 1
p=Vf ( X判) +L Ak Vhk(X伊 1) +エ sj VCj (X' 計' )
1<=1 j=IVjeJ
- Vf( Xq) +ヱ入 ' kV + L s
V gi
1<=1 j =l l tj eJ
y =Xq+I _ X
[Hq+l] = 間 J +単 一 耳 目 阻
p' y y・[Hq]y
j E J i mpIi es theset of acti vei nequal i tyconstraints. It isrecommended白 紙 白 巴 me凶 c
be kept posi ti vedefiniteas much as possi bl eeven if it is likely 白 at it wiIl nol be
posi ti vedefiniteat thesolution. One way of ensuri ng白 is is toverifythat
for eachiteration.
A匂 ' orl thm: Sequenti al Quadrati c Pr ogr amml ng ( SQ門 伺 7. 勾
Step1. Choose X I , 1叫 (no. of iterations).
E/S(for convergence andstoppi ng)
q = I (iterationcounl er)
Step2. Cal l QP toopti mi ze(7.37)一(7.40) ( X
= xq)
Output: S
Use a constrai nedぜ cal cul ati on
AX=α" s
Xq+1 =Xq+AX
Step3. Convergence for SQP
I f hk=O,f ork=1. 2,....l ;
立 身 , f orj =1. 2,..., m;
If K T condi ti ons aresatisfied
百 四 nConverged. Stop
Stoppi ngCriteria:
AX=xq+l _xq
AX EI: Stop(desi gnnot changi ng)
If q = N
: Stop( m位 回 umiterations reached)
Conti nue
Updat eMe肘 c[町
q ←q+I
nn rnStpnヲ
l 沼 i
Appl i cati on 01 SQP: Th巴 SQP techni que is appl i ed to Exarnpl e 7. 1. There are
several ways toaccompl i shthis. The appl i cati oncanusegraphi cal support as shown
wi th the SLP (this is an exercise). The appl i cati on can use the QP programf rom
MATLAB (alsoanexerci se).l n曲 is book白巴 QPprobl emis sol vedby appl yi ng曲 eKT
condi ti ons-wbi ch is thefeatureofthi s subsecti on. Ati飽 r all for 出 is specificexampl e
白 efuncti ons areanal yti cal l ydefi nedand血 ei rstruc組 問 is si mpl e. We canhamess the
symbol i c operati ons泊 MATL油 tocal cul ate白 ederi vati ves and白 eactual Hessi an. As
in 山 e case of SLP, the actual chang巴 in tl X is computed [Equati ons (7.36).
(7.32)ー(7.34)] rather thanthesearchdi recti onS for theQP subprobl em.
Sec7 _3_2. mappl i es theSQP me出 odsymbol i cal l y toExampl e 7. 1. lt identifies
thesol uti onusi ng出 eLagrangemethod (Chapter 5). It promp凶 for thecurrent desi gn
vector. It sol ves theQP anal yti cal l y appl yi ngtheKT condi ti ons andprovi di ng白 e
results for thetwo cases. C邸 ea corresponds tos = 0 (gmust be less 由 anzero). Case
b corresponds tog = 0 (s> 0). The output f romrunni ngtheprogramisrecordedbel ow
for several iterations in 出 is i mpl ementati onof SQP. Ana1ytical sol uti onwi thuser
i nterventi onisus叫 . Sec7 _3_2. mmust berunfor eachiteration. You shoul dbeabl e
tocopy and past巴 desi 伊 vector in自己 commandwi ndow.
Iterati on2:
=[ 3 勾
M 町叶=イ[戸2 lトい問岡=[宝 ? 可
1; 寸 叫 叫 叫 ω ω   刊
=- 1. 0591; =ー1. 1613
五=0; g = - 0.8225;
= 1.9409; 毛 =0. 8387
= [1. 9409 0.8387]
州 3.123; サ = U Z ;什 1 2 2 2 剣
山 705; Vh=[悦 J g=0. 4693; η =1 1矧
The soIuti ontotheQP probl emis Case a:
=ー0. 6186; d.x
=- O.0041
h=O; g=ー1. 8281;蕩 f
茸 1= 1.3222; 毛 =0. 7975
= [1. 3222 0.7975)
f =4. 2126; 町=ド立 J 問=似品 2 2 8 1
=0 牧町=! ?剣山 0858; Vg刊 ;;;l
7. 3. 3 General l zed Reduced Gradl ent ( GRG) Met hod
百 1巴 GRGmethod is anothcr popul ar techni que for constrai ned opti mi zati on. 百 le
origina1 method, the Reduced Gradi ent Method [13] , has seen severa1 different
customi zati ODS due to severa1 researchers [14). Li ke 出巴 SQP 白 is is ao active set
method- deal s wi th active i nequal i ti es-hence the method iocI udes 珂 uality
cons回 ints only. The in叫 uality cons回 ints 釘 E位 四 sformedto equa1ity co田 町 aints
usi oga linear sl ackvari abl eof thetypeusedinLP probl ems. τne genera1opti mi zati on
probl emis restructuredas
The SOIUti ODtotheQP probl emis Case a:
Mi ni mi ze f(XI ' X2, • • . ,Xn) (7.45)
= - 0.2685; dx
= +0. 2042
Subj ect to: ht(XI' x2' ・・・ ,x,,) = 0, k = 1, 2, . . . , 1 (7.46)
h = 0; g = -0. 0190;
8J (XIoX2' . . . ,x,,) + x,t+j = 0, j = 1,2, • . . ,m
= 1.0537; ち =1. 0018
xl sx; sxr i =1,2,..., n (7.48)
Xn+j 0 j = 1,2, • .. ,m (7.49)
= [1. 0537 1. 0018]
f = 3.0685; ド 法 昨 同
…沼 ; v h=I 2124; g=ー0問 内 =l T Z l
The total number of vari abl es is n + m. The active !:et !:trntegy ao; sumes that it is
possi bl eωdef i ne or choose n subset of (11- 1) i ndependent vari abl es 合 omtheset of
(l + m) equal i ty constraints. Once these vari nbl es are estabIi shed, lhe dependent
vari abl es (l + m) canbe recoveredthrough出巴 consLraints. The number of i ndependent
vari abl es isbasedon山 coriginnl desi gnvari nbl cs andtheoriginal equa1ityconstrai nts
of theprobl em. The fol1owi ng devel opment is ndaptedf romReference 2.
The set of thedesi gnvari abl es [X] is parti ti onedinto出 ei ndependentset[Z] 叩 d
dependent set [11.
dx. =- 0.0523; L1xz=- O.0017; ト 0. 9275
五=0; g=O;








1. = 1. 0014; 毛=1. α胸 ;
The constrai nts areaccumul atedas
f =3. 0014; h=O. ∞27; g = 0. 0006
H=[ : ]
The sol uti onis a1most convergedand furthcr iterations areleft tothereader. The
next iterationshoul dproduce va1ues for convcrgence.
The SQP heretakes fiveiterations toconverge. This was basedoncomputi ng the
actua1 Hessi an. The SLP took four. The AL M took five. For Exampl e 7. 1出 eSQP
does not app回 r tohave姐 y significaot advaotage, whi ch is unexpected. The AL M
slarts to appear qui 飽 attractive. The SQP in Sec7 _3_2. mis i mpl emented
anal yti cal 1y basedonKT condi ti ons. It is verydifferent thanAl gori thm( A 7.5) whi ch
rep陀 sents a numeri cal i mpl 巴 mentati on.
Re1ations (7.45)ー(7.49) canbe recasl as
Mi ni mi ze f(Z , Y) (7.50)
Subj ect 10: H(Z , Y) = 0
z!S z; Szf i =1,2,..., n- l
A si mi l ar set of si deconstrai nts for Y can be expressed. 1nac伽 al i mpl 巴mental i onthe
i nacti ve in叫 ua¥ i ty constrai nts are not inc1uded in 白巴 probl em. Al so, the sl ack
vari abl es can b出 ng to the i ndependent 制 . I n GRG, 山 lin削 zed 01吋 叫 ve is
mi ni mi ztd suhdect tothel i neari zcdconstrai nts about thecumnt desi gnr.
Mi ni mi ze ](l:J.z' l:J. Y) = f ( X
) + V zf( Xi ? d. Z+ V yf ( X
? l:J. Y (7.53)
Subj ect to: 払(!:J.Z, l:J. Y) = H. ( X
) + VzH. (Xi )Td. Z+V.;.叫 (Xi )Tl:J. Y = 0 (7.54)
lf theGRG began at a feasi bl epoi nt.出 eB( Xi ) =0,出 elin巴ari zedconstrai nts area¥so
zero,出巴 n恥 i bl eari zedequati ons in問 ] detenni ne a set of l i near equati ons inl:J. Z and
d.Z. Consi deri ng出巴 vector [H] thel i near equati oncan be wri ttenぉ
[A] d. Z+匝]l :J . y=o
r r VvHT 1
I Y ZI I J I I • ya &1 I
1   1 1 Vν 耳 │
[A] = 1 - . - 1 ; [B] = 1 " - 1
・ ・ ・ ・
z味 mI I V)正札
Fr omEq凶 ti on (7.55) the change in山 edepcndent vari abl el:J. Y can be expressed in
l erms of thechanges inthc i ndependent vari abl es as
l:J. Y =ー[ Br
[A] d. Z
Substi tuti ng Equati on (7.56) i nto Equati on (7.53) reduces i t ωan unconstrai ned
probl 巴 mind.Z:
] ( 必,) = f ( X
) + V zf(Xi )T d. Z- Vyf (Xi )T[Brl [A] d. Z
= f ( X
) + {v zf ( XうT- Vyf lほう T[ Brl [A]) l:J. Z (7.57)
=バ Xう+{Vzf ( X
) ー( [ 町 l [ Al f vdxt )「 必
= f ( X
) + [GR]T l:J. Z
[ G
] is出 ereducedgradi etJt of thefuncti onj{X)・It provi des thesearchdirection. The
蹴ふ i zeαi s f ound by J 1叩 i ri ng伽 t恥 伽 ng巴si nl:J. Y,∞J1J1e5pondi ng10恥 ch叫 es
in hz ( αS) , determi ne a f l回 sible d叩 gn. Thi s is i mpl emented 血 rough Newt on' s
met hod as descri bed bel ow. Si nce t hehnct i ons ME l i near,t hemis no quadrati c
convergence associ ated wi 白 山 is choi ce. The i terati ons start wi th a feasi bl e current
desi gnvector [Xi] = [ Z汀 Y吋
Stepsl ze Comput at l on: A旬。 r l t hm( GRG)
Step 1. Fi nd G
, S = - G
; Sel ect a
Step2. q = 1
d. Z=αS; Z = Zi +!:J.Z;
l:J. yq=ー[ Br
[A]d. Z
Step3. y q+l = Yq + l:J. yq
x1+1=1 ZI
I yq叶 │
If [ H( X削) ] = 0; Stop, Converged
日 seq←q +1
l:J. Y9=凹 r
[-H(Xi +I )]
GoTo Step3
白山 E功。 vea¥gori 白 m,nei 伽 Z,[A] , [BJ , nor αi s 伽 nged. Thc si ngul ari ty of [B]
is a concemand can be con町 OI l edby thEsel ecti onof i ndependent vari abl es. I tapPE郡
山 at az zs responsi bl e for decrEasei n thE functi on whi l e A Y is rEsponsi bl Ef or
feasi bi1i ty. The above al gori thmis usual1y 巴xecutedfor l wo va¥ues of α. A quadrati c
i nterpol ati onbetween thethr田 poi nts ( α= 0 is avai l abJ ef romtheprevi ous i terati on)
is 日叫 to detenni ne 出 e opt i mum蹴 psi ze. One f 伽in】施凶 al i 加t飽era 則 on is perf ormed for 出 i
opt i mumstepsi ze.
A句 01鋤 m: Genele!l l zedReduced Gradi ent ( GR句 併 又 旬
Step1. Choose X
(must be feasi bl e)
N. (no. of i terati ons) ,
E;'S(for convergence and stoppi ng)
p = 1 (i terati oncounter)
Step2. I denti fyZ, Y
Ca¥cul ate[A] , [B]
Ca¥cuJ ate [ G
Cal cul ateOpt i mumStepsi zeα・ (seeal gori thm)
Cal cul ateXP+l
Step3. Convergence for GRG
If hk = 0, for k = 1, 2, . . . , 1;
If 8j S; 0, for j = 1, 2, . . . , m;
If K T condi ti ons aresati sfi ed
Then Converged, Sl Op
Stoppi ngCriteria:
Ll X= X
- X
If Ll X
Ll X EI: Stop(desi gnnot changi ng)
If p = N.: Stop( maxi mumiterations reached)
Conti nue
p ←p+l
Go toStep2
Appl i catl on 0 ' the GRG Met hod: The GRG al gori thmis appl i ed 10 Ex倒 npl e
7.1. Once agai n the iIl ustrati on is through a set of cal cul ati ons genernted usi ng
MA1LAB. For a 5tart Exampl e 7.1 is rewri ttenusi nga sl ack vari abl efor th巴 i nequal i ty
Mi ni mi ze f (Xh x2・X3): x1- 2xrX2+ xt + - 2xl + 4
Su同国 tω H1(xl ・X2,X3):  
H2(XhX2):   +X3=O

For 山 is probl emn = 2, 1 = 1, m = 1, n + m = 3," - l = l ,l + m = 2.
Exampl e 7.1 is not a good probl emtoiIl ustratetheworki ng of theGRG method
because出巴 onl y feasi bl epoi nt where g is acti veis al so山 esol uti on. Thi s makes Z,
and 山巴 refore G

a scal 釘 . The exampl e is retai ned because of uni formi ty. In 山
i mpl ementati onbel ow, Z = [XI] and Y = [X2 X3)T.
Sec7 _3_3. m: 百l I s m- fiI e handl es Exampl e 7.1 through 血 e GRG method
symbol i cal l y. It requests a startingdesi gnwhi ch shoul dbe feasible. It sets up出 e Z
and Y automnti cal l yand computes [A) nnd [8]. Experi menti ng wi ththecode i ndi cates
that not all feasi bl epoi nts provi dea nonsi ngul ar [8]. It thenprompts theuser for two
v凶 uωof αwhi chit uses toperforma quadrati c inte中 ol ati on. It computes anopt i mum
sl epsi zeand compl etes theiterationfor d. Y. This∞mpl el es one iterationof 白色 GRG.
Experi menti ngwi th白 ecode iI1ustrates 出 at thechoi ces of αneed some thought for
theiteration10succeed. The fol l owi ngis transcri bedf romtheCommand wi ndowfor
two iterations. Note出 eprompts for al pha:
l tera“on 1
持 sec7_3_3
I nput t he f easi bl e desi gn chosen f or eval uat i on.
[ 1. 2 0. 74833] % Thi s i s x
The desi gn vect or [ 1. 2000 0. 7483 0. 2200]
f unct i on and const r ai nt s( f h1 h2) :
3. 6304e+000 - 2. 2111e- 006 0
2. 4000e+000
6. 0000e- 001
1. 4967e+000 0
1. 1225e+000 1. 0000e+000
- 6. 0183e+000 も ( Reduced Gr adi ent )
I nput t he st epsi ze f or eval uat i on.
も ( Pr ompt f or f i r st st epsi ze)
0. 1
al pha
1. 0000e- 001
No. of i t er at i ons and const r ai nt er r or : 29
1. 0000e- 001 8. 2187e- 009
desi gn vect or : 5. 9817e- 001 1. 2815e+000 - 3. 2110e- 001
f unct i on and const r ai nt s (f h1 h2)
3. 3548e+000 7. 2526e- 005 5. 4394e- 005
s ' "
- 6. 0183e+000
I nput t he st epsi ze f or eval uat i on.
0. 05
al pha
5. 0000e- 002
も ( Pr ompt f or second st epsi ze)
No. of i t er at i ons and const r ai nt er r or : 12
5. 0000e- 002 6. 2635e- 009
desi gn vect or : 8. 990ge・ 001 1. 0916e+000 - 9. 5823e- 002
f unct i on and const r ai nt s (f h1 h2)
2. 9702e+000 ・ 6. 3314e- 005 ・ 4. 7485e- 005
al pha
S5・6595e暢 002 も ( quadr at i cal l y i nt er pol at edα}
- 6. 0183e+000
No. of i t er at i ons and Const r ai nt er r or : 14
5. 6595e- 002 4. 3237e- 009
desi gn vect or : 8. 5940e ・ 001 1. 1231e+000 - 1. 3072e- 001
f unct i on and const r ai nt s (f h1 h2)
2. 9903e+000 ・ 5. 2604e- 005 - 3. 9453e- 005
Thi s ∞mpl e出 onei terati on. Thi s is far f romsati sfactoryas X3, thesl ack vari abl e, is
negati ve. The stepsi ze and desi gnchange cal cul ati onmust be mad巴 sensi ti veto出 e
si deconstr冨 ints. Inactual i mpl ementati onof theGRG, si nceg(X) Was i nacti veat 出 e
start, it woul d nol have been inc1uded inthecal cul ati ons at all. Usi ng thesol uti onto
s凶 t thenext addi ti onal i terati ons canbe obtai nedl eadi ng10thesol uti on.
l terati on2
Rerun Sec7 _3_3. mwi thX
= [8. 5940e-001 1. 1231e刊 )00]
The rest of 出 e i terati ons are left as an exerci se for 出巴 reader. To compl ete
Sec7 _3_3. mthecal cul ati ons for K T condi ti ons must be i ncl uded. Intheabove, si nce
g(X) is invi ol ati on, theK T condi ti ons were not computed. This is currentl yone of the
more robust methods. For this pa凶 cul ar exampl e theprogress of sol uti on is qui l e
sensi ti veto出 estepsi zechosen for i nterpol ati on.
7. 3. 4 Sequent l al Gr adl ent Rest orat l on Al gor l t hm( SGRA)
The SGRA is a two-phase approach to 出 esol uti onof NL P probl ems. Starti ngat a
feasi bl edesi gn, theGradi ent phase decreases theval ue of theobj ecti vefuncti onwhi l e
satisちIIng 自己 l i neari zed acti ve constrai nts. This will cause cons町 ai nt vi ol ati on for
nonl i near acti ve cons回 ints. The Restorati on phase bri ngs back the desi gn 10
feasibilitywhi ch may establ i sha new叩 ddi fferenl set of acti veconstrai nts. Thiscycl e
of two phases is悶 peal eduntil theopt i mumis f ound [15, 16]. The me出 odi ncOJ porates a
d凶 cent property wi th回 chcycl e. Li ke other al gori thms in血 is sωti on, it uses an 8Ctive
∞nstrョint strategy. In compari s叩 the GRG combi nes the two phωes togethぽ . 明 記
method has been favorabl ycomp81吋 toGRG and theGradi ent Proj ecti onmethod [17].
The SGRA as ori gi nal l yin位 。 ducedi n出 elistedreferences assumes thei nequal i ty
cons町 ai nts of thef orm
gj (X)注 0; j = 1,2,・…m
To be consi stent wi 白 血 edi scussi oninthebook it is changed tothestandardf ormof
NL P probl ems ぉ defi nedin出 is text. Inthefol l owi ngdevel opment of theal gori thm,
onl y thesalient features arei ndi cated. They areal so transcri bedconsi stent wi th出 e
other al gori thms in this secti on. It is 隠 commended 白 紙 the reader revi ewthe
references for a detai¥ed descri pti on. The general probl emis
Mi ni mi ze
Subj ect to:
[X]n f (X),
[g(X)] ", 0
In恥 SGRAonl y acti vein珂 ual i tyconstrai nts areof i nterest. Equal i tyconstrai nts are
al ways acti ve. Acti ve i nequal i ty constrai nl s al so inc1ude vi ol ated constrai nts. If 11
i ndi cates theset of acti veconstrai nts, 山 印 刷 uati ons (7.6) 加 d(7.7) arecombi ned i nto
a vecl or of acti vecons回 ints (や):
中( X) =[ぷ J ν
The number of acti vei nequal i tyconstrai nts is v - l. The Lagrangi anfor theprobl em
canbe expressedinl erms of 恥 acti vecons回 i臨 al one[si nce恥 mul ti pl i ers for 白 e
gJ {X) < 0 will be set to zero as part of K T condi ti ons]. 百 le K T condi ti ons a応 then
VxF(X, ν ) = V x. f (X) + [ V   (7.59a)
。( X) =O (7. 59b)
λ吋 ミ O (7.59c)
where [ V X+1 = [ V X+I V
x中z Vx'!>v]; ν=l λ1 九世 λ.JT
GrBdi ent PhBse: Gi ven a feasi bl edesi gnX' , fi ndthenei ghbori ngpoi nt
主s=Xi +AX
such that 5j < 0制 御 =O. I mposi ng a quadrati c constrai nt on thedi spl acement dX
[16] ,出 eprobl emcan be set up as an opti mi zati onsubprobl emwhose K T condi ti ons
determi ne
tl X= - aV Fx(X
, A.う =αS (7.60)
The i nterpretati on of the search di recti on S is used to connect to 山 e other
al gori thms in血 is book. It is not intheori gi nal devel opment. The searchdi recti onis
opposi teto出 egradi ent of theLagrangi an, whi ch is a novel feat山 宮 . To comput e出 e
Lagrangi an, l heLagrange mul ti pl i ers have tobe cal cul ated. Thi s is done by sol vi nga
systemofν Ii near equati ons
[ V V xj (X
) + [ V ほ う   = 0 (7.61)
St eps; ze f or Gr adl ent PhBse: The stepsi zeαcal cul ati onis basedon dri vi ng出 E
opti mal i tycondi ti ons inEquati on(7.59a) tozero. Therefo問 , if
- ( J . Vん (Xi ,i,.v)
thcnopti mumα ・is foundby cubi c i nterpol ati onwhi l etryi ngωsati sfy
VxF(え,')..l' l V
F ぽ,i..I' ) =o (7.62)
Care must betaken白 紙 this stepsizedoes not causesignificant constrai nt violation.
Thi s is 巴nfol"cedby cappi ngthesquarederror intheconstrai nts by a suitableupper
bound wJ uch is set upω
ゆ(主 g) T中(え ) S P
Reference 16suggests 伽 t Pm田 is relatedtoano出 er performance i ndex Q, whi ch is
出 eerror intheopti mal i tycondi ti ons:

• λ   , λ   (7.64)
Restorati on Phase: It isexpec旬 dthatat 白巴巴 ndof thegradi ent phase thefuncti on
will have decreased but 由 巳 陀 woul d be some constrai nt dissatisfaction (assurni ng
there was at least one nonl i near acti ve constrai nt at the begi nni ng of the gradi ent
phase). The restorationphaseestablishes a nei ghbori ngfeasiblesolution. 1t does this
by ensuri ng that the I i neari zed constrai nts are feasible. Pri or to 血 is. 出 e active
constrai nt set has tobeupdated (ii) si nceprevi ousl y feasi bl econstraints coul dhave
beC9me infeasible(andprevi ousl yinfeasibleconstrai nts coul dhave become feasible)
wi th白巴 desi gnchange causedby白 egradi ent phase. The desi gnvecωr andthedesi gn
changes for 也 is phasecanbe wri ttenas
X,=Xg+dX, (7.65)
The desi gnchange for this phasedX, is obtai nedas a I伺 st squaree.π' or inLhedesi gn
changes subj ect to 由 e sa出 faction of the lin阻 r cons住 aints. Setti ng up 印 NLP
subprobl emthis cal cul ati onfor 血 edesi gnchanges is
= - V xCT( X
)σν (7.66)
Here σ"i s theνvector Lagrange mul ti pl i er of thequadrati c subprobl em. The val ues
for themu1tipliers arees悩 bl i shedthroughthelinear equati ons
帥( 支 g) - VAds) TVJ dE) σν=0
Thefactor μi s a user-control l edp紅 ' ameter todi scouragel argedesi gnchanges.


The Restorati onphase is iterativelyappl i eduntil
中(支, lCT(主,) S 8
where EI is a smal l number.
At theconcl usi onof therestorati onphase白 econstrai nts arefeasibleandthenext
cycl eof Gradi ent-Restorati onph筒 ec岨 beappl i edagain. A summary of theabove
steps is capturedin白 eal gori thmgi venbel ow.
A匂 ori thm: Sequenti al Gradi ent Restorati on A旬。 ' ri thm( SGRA) (A7. η
Step]. Choose X
(must befeasible)
N. (no. of iterations).
8j ' S (for convergence and stoppi ng)
p = 1 (iterationcounter)
Step2. Execute Gradi ent Phase
Cal cul ateStepsi zeusi ngcubi c inte中 olation
Cal cul atexr
Step3. Execute Restorati onPhase
Step4. Convergence for SGRA
If h
= O. for k = 1. 2. . . . • 1;
If gj S 0, for j = 1. 2. . . . • /11;
If KT condi ti ons aresatisfied
Then Converged, Stop
Stoppi ngCriteria:
a X = XJ *I - X!'
I f aX
企xS EI : Stop(d巴 si gnnot changi ng)
If p = N.: Stop( maxi mumiterations reached)
Conti nue
p ←p+l
Go toStep2
Appl l cati on of t he SGRA伺 7. 刀: Exampl e 7.1 is usedtotest theappl i cati onof
theSGRA. The m・fileSec7 _ 3_ 4. mcontai ns thecode that appl i es theSGRA through
a combi nati onof symbol i c andnumeri c calculations. As in山 eprevi ous ex創 npl es it
iscustomi zedfor two variables. one equa1i tyand one i nequal i tyconstrai nt. Unl i kethe
GRG i mpl ementati onit i ncorporates 白 eacti veconstrai nt strategy. It executes one
cycl eof the SGRA whi ch has a single iterationin自己 Gradi ent phaseand several
iterations in山 eRestorati onphase. Inthegradi ent phasethecubi c i nterpol ati onof the
original techni queis 陀 pl acedby thequadrati c i nterpol ati onusedeverywhere inthe
book. The startingi nput is a feasibledesi gnvector. In出 egradi ent phasethemethod
reques包 a val ue for the stepsi ze to scan for the opt i mumぱ usi ng quadrati c
int巴中 olation. Thi s is i mportant for successi vecycl es start at theopt i mumval ueof the
stepsi zefor theprevi ous cycl 巴. The val ueof μi n Equati on(7.67) is set at 0.5. Pri nti ng
is controlI ed出 roughthesemi col onand白 ereader is encouragedωf ol l owdetails by
f宮 町 l ovmg11.
In the folI owi ng theprogramis run wi 白 山 e starting desi gn usedfor the GRG
method. 百 1巴 output f romthe Command wi ndowis pasted bel owwi th some bol d
annol ati ons 10drawauenti on.
>> Sec7 3 4
工npuヒヒ he f easi bl e desi gn chosen f or eval uat i on.
[ 1. 2 0. 74833] ( f easi bl e desi gn vect or i nput )
The desi gn vect or [ 1. 2000 0. 7483]
f unct i on and const r ai nt s( f h g) :
3. 6304e+000 - 2. 2111e- 006 - 2. 2000e- 001
Gr adi ent Phase
* * * * * * * * * * * * * * * * * * * 合 合 * *
The Lagr ange mul t i pl i er s
- 1. 0812e+000
I nput t he st epsi ze f or eval uat i on.
1 ( al pha used t o st ar t scanni ng pr ocess)
al pha f or quadr at i c i nt er pol at i on
o 1. 2500e- 001 2. 5000e・ 001
( al pha t hat br acket s t he mi ni mum)
f unct i on val ues f or quadr at i c i nt er pol at i on
1. 0142e+001 6. 6291e+000 1. 1300e+001
( cor r espondi ng f unct i on val ue)
opt i mumal pha: 1. 1615e- 001
The desi gn vect or [ 1. 0043 1. 0622]
f unct i on and const r ai nt s( f h1 h2) :
3. 0077e+000司 1. 3681e- 001 9. 8328e- 002
( bot h const r ai nt s vi ol at ed)
The per f or mance i ndi ces Q, P
6. 3181e+000 1. 8718e- 002
(P measur es t he squar ed er r or i n t he const r ai nt s)
合 * * * * * * * * * * * * * * * * * * * * * *
Rest or at i on Phase
* * * * * * * * * * * * 会 合 * * * * * * * * *
Number of Rest or at i on i t er at i ons: 21
The desi gn vect or [ 1. 0000 1. 0000]
ヨ !
世 l

f unct i on and const r ai nt s( f h1 h2) :
3. 0000e+000 6. 712ge- 005 4. 6741e- 005
( Const r ai nt s sat i sf i ed - 9 act i ve)
The per f or mance i ndex P
6. 6911e- 009
The sol uti onhas converged凶 oneiteration. The SGRA is of thesame cal i ber as th巴
GRG (some compari sons showit to be better). It is difficult to compare di fferent
al gori thms because of di fferenl codi ng strucl ures and nonstandard i mpl ementati on
details. Neverthel ess theSGRA has consi derabl emeri t. It is defi ni tel ybetter thanthe
GRG as far as Exampl e 7.1is concemed (Secti on7.3.3). Thi s compari son may not be
fair s ~ n   e theacti veconstrai nt strategywas not usedfor theGRG even though it is part
of theGRG. Thi s demonstrates theneed for sucha strategyinall al gori thms 白 atdeal
wi 白 cons回 i nedopti mi zati on.
Threeaddi ti onal exampl es arei ntroducedhere10ilI ustrateand expand the凶 eof 由巳 se
techni ques. Duri ng theearl ydevel opment of thesetechni quω 由巳 rewas a si gni fi cant
emphasi s on effi ci ency, robustness, and keepi ngcomputati onal resources as l owas
possi bl e. Most of 山 e techni ques were devel oped on mai nf rames usi ng FORTRAN.
Today Ihesearchfor gl obal sol ul i ons (出 at arel argel yheuristic) , coupl edwi thsi mpl e
cal cul ati ons rep伺 led al most endl essl y, performed on dωkt op PCs wi 白 i ncredi bl e
computi ng power, I hroughsoftwru芭 systems 出 al provi dea wi de range of resource唱
has shi ftedthefocus toexperi mentati onand creativity. The author feels suchan effort
must be coupl ed wi th knowl edge and i nsi ght gai ned through a wi de sel ecti on of
appl i cati ons 佃 d understandi ng白 eprogress of numeri cal cal cul ati on. Inspiteof all
白 e expos町 e so far in the 民 民 for many real desi gnprobl ems the sol uti on is not
automati c. D国 ling wi th nonl i near equati ons is a chal l enge and often surpri si ng.
Tryi ng 10 understand why the l echni que does not work is as much a l earni ng
expel"色 nceωarouti nesol uti ontotheprobl em. Si ncel heseexampl es use themel hods
of 山 ep問 vi ous secti on wi thdiffer芭 nt functi ons, all of thecode devel oped before is
revi sedwi thand theappropri atechanges and renamed. 泊 施 semodi fi cati ons 偲 qui re
that f ewlines be changed f romthe ori gi nal code. 百 lis al l ows the user to make
i ncremental changes as needed. Thi s is not an efficient approach bUl capi tal i zes on the
unl i mi ted hard dri ve space avai l abl e today. Students are urged to understand the
al gori thm, thefl owof thecode, and出 eresult bei ngsought .
7.4.1 Exampl e 7. 2一 円 agpol ePr obl em
Thi s is thesame probl emwhi ch was sol vedgraphi calI y inChapter 2, anal yti cal l y
in Chapter 4, and now consi dered for numeri cal sol uti on. In 出 is subsecl i on the
scal ed versi on of I he probl emis sol ved. Fr omChapter 4, Exampl e 4.3, it is
rei ntroducedas
Exampl e久 2
Mi ni mi ze (7.69)
gl: 2.55ωヰ+O.l 36l x
- 33.148( x1- 0 (7.70吋
g2: (0. 似 88+

  イ 内 ー   - 0 (7.70b)
む 1 . 0868x
+ 0.3482 - 細 川 (7.7Oc)
g4: x
- x
+0. 001
0. 02 x 1 1. 0; 0.02 x
1. 0
I ns勿 ht about Numer i cal Sol uti on: In Chapter 4 it was i nvesti gated 出 創 出 e
soI uti ondoes not satisfytheKT condi ti ons because th巴 sol uti onis not a regul ar poi nt.
It was ascertai ned伽 t 曲 egradi ents of 出 etwo acti veconstrai nts 伽 t determi ne白 e
sol uti on(810g3) arenearl yparallel inthenei ghborhoodofthe solution. How will this
affect thenumeri cal searchfor thesoI uti on?
Typi caJ ly what a numeri cal techni quedoes is pi ck a useful di recti onand seehow
best it canmove al ongit. Suppose thetechni quedeci des tomove al ongthegradi ent
of 83todevel opit intoanacti veconstrai nt . Thcre is obvi ousl y goi ngtobea probl em
inpushi nggl tobecome active. Pushi nga constrai nt to be acti ve is a basi c way to
reduce the obj ecti ve functi on in wel l -formul ated NLP probl em. Therefore, most
al gori thms (unl ess 曲 ey are trai ned to compromi se in some si gni fi cant way) wilI
underachi eve by onl y pushi ngone ofthe constrai nts tobe acti veand ensuri ngtheother
is feasible.
ALM_ 7 _ 4_1. m: This m- fiI e uses theAL M method 10sol ve山 eprobl em. lnorder
topreservetheol dALM. mto陀 flect thedi scussi oninSecti on7.2.2a copy of thefile
is made. Of un341. m, Hf un341. m, Gf un341. m, and FALM341. mreflect 白 e
functi ons f romExampl e 7.2. A1so, ALM_7_4_1. mis adj usted ωavoi d handl i ng
equal i tycons町 aints 叩 dprovi desome pri nti ngsupport. 0山 erwi seit is essenti al l ythe
same. Once agai n the output f romthe Command wi ndowis p邸 ted bel owand
annotatedtodrawattenti ontosome issues. Onl y 由巴 initial prompts wi 山 thei nputs,
theinitial iteration, and theconsol i datedi nformati onafter thefinal i temti on眠 copi ed
bel ow.
持 ALM_7_4_1 ( i nvoke pr ogr am)
I nput t he st ar t i ng desi gn vect or
Thi s i s mandat or y as t her e i s no def aul t vect or set up

The l engt h of your vect or i ndi cat es t he number of
unknowns (n)
Pl ease ent er i t now and hi t r et ur n f or exampl e [ 1
2 3 . . . ]
[ 0. 75 0. 55] ( i ni t i al desi gn vect or )
The i ni t i al desi gn vect or :
7. 5000e- 001 5. 5000e- 001
I nput t he mi ni mumval ues f or t he desi gn vect or .
These ar e i nput as a vect or . The val ues ar e
3 * st ar t val ue bel ow t he st ar t val ue unl ess i t i s
zer o.
I n t hat case i t i s - 5:
[ 0. 02 0. 02] ( i nput l ower bound not cur r ent l y used)
The mi ni mumval ues f or t he desi gn var i abl es ar e
2. 0000e- 002 2. 0000e- 002
I nput t he maxi mumval ues f or t he desi gn vect or .
These ar e i nput as a vect or . The def au1t va1ues ar e
3 * st ar t val ue above t he st ar t val ue.
工f st ar t val ue i s 0 t hen i t i s +5:
[1 1] ( i nput upper bound - cur r ent l y not used)
The maxi mumval ues f or t he desi gn var i abl es ar e 1 1
Number of equal i t y const r ai nt s [0]
( def aul t val ue of 0 i s ok)
Number of i nequal i t y const r ai nt s [0] 4
I ni t i al val ues f or bet a [1 1 1 1]
( i nput number of i nequal i t y const r ai nt s and i ni t i al
bet a)
ALM i t er at i on number : 0
Desi gn Vect or ( X) 7. 5000e- 001 5. 5000e・ 001
Obj ect i ve f unct i on 2. 6000e- 001
( I n t he f ol l owi ng vi ol at ed const r ai nt s ar e shown i n
bol d)
I nequal i t y const r ai nt s - 5. 9180e+000 - 3. 5997e+001
5. 2383e+000 - 1. 9900e- 001
Squar e Er r or i n const r ai nt s( h, g) 0 0
Lagr ange Mul t i pl i er s ( l ambda bet a) : 0 1 1 1 1
Penal t y Mul t i pl i er s ( r h r g) : 1 1
ALM i t er at i on number : 4
Desi gn Vect or ( X) 6. 4726e・ 001 6. 1013e- OOl
Obj ect i ve f unct i on: 4. 6695e- 002
Er r or i n const r ai nt s (h, g) 0 2. 1366e・ 009
Lagr ange Mul t i pl i er s ( l ambda bet a) : 0 0 4. 8227e・ 002
Penal t y Mul t i pl i er s ( rh r g) : 1000 1000
The val ues f or x and f and 9 ar e
7. 5000e- 001 5. 5000e- 001 2. 6000e- 001 0
6. 5647e- 00l 6. 0173e- 001 6. 8871e- 002 0
6. 4726e- 001 6. 1017e- 001 4. 6638e- 002 1. 5525e- 006
6. 4726e- 001 6. 1013e・ 001 4. 6695e- 002 2. 1366e- 009
6. 4726e- 001 6. 1013e- 001 4. 6695e- 002 2. 1366e・ 009
( Not e desi gn and f unct i on ar e not changi ng hence
st opped)
The val ues f or bet a ar e
1. 0000e+000 0 0 0
1. 0000e+000 0 0 0
1. 0000e+000 1. 4062e- 002 3. B9B3e- 002 4. 8227e- 002
1. 0000e+000 8. 9252e・ 001 1. 7073e- 001 0
( Each col umn r epr esent s val ues of bet a f or an
i t er at i on - bet a f or const r ai nt s 1,2 and 4 ar e zer o and
i t can be seen bel ow t hey ar e f easi b1e but i nact i ve.
Onl y t he t hi r d const r ai nt i s act i ve)
I nequal i t y const r ai nt s ・ 6. 7858e- 002 - 4. 9734e+OOO
4. 6223e- 005 - 3. 6136e- 002
( Not e t hi r d const r ai nt i s act i ve. The f i r st one i s
al most act i vel
The anal yti cal sol uti onfor 出 is probl emis
X;=O后 774; 弓 =0. 6445; I = 4.3577E - 02;
g; -O. O; g; =- 4.9837; g; =O. O; g; =- O. 319也 - 02
Incompari son, theAL M has done a decen1j obof geui ng cJ ose 10thesol uti onbefo悶
staIli ng. Some of 白 is canbe traced10自 己 di scussi onpri or 10runni ng山 eprogram.
7. 4. 2 Exampl e 7. 与 一 BeamDesl gn
Thi s sec1i on r叫 uires 白 e use of 出 e Opti mi zati on Tool box f romMATLAB. Mor e
speci fi caJl y, it uses 1heQP functi onωsol ve 1heQP subprobl emintheSQP me曲。 d.
The probl emis sol vedusi ngthedirect proced町 eilI ustral edthroughtheexampl e in
Secti on7.3.2. 噴 出 secti oncombi nes thesymbol i c capabi l i tyof MA札 油 明 白 theresou民 凶
avai l abl eintheOpti mi zati onTool box toi mpl ement anorigina1interpretationof theSQP.
泊 施 exampl e is 白 edesi gnof anI -beamfor useina par討c叫 ars回 ctura1probl em. 百 出 is
a 回 ditiona1 m∞hani caVcivil engi neeri ng strucωra1 opti mi zati on probl em. It was
i ntroducedinChapter 1 for ilI us回 tion. It a1soappears inChapters 8佃 dI Ofor iIl us回 tion
of addi ti ona1issues. I n血 eexampl e here,出 eI -b担 mwiI1回 rrya wei ght at 福 田 R包 r. 百 le
田 n凶 cl oad泊 g佃 d血 edesi 伊 vari abl esfor 出 eprobl emaredefu:ted加 F i gure8.7(Chap町
8). 百 lefigureisnot reproducedhe時 but X I is thedepthof thebeam, X2 is thefl angewi dth,
X3 i sthew出 血 i ckn郎 . 四 dX4i s 出 ef1ange tJucknωs. Other parameters (g, ム W. E,σpτ y)
arenotr吋 efi nedhe毘 .
百 lemathemati ca1model is
Mi ni rni ze J (X) = pLAc
Subj ec1to: gl (X): WL
- 12EI 0.25 (de自 国 tionconstrai nt)
82(X): WLx
- 8/1σy 0 (normaI stress constrai nt)
g3(X): WOc- 21x
τ: y:S; 0 (sh間 町 ss co田 町 血 t)
Some geometri c cons1rai nts torelatethevari ous desi gnvari abl es are
g4(X): XI - 3X2 0
8' (X): 2x2 - XI 0
g6(X): X3 - 1.5x4 0
g7(X): 0.5x4- X3 0
The si deconstrai nts on出 edesi gnvari abl es are
3 :S; x
20, 2 15, 0. 125:S; x
0.75, 0.25:S; x
:S; 1. 25
The fol l owi ngrel ati ons defi ne出 ecross-seclI onal area(A
), moment of i ner由( 乃,印 d
伝 説 moment of areaabou1thecentroi d(Qc) interms of thedesi gnvari abl es:
Ac=2ら X
- 2x
ー ら
xf (ろ -   - 2x3)3
一 一 ー
12 12
Qc=0. 5x
I - xJ + 0. 5x3(x1 - xJ 2
The desi gnparameters for theprobl emareW = 1.4x 2000 (1.4is 山巴 factor of safety) ,
L = 96, E = 29 E+06, p = 0. 284,σy=36 E+03,τ:,=21 E+03.
EX343. m: Thi s is 出 e m・file 由 al wi l l i mpl ement the Sol ul i on l hrough the SQP.
Whi l e this is a tradi ti onaI desi gn probl em. it is aI so nol trivial . It is rare to fi nd a
publ i shed detai l ed procedu問 for sol uti on. The sol uti on is usuaI ly obtai ned
numeri cal l y. Inany event. anaI yticaI appl i cati onof the K T condi ti ons is usuaI ly not .
contempl ated because of 2
cぉ ω( f or 山 e7 constrai nts) even if many of thosecases
are trivial . IncI usi on of the si de constrai nts wi l l provi de an addi ti onaI 8 l i near
cons剛 山 (agrandto凶 of 15constrai nts) of thef orm(iII us凶 tedfor X) onl yぉ)
gs(x): - x
+ 3 S 0
gI 2(X): x
- 20S0
The reader is strongl yr ecommended 10revi ew出巴 codeinthem- fiI e for 由 ef ol l owi ng
・ The power of symbol i c mani pul ati on is 回 pl oi ted further usi ng vector
defi ni ti ons 10make the i mpl ementati on fairlyeasy. The symbol i c cal cul ati ons
areused togenemte山 edal afor theQP subprobl em.
• Th巴 symbol i c cal cul ati ons are effortl essl y i nl egral ed wi th numeri cal
computati ons (th巴 sol uLi ontotheQP is numeri cal ).
• The i mpl ementati on of 恥 SQPis s凶 i ghtforwardand di rect (andsi mpl e).
・τ'he sol uti on is si 伊 i fi cantl y better 白 an the numeri cal sol uti on by MATL油 、
own SQP program( Chap町 10).
The programrequi res th巴 starti ngguess for thedesi gnvector. The functi ons for the
exampl e ar芭 hardcoded. The final sol uti onis pri ntedtotheCommand wi ndow:
Initial Guess:
Fi nal Desi gn:
Obj ecti ve:











AlI constrai nts ar芭 satisfied:
Ac¥i ve constrai nts: 1 (defl ecti on). 7. 9 (X2ー l owerbound). 10(.均 一 l owerbo叩 d).
11 (X4ー l ower bound)
The reader is encouraged toexpl orewhy thesol uti onobtai nedhe問 is si gni fi cant1y
better 出 回 theone usi ngtheSQP f romthetool box. grantedthat thei mpl ementati ons
arenot thesame. Some thought shoul dbe gi vento山 penormanceof quasi - Newton
(me¥ri c) methods and their adaptati on¥0constrai ned l echni qu巴 sif theHessi an is no¥
posi ti vedefi ni te, whi ch inthis case it is.
7. 4. 3 Exampl e 7. 4- 0ptl mal Cont rol
This secti on i ntroduces another cI ass of probl ems 白 紙 C加 be addressed by desi gn
opti rni zati on techni ques. It is the basi c traj ec¥ory determi nati on probl eminopti maI
control . Gi ven t wo poi nts ina pl ane, fi nda pa出 血 at wi1I be tr温 versedbetween t hem
usi nga steeri ng v田 tor of constant magni tude. The pathhas to be釘 aversedinuni t
ti me. The probl 巴mc叩 bedescri bedusi ngFi gure 7.11. If theprobl em回 nbe descri bed
in出 ephysi cal pl ane. thesl 田 ri ngvector consi deredas vel oci ty. 出 en theki nemati c
叫 uati ons for a parti cI e movi ng on 白 e pa山 can be descri bed by the fol l owi ng
di fferenti al equati ons (d邸 i伊 opti mi zati onprefers todeal wi tha1gebrai c functi ons).
whe隠 X. Yl ocate出 eprobl eminphysi cal space:
土 =V cos 8(t); 0 S t S 1
タ=V sin8(1); 0 S t S 1
The initi aI and final condi ti ons are(t11esewere theval ues chosen for theexampl e)
λ(0) = 0; y(O) = 0
x(l ) = 3; y(l) = 2
Here. 8(t) is regard巴das a control vari abl e. II is a conti nuous functi onof t between 0 S
t S 1 that will transfer thepa凶c1ef rom自己 initial poi nt 10thefinal poi nt . Incontrast.
desi gnopti mi zati on(i. e.. ¥his book) determi nes scal ar val ues of thedesi gnvari abl es.
These types of probl ems bel ong to the area of opti mal control . Th巴 閃 are speci al
numeri cal techni ques 出 al are avai l abl e 10 sol ve I hese categori es of probl ems. One
approach totheseprobl ems is tousedesi gnopti rni zati ontechni ques. This is ai dedby
usi nga parame町 ic representati onfor thecontrol functi on. Many di fferent par滋 ne凶 c
隠 p問 sentati ons for the control have been used. such as cubi c spl i nes and Hermi te
pol ynorni al s. I n this s¥udy a Bezi er curvc is used to rep陀 sent the control. The
di fferenti al equati ons ar宮 山 ennumeri cal l y i nl egrated10es阻 bl i shth巴町司 j ecl ory.
FI gure7川 Descri ptl onof Irajeclory∞nlrol. 1告書.
whi ch requi res br achi . mto retumthe state space defi ni ti on of 由巳 probl em.
br achi . mal soi ncl udes theBezi er i nterpretati oncode.
Fi gure 7. 12descri bes 出 eBezi er ve氏 i ces of thecontrol functi onat thestanand出巴
final iterations. The final conti nuous control is al sopl ottedon the figure. The start
curve is a s位 ai ght linethroughtheverti ces and is not shown. Fi gure 7. 13shows 白 e
start traj ectory組 d出 efinal 位司 j ectory. It is cl 回 rf rom白 efi gure白 紙 白 einitial and曲 e
final condi ti ons are sati sfi ed for the final tr司j ectory. Si nce 白 e equati ons we陀
i ntegrated. thedi fferenti al constraI nts arethereforecompl etel y satisfied.
The final tr吋 ectory shown is onl y feasi bl eー it is one of many一 白 紙 canp蹴
betw田 nthet wo poi nts. Thi s opens thepossi bi l i ty山 at a further op自 国 国 ti onc卸 be
perf ormed wi th毘 spect to出巴佐高 j ectory. Forexampl e. fi nd出 eshortest pathin出 eti me
provi ded. Not巴 it is not 白 e strai ght pa山 si nce for a gi ven vel oci ty magni tude the
desti nati onmay be犯 achedsooner 山 anp問 scri bed. The c¥assical probl eminopti mal
∞ntrol is 曲 e brachi sl ochrone probl em. lt is 出 e mi ni mumti me for thep紅 白 le 10
tr祖 vel between t wo prescri bedpoi nts dynami cal l y (obeyi ngpri nci pl es of dynami cs).
The reader is cncouraged to expand 由巳 techni que and 山 e code to expl ore 自己 se
addi ti ona¥i nteresti ngprobl ems.
Sol utl on Techni que (adoptedin出 is secti on)
• Use a Bezi er curvetoparametri zethecontrol functi on.
• I ntegrate the differential equati ons forward usi ng a Runge- Kutta met hod
(ode45. m)
• Use叩 uncons回 i nedl echni que(f mi nunc. m) mi ni mi zer to命 i ve山 esquareof
出 eerrori n白 efinal condi 曲 目 tozero0回 st squar吋 error).
Feasl bl etroJectory
Thi s exampl e nωds the M ATI . AO tool box to execute f mi nunc. m. Al tematel y. 出 e
unconstrai nedmi ni mi zer can be among thoseprevi ousl y devel oped. 百 is transl ates
inloan uncons回 i nedmi ni mumprobl em, whi ch thereforeshoul dhave been handl ed
in Chapter 6. It is inc¥uded here because 山 e di fferenti al equati ons are actual l y
consi deredd留量 renti al consl rai nts. The obj ecti vefuncti onthereforeis
f {X}=[ ズ 'finnl-x(l )f + [Yfinal- y( l) J2
Ex744. mwill executetheexampl e and produce pl ots of thecontroI and thetr司j田 tory.
It uses t wo standardMATLAO functi ons menti oned before(odc45. mand fmi nunc. m).
The Bezi er curve generati onr叫 ui res coef f . m. Combl nat l on. m. Factorl al . m. 百 le
obj ecti vefuncti onis constructedinobLopt cont . m. It 陀 qui 問 stheuseof ode45. m.
M泊 泊 語 ze
The, vertl ces and 0 (1)

llnal polnl ,

3.5 2



- - -

>- 1.5
3.5 3 2.5
Fl gu陪 7.13 Trajeclories→tart 則 前 nal .
1.5 0.5
- 0.5








0.9 O. 自 0.7 0. 6
Control funclion0(/) ,
tl me
Fl gu悶 7.12
0.3 0.2 0.1
1. Fi acco, A. V. , and McConni ck, G. P. , Nonl i near Progral l l mi J l g・Sequenti al
Unconstrai l l edMi ni mi zoti ol l Techni ques, Wi l ey, NewYork, 1968.
2. V畑 氏 rpl 細胞, G. N. , Numeri cal Opti mi zoti on Techl l i ques f or Engi neeri ng Desi gl l ,
McGr aw- HiII NewYork, 1984.
3. ANSYS Software, Ansys Inc. , Pi ttsburgh, PA.
4. Bertekas, D. P. , Constrai nedOpti mi zoti onand Lagrange Met hods, Academi c Press, New
5. KeJl ey, J. E. , The Cutti ngPl an巴 Me白 od,J oumal of SI AM, Vol. 8, pp. 702- 712,1960.
6. Gill , P. E. , Murray. W. , andWri ght , M. H. , Practi cal Opti mi zoti on, Academi c Press, New
7. Boo1 , J. C. G. , Quadrauc Programmi ng, Sl udi es itl Mathemati cal and MQl l ageri al
Economi cs, H. Theil ( 巴 dよ Vol. 2, North-Hol l and, A msterd町 n,1964.
8. Bi ggs, M. C. , Constr泊 nedMi ni mi zati onUsi ngRecursi veQuadrauc Programmi ng: Some
Al temateSubprobl emFonnul auons, Towards Gl obal Opt初 IIzoti on,L. C. W. Di xonand
G. P. Szego (eds.) , pp. 341- 349, North-Hol l and, Amsterdwn, 1975.
9. Powel l , M. J. D. , A Fast Al gori thmfor Nonl i near Consl rai nedOpti mi zauonCaI culalions,
No. DAMPr P77I NA 2, Uni versi tyof Cambri dge, Engl and, 1977.
10. Han, S. P. , A G10bal l y Convergent Method for Nonl i near Programmi ng, J oumal ザ
Opti mi zoti onTl reoryand Appl i cati ons, Vol. 22, p. 297, 1977.
11. Arora. J. S.. I ntrnducti ol l l oOpl i mwn De. ri sn, McGr nw- HilI, NewYork, 1989.
12. Branch, M. A., andGrace, A., Opti mi zoti otl Tooめ 'ox,User' s Gui de, MathWorks lnc. , 1996.
13. Wol f e, P. , M白血 ods of Nonl i near Programmi ng, Recenl AdvQl l ces i/ l Mathemal i cal
Pr ogr ammi ng, R. L. Graves andP. Wol f e (eds.), McGr nw- HiIl, NewYork, 1963.
14. Gubri el e, G. A. , DndRagsdel J , K. M. ,百 leGcnerol i zedGradl cnt Method: A ReliableTool for
Op伽aI Design. ASME J oumal of Engi neeri ngand I ndJ凶 り " SeT加 B,Vol . 99, May 19η.
15. Mi el e,A吋 Huang. H. Y. ,加 dHei deman,J.C. ,S叫 uenti aI Gradi ent RestorauonAI gori thm
for the Mi ni mi zauon of Constrai ned Functi onsーOr dinary and Conj ugate Gr adient
Versi ons, J oumal of Opti mi zoti onTheoη and Appl i cati ons・ , Vol. 4, No. 4,1969.
16. Levy, A. V. , and Gomez S. , Sequenti aI Gradi ent-Restorati on AI gori thmfor the
Opti mi zauonof a Nonl i nωr Constrai nedFuncti on. J oumal ψ 'heAstronauti cal Sci etl ces,
Speci al l ssueon Numωi c叫 Me曲。 ds inOpti mi zauon, Dedi catedtoAngel o Mi el e, K. H.
Wel l (sp. ed.), Vol . XXX, No. 2. 1982.
17. Rosen, J. B. , The Gradi ent Proj ecti on Method for Nonl i near Programmi ng, Part D:
Nonl i near Constrai nts, SI AMJ oumal of Appl i ed Mathemati cs, Vol. 9, No. 4, 1961.
7.1 Modi f y Sec7_2_1. J )l oLmto revi ewcontour val ues and pl ot automati cal l y
sel cctedcontours.
7.2 Modi f y Sec7 _2_1_cal c. mto automati cal l y cal cul atethe initial mul ti pl i ers,
l oopover pena1ty mul ti pl i ers. and toautomati cal l y pi ck thedesi gnvector 白 紙
satisfies thesi deconstrai nts.
  ~ (uses Opti mi zati on Tool box) Devel op山 eSLP by usi ngtheL P sol ver 合 o mthe
Opti mi zati onTool box and appl y toExampl e 7. 1.
7. 4 Sol ve Exampl 巴 7.1 by sol vi ng山 eQP probl emgraphi ca1ly.
7.5 Dcvel op自己 code to fi nd a feasi bl e sol uti on f roma gi ven sl arti ng vector for
叩 nslt司i nedprobl ems.
7. 6 Devel op a programtocal cul atethemul ti pl i ers for theNL P probl emat a gi ven
desi gn.
7.7 Devel op a programfor a constrai nedstepsi zecal cul ati onprocedure.
7. 8 (uses Opti mi zati onTool box) Use theQP programf romthetool box and devel op
your own SQP i mpl ementati on.
7.9 (uses Opti mi zati on Tool box) Use 出 e SQP programf romMATLAB to sol ve
Exampl e 7.1.
7. 10 Devel op Sec7 _3_2. m to automati ca11y execute severa1 i terati ons to
7. 11 I mpl ement a newversi onof SQP where theHessi an matri x is mai ntai ned as the
i denti tymatri x for a11iterations.
7. 12 Fi ni shsol vi ngExampl e 7.1 usi ngtheGRG me出 od.
7. 13 Sol ve Exampl e 7.1wi thZ = [X2] and Y = [x, x)]T.
7. 14 Modi f y the GRG code to i ncl ude consi derati on of acti ve constrai nts and
comp脱 出 eperf orm剖 ce.
7. 15 Bui l dinK T condi ti oncal cul ati oni ntotheSGRA.
7. 16 Sol ve theBrachi stochrone opti mal control probl em.
Thi s chapter i mroduces some concepts, methods, and al gori thms associ ated wi 白
di screte opti 凶 zation. It is not possi bl e to compress 出巴 su均配 t of Di screte
opti mi zauon to a si ngl e chapter as the topi c itself is capabl e of spawni ng several
courses. Di screte opti mi zati on is very di fferent, difficult, di verse, and conti nues to
devel op even today. These probl ems are l argel y combi natori al and are
compu凶 ona1ly more ti me i ntensi ve 白 叩 恥 correspondi ngconti nuous probl ems.
The probl ems addressedby thedi screteopti mi z凶 onresearchcommuni ty aremai nl y
in白巴 釘 e伺 aof op 戸eratioロ回s児 邸s鈍e釘 卸 制 chch紅 回a低 c低削riz吋 usual l yby li泊nea 訂r口mo
opti mi zati o 叩 nmostl yinc∞or中porat,匂.esno 叩 nl i ne伺 ar口re 芭叫la 瓜副tio 叩 n】 IS. Fr oma real perspecti ve, di screte
desi gn vaI tabl esMBf undamental i nengi needng opti mi zati on. h 出 e beamdesIl m
p帥 l巴 m in Chap総 r 1, a p削 ical sol uti on shoul d i nvol v巴 i denti fyi ng an
‘off-the-shel f' beamas 出 erol l i ngrnill will probabl y make it prohi bi ti vel yexpensi ve
for a l i mi tedproducti onof 出巴 uni quebeam白 紙 was i denti fi ed路 thesol uti onto出 e
conti nuous opti mi zati onprobl em. Intheprobl emregardi ng白 enumber of di fferent
pl acement machi nes the sol uti on was expected to be i ntegers (di screte val ue).
平rnilarl 円 choi ce of di an附 rs,l engths, washers, val ves, men, components,陶 ck
SlzeS, 叩 G soon,areusual l ygui dedby avoi di nghi ghprocurement costs associ ated
wi 曲 nons凶 dard components unl ess cost recovery is possi bl e because of l arge
vol umes. Co 叩 n 凶】 u刷ti 加IOl 附 I l
i tem路s to be compl etel y manufactured i 加nル帥"占h加ou 凶 s巴 (no oflι 恥 恥 lf com戸 nenl is
necessary) ・
Practi cal engi ne泡 ri ng desi gn 陀 quirl邸 由 al some desi gn vari abl es bel ong ( 0 an
ordered set of val ues. -di screte vari abl es. Thi s makes it a di screte opti mi zati on
probl em. However, there are very f ewengi neeri ng probl ems 白 紙 are currentl y
sol vedas a di screteopti mi zati onprobl em. Fi rst, di screteopti mi zati onal gori thms
are difficull to appl y. Second, they are ti me consumi ng. Thi rd, most di screte
al gori thms and code address l i near mathemati cal model s. Di screteopti mi zati onin
engi neeri ng wil1 necessari l y i nvol ve adaptati ons f rom白 e currentl y avai l abl e
techni ques used by the deci si on-maki ng communi l y. Such adaptati ons ar巴
uncommon. It is al soraretofi nd叩 y book on desi gn opti mi zati on白 紙 addresses
or i ncl udes thesubj ect of nonl i near di screteopti mi zati on, notwi thstandi ngthefact
出 at it is enormous inextent. A modest effort is bei ng made inthis book toacquai nt
thereader wi ththesubj 巴 ct area.
百 letypi ca1approach for i ncorporati ngdi scretedesi gnvari abl es inengi neeri ngis
tosol vethecorrespondi ngconti nuous opti mi zati onprobl emand adj ust theopti mal
desi gnto出巴 nearest di screteval ues (白 is is sirnilar totheroundi ng process toarri ve
at ani nteger number). Accordi ng toFl etcher [1], thereisno guarante巴 白 紙 白 i sprocess
is correct or 白 紙 agood sol uti oncan be obtai ned出 is way. Very often出 is roundi ng
may result in i nfeasi bl e desi gns. Gi ven 白 紙 the al temati ve is to sol ve the di screte
opti rni zati on probl emitself, a systemati c approach to 山 is roundi ng process has
become acceptabl e. A roundi ng process 白 紙 is based on mai ntai ni nga feasi bl edesi gn
is avai l abl einReference 2.
τ'hischapter and thenext wi l l differ si gni fi cantl y合 o mtheprevi ous chapters inboth
白 血 content and organi zati on. Thei r pri mary focus is presenti ng newi deas and
content rather 白 andevel opi nga techni queor assi sti ngincode dev巴 l opment . Di screte
probl ems requi re si gni fi cantl y di fferent concepts 山 間 出 e traditional conti nuous
mathemati cal model s of theearlier chapters. Si mpl e exampl es areusedtobri ngout
白 e di fference. For exampl e, deri vati ves, gradi ents, and Hessi an do not appl y to
di screte probl ems. By extensi on, search di recti ons and one-di mensi onal stepsi ze
computati on l ose their rel evance. Whi l e 白 ere are s巴veral excel1ent references on
di screteopti mi zati on, al most all of t hemdeal wi thl i near mathemati cal model s. There
are f ew, if any, about nonl i near di screte opti mi zati on appl i ed to engi neeri ng
probl ems.
Di screteopti rni zati onprobl ems (DP) arei mpl i edwhen thedesi gnvari abl es arenot
conti nuous. Extendi ng出巴 concept of thesi deconstrai nts, thesedesi gnvari abl
加 teger val ueof 8 =>bi nary val ueof 1000
Therefore, if 出 edesi gnvari abl ex is res凶 ctedtoi ntegers between 0 S; x S; 16, thenx
can be問 pl acedby fi ve0- 1 desi gnvari abl es [y.. Y2, Y3, Y4・YsJ f romwhi ch x c加 b巴
assembl ed as
x = YI (2
) + Y2(2
) + Y3(2
) + Y4(2
) + YS(2
Such a transfoI I Dati on is not recommended if 白巴 re is a l arge number of i nteger
vari abl es. Engi neeri ngdesi gnprobl ems can be expectedtocontai nbothconti nuous
and di scre飽 vari abl es. These aretermed Mi xed Programmi ng ( MP) probl ems if 出 e
mathemati cal model is line低 Inthis book thecl assi 白cati onis extended tononl i near
probl ems too.
Intheinterest of manageabi l i ty, onl y threemethods 釘 epresentedin出 is chapter.
These methods are onl y representati ve and inno way address 由巳 topi c of di screte
op山 ni zati onsuf白ci entl y,let al onecompl etel y. The thi rdone has seenl i mi teduse, but
wi thcompu匂 tional resources not b巴i nga hurdl etoday, it is defi ni tel yattrac首ve. The
sel ecti on does not represent any pri ori ty, though血 emethods are among those出 at
have been appl i edofteninengi neeri ng. The methods of thenext chapter have evol ved
through their initial appl i cati ontodi screteopti mi zati on. They coul dal sobel ong to
血 is chapter but have been kept sep紅 ate because 出 ey are the dri vi ng force in the
search for gl obal opt i mumtoday. The methods of 出 is chapter are (l ) E油 austi ve
Enumerati on, (2) Branch and Bound (partial or sel ecti ve enumerati on), and (3)
D戸I ami cProgrammi ng.
In the openi ng di scussi on some defmi ti ons of di sc閃 te programmi ng probl ems,
regardi ng the nature of di screte opti mi zati on were in紅 oduced. l n 出 is secti on the
concepts aredetai l edusi nga si mpl e unconstrai nedmi ni mi zati onexampl e. As a result,
白 e opti mal i ty issu回 do not in加 de on presenti ng the i mportant i deas in di screte
opti mi zati on i ncl udi ng the treatment of fi ndi ng a conti nuous sol uti on fol l owed by
adj usti ngthedi scretedesi gnvari abl es tonei ghbori ngdi screteval ues.
Exampl e 8.1 Mi ni mi ze出巴 obj ecti vefuncti ongi venbel ow, wherexl is a conti nuous
vari abl eand X2, X3aredi scretevari abl es. X2must have a val uef romtheset [
3.5J and x3must have a val ue合u mtheset [0.220. 75 1. 732. 24 2.78J.
Mi ni mi zef (x


) = (x
- 2)2+ (x
- ~   2 + (x
- X
)2+ (x
- x
)2 (8. 1)
The si deconstrai nts on thedesi gnvari abl es canbe set up as
E R (8.2a)
E 卯 .5 1. 5 2.5 3.5J E xu (8.2b)
ε [0.22 0. 75 1. 73 2. 24 2.78] E X
The symbol E i denti fi es 出 at thevari abl eon theleft can have one of theval ues on the
ri ght side. InEquati on(8.2a), R stands for a real val ue. Thi s represents a standarduse
of 自己 symbol E 加 dis an effecti veway of expressi ngthei deathat thedi screteval ues
C佃 onl y be sel ected f roma group of val ues. Tbi s is a mi xed programmi ng ( MP)
probl em.
8.1.1 Pr obl emRel axat l on


E 出 is were a unconstrai nedprobl eminconti nuous vari abl es, thesol uti oncoul dbe
obtai nedby setti ngV f = 0 and sol vi ng白 e由 民 eequati ons for 白 eval u巴sof x.. X2' and
X3・Eq回 ti ons 8. 2b and 8.2c h鎚 to be overl ooked. Al tematel y, for a conti nuous
probl em, i nspecti onof theobj ecti vefuncti onyi el ds 白 efol l owi ngsol uti on:
x; =2; x; =2; x; =2; I =O (8.3)
For theori gi nal probl em伽 'dXl is defi ned, but not df/ dx2or df/ dx3' si nceX2佃 dX3
aredi screteval ues. Deri vati ves aredef med by taki ngthel i mi t of theratioof change
of obj ecti vefuncti onfto thechange intheval ueof thedesi gnvari abl e, as thechange
inthevari abl eapproaches zero. The val ueof 由巳 obj ecti vefuncti onfi s onl y defi ned
at sel ected combi nati on of X2・X3 in Equati on (8.2) and is noL defi ned el sewhere.
Smal l l i nfi ni tesi mal changes in the di screte desi gn vari abl es and therefore in 由 e
obj ecti ve functi on are not defi ned in Exampl e 8.1. Deri vati ves wi th respect to 恥
di scretevari abl es do not eXI st. Thi s concl usi onisof m句 01' si gni fi canceas it makes this
book' s previ ous body of wor k of I i mi ted use in出 epursui t of tbesol uti ontotheDP.
To recol l ect,白 enecess紅 yand suffi ci ent condi ti ons, whi ch drove theal gori thms, were
based on thegradi ents and thei r deri vati ves. Two areas intheprevi ous chapters escape
wi th a l i mi ted i mpact of 曲 is statement. τbeyar芭 L P and zero-order methods for
numeri cal sol uti on to unconstrai ned nonl i near opti mi zati on. Both of t hempl ay a
si gni fi cant roleindi screteopti mi zati onprobl ems. It is thereforepossi bl etoconcl ude
出 at sol uti on to Exampl e 8.1 ぉ establ i shed in Equati on (8.3) is i ncorrect (出 esi de
constrai nts areinvi ol ati on).
InDP thesol uti oninEquati on(8.3) represents thesol uti ontoa rel axati onprobl em.
Probl emrel axati oncan takeseveral forms. Mostl y it is appl i edto由 erel ax副 onor 白 e
weakeni ng of the cons町 ai n匂 or the obj ecti ve functi ons. There 眠 no expl i ci t
constrai nts inExampl e 8.1. I n出 is i nstance, therel axati onrefers totheremoval of the
restrictionof di screl eness of the vari abl es. Thi s is i denti fi edas conti nuous rel axati on.
The rel 出 edprobl emor rel axati onhas several advantages [3].
. If a constrai nt r l巴l axati onis i nfeasi bl e, sois theprobl eml model it rel axes.
・Constrai nt rel axati onexpands theset of feasi bl esol uti ons. The rel axedopti mal
val ue must i mprove or equal the best feasi bl e sol uti on 10 出 e ori gi nal
probl eml model.
• The opti mal val ueof any rel axedmodel provi des a l ower bound on山 eopti mal
sol uti onif il is a mi ni mi zati onprobl em. Si mi l arl y il establ i shes an upper bound
for maxi mi zati on probl ems.
• Many rel axati onmodel s produc巴 opti mal sol uti ons 山 at are easi l y rounded to
good feasi bl e sol uti ons of the ori gi nal probl eml model . 羽 lis appears to 創 刊
di screteopti mi z副 onfor engi neeri ngdesi gnprobl ems.
The val ues establ i shedinEquati on(8.3) area sol uti onto由巳 conti nuolI srel axati onof
Exampl e 8. 1. It was easy toobtai n出 is sol uti onwhi l e a techni que for thesol uti onto
白 e di screte un∞ 出 国 i ned op也 凶 zati on is still unknown一 白 e Kuhn-Tucker
condi ti ons areno l onger useful . 百 lesol uti oninEquati on(8.3) is not acceptabl e- xi
and xj arenot el ements of thepermi ssi bl ese1(8.2). The val ue of I = 0 wi l l be betl er
白 剖 恥 best di scretesol uti on- l ower bound 0" thesol uti on10批 ori gi nal probl em.
8. 1. 2 Dl screte C! ptl mal Sol ut l on
A standard approach 10 sol vi ng di scre1e opti mi zati on probl em, parti cul arl y in
engi neeri ng, is 10 use the conti nuous rel axati on of the mathemati cal model and
genemte a conti nuous sol uti on. For Exampl e 8.1this is gi ven inEquati on(8.3). Mor e
thanl i kel y出 edi scretevari abl es wi l l not bel ong tothep悶 defi neddi scretesets. These
vari abl es arethenchanged (or rounded) to山 enearest di screteval ues. For probl ems
wi th constrai nts, feasi bi l i ty is checked. The best feasi bl e sol uti on is l hen chosen.
Unl i ke conti nuous probl ems, there a問 no necessary and suffi ci ent condi ti ons to
ln 白 e case of Exampl e 8. 1 , Equati ons (8.2) and (8.3) provi de the rel evant
i nformati ontoestabl i shthesol uti on. For conveni ence, tJ l ey arereproduced here:
XJ ε R (8. 2a)
e [0.5 1.5 2.5 3.5] e X
ε [0.22 0. 75 1. 73 2. 24 2.78] E X
(8. 2c)
(8.3) x ; = 2; X; = 2; x ; = 2; I = 0
ηl e conti nuous vari abl eXJ is chosen 10have a val ueof 2.0. Fi gu児 8.1 i ndi cates four
sets of di sc附 eval ues for X2 and X3 around theconti nuous sol uti on. The least val ueof
theobj ecti vefuncti onat thesepoi nts wi l l be consi dered白 esol uti on. Eval uati ngthe
val ues of theobj ecti ve釦 ncti onat thosepoi nts:
x ; = 2; X; = 1.5; X; = 1.73; 1 =0.3758 (8.4a)
x ; = 2; x ; = 1. 5; X; = 2.24; 1=0. 8552
掴 l '










ー ー ー ー 守 一 一 ー ー ー ー ー ー ー 叶 ー ー ー ー ー ー ー 吋 ー 一 一 ー 一 一 一 一 一 一
ー ー ー ー 」 ー ー ー ー ー ー ー ー ー _ L _ _ ー ー 十 ー ー ー ー 」 ー ー ー ー ー ー ー ー ー ・

0.5 1.5 2 2.5
Fl gure8.1 Dl scretenel ghborhood01conti nuous solution.
  ~ =2; X; = 2ふ xi = 1. 73; ] = 0. 9158
x ~ = 2; x ; = 2.5; xi = 2.24; ] = 0.3752 (8.4d)
Fr omthis exerci se,出 e 同 8t val ueof 山 eobj ecti ve functi onis avai l abl einEquati on
(8. 4d) and woul d be regarded as the adj usted opt i mumsol uti on to the ori gi nal MP
probl em. 百 lis is al sothecommon practi ceinengi neeri ngdesi gnopti mi zati on.
The best sol uti on, however, is obtai nedat
x;=1. 7433; X;=1. 5; x;=1. 73; ] =0. 1782 (8.5)
The di fferencebetween Equati ons (8.5) and (8.4a) , onl y in白 evari abl eXI , suggests
伽 t addi ti onal conti nuous opti mi zati on needs to be perf ormed for each set of 曲 E
di screte val ues sel ected, conf i rmi ng the 回 rlier observati ons by Fl etcher. 百 us
opti mi zati onshoul d be easi er as theorder of themathemati cal model wi¥l be 問 duced
by thenumber of di scretevari abl es si ncethey have beenωsi gned numeri cal val ues.
Thi s si mpl e exampl e問 commendsat 血 every least a three-stepprocedwち for probl ems
白 紙 inc¥udedi scretevari abl es:
Step 1: A conlI nuous rel axati onthat i denti fi es several sets of di scretevari abl es for
further expl or副 on.
Step 2: For each such set of di screte vari abl e combi nati on, a conti nuous
opti mi zati onis perf ormed 10establ i sha newopt i mumva1ue of theconti nuous
vari abl es and thecorrespondi ngobj ecti ve functi on. If all of l hevari abl es are
di screte, thenonl y thefuncti onand constraI nts need tobe eva1uatedat eachof
thesel of vari abl es.
Step3: A si mpl e comp釘 i sonof theabove sol uti ons/va1ues inStep2 toi denti fy出 e
opt i mumsol uti onto山 edi screteprobl em.
Thi s unconstrai nedopti mi zati on exampl e in three vari abl es has demonstrated
that di screte opti mi zati on requi res a lot of wor k compared to conti nuous
opti mi zati on. Thi s wor k expands si gni fi cantl yif thenumber of vari abl es i ncreases
or if the mathemati cal model is enhanced through the i ncl usi on of constrai nts.
Another essenti al f l回 旬 re in the above expl orati on is 白 紙 no newmathemati ca1
condi ti ons were necessary for 巴 stabl i shment of the di screte sol uti on beyond a
si mpl e compari son of the obj ecl i ve functi on. The natureof di screte vari abl es and
di screl e functi ons precl udcs any sophi sti catcd mathemati cal condi ti ons
establ i shed by deri vati ves of the functi ons i nvol ved in tbe model. Trappi ng and
branchi ngbased on compari son of val ues are the mai nstay of di screte al gori thms.
There techni ques arecl assi fi edas heuri sti c methods. Thi s encourages uni que and
personal i mpl ementati ons of the search for di screte opti mi zati on that can h巴
tai l oredfor a cl ass of probl ems.
There arethreedi screteopti mi zati ontechni ques inthis secti on. The first one is
Exhausti ve Enumerati on. Thi s i nvol ves i denti fyi ng tbe sol uti on to 白 e
mathemati cal model for all possi bl ecombi nati ons of thedi scretevari abl es. Thi s
is suggesti ve of the zero-order numeri cal techni ques of Chapter 6. Those
met hods l ackcdsophi sti cati onas they i nvol ved onl y eval uati onof thefuncti ons
at a phenomenal number of poi nts. They were abl e to take advantage of th巴
pl enti ful computer resources avai l abl e today. I magi ne operati ng in the
peer-to-peer computi ng envi ronm巴 nt afforded by “MP3" or “Gnutel l a" and
sol vi ngtheprobl emusi ngall of thePCs intheworl d. The second met hod is the
Branch and Bound method. Thi s is based on parti al enumerati on where onl y part
of thecombi nati ons areexpl ored. The remai ni ng arepruned f romconsi derati on
because they wi11not determi ne thesol uti on. Thi s is currentl y themost popul ar
met hod for di screte opti mi zati on for engi neeri ng desi gn. The last met hod is
Dynami c programmi ng, an el egant approach to opti mi zati on probl ems, but
whi ch di d not gai n favor because it i nvol ved si gni fi cantl y l arger amount s of
computati on thancompeti ti ve met hods even for probl ems of reasonabl e size. It
is restri cted to probl ems thal requi re a sequenti al sel ecti on of the desi gn
vari abl es. Today, such resourc巴I i mi tati ons are di sappeari ng in the worl d of
powerf ul PCs and hence l hemet hod deservcs lobc revi si ted.

St andard Format - Dl scret e Optl m/ zatl on: The author is not aWllI右 of a
standardformat for 山 di screteopti mi zati onprobl em. It subsumcs 恥 format of l he
coπωpondi ng conti nuous rel axati onprobl cm. Inthis book Ihc fol1owi ng format fol'
themi xed opti mi zati onprobl emis used.
Mi ni mi ze f (X, Y ) , 医 ]n<; [Y]n
Subj cct to: h( X, Y) = [0]; [hJI
g( X, Y) 壬 [0]; [g]m
J; S Xj S.i;'; ;=1,2,.. . , n
Yi E Yd, ; [ Yd,]p, ; i = 1, 2,. . . , nd (8. 10)
X represents l he set of nc conti nuous vari abl es. Y rep陀 senl s the set of nd di scrcte
vari abl es. f is 山 eobj ecti vefuncti on. h is l heset of 1 equal i l yconstraI nts. g is l hesel
of m i nequal i ty constrai nts. Expressi on(8.9) rep民 :sents 出 esi decons回 ints on each
conti nuous vari abl e. Expressi on(8.10) expresses 出 esi deconstrai nts on thedi screte
vari abl es. Each di screl evari abl eYi must bel ongtoa preestabl i shedset of Pi di screl e
val ues Yt/I' If nd = 0,出巴 nil is a conti nuous opti mi zati onprobl em. If 11c = 0,出 cnit is
a di screteopti mi zati onprobl 巴 m. If botharep陀 sent,thenit is a Mi xed Probl em.
Cont l nuous Rel axatl on: The conti nuous relax甜 on of lhe mi xed opti mi zati on
probl cm(8.6ト (8.10) is identi ca110probl cm(8励ー (8.10) wi thY as a conti nuous vari abl e
邸t . 百 lCdi scre飽 ∞ns位凶 nl (8.10) is repl ac吋 by a conti nuous si de cons回 int for 出 e
di scretevari abl es of thef ormof (8.9). 官 邸 is not explicitlydevel opedsi nce(8.6) ー(8.9)
and a modi fi ed (8.10) is fairly representati ve of 出 econti nuous rel axedmathemati cal
model .
Reduced Model : The 問 ducedmodel is i mportant insubsequent di scussi ons. It is
山 emodel sol vedafter a part of theset of di scretevari abl es areset at some a¥l owabl e
discret巴 va1ues. Thi s removes l hosevari abl es f rom出 eprobl emas 山巴 ir val u巴shave
been set. The mathemati cal model is thendefi nedby theremai ni ng di scretevari abl es
ぉ we¥l ω 出 e ori gi nal conl i nuous vari abl es. If Z( n
) are 白 e remai ni ng di screte
vari abl es 白 紙 need10be sol ved,出 en
Mi ni mi ze l ( X, Z) , [X]", ; [Z] n, (8.11)
Su同ect 10: 五( X,Z) = [0]; [11]/ (8.12)
g( X, z) [0]; [i ]m (8.13)
J; S Xj S xj'; i = 1, 2, . . . , nc (8.14)
Zi e z.t,; [z.t):, (8.15)
Tobl e 8.10 Op“mal Vol ue of Coououous Vori abl ex. for Di screteCombl oauoo
。f Xl oodx3
X21x) 0. 2200 0. 7500 1. 7300 2.2判 。 2.78叩
0.5 0. 9067 1. 0833 1. 4100 1. 5800 1. 76∞
1. 5 1. 2柑 O 1. 4167 1. 7433 1. 9133 2. 0933
2.5 1. 5733 1. 7500 2. 0767 2. 2467 2. 4267
3.5 1. 9067 2. 0833 2.4100 2. 5800 2. 7600
Not e 伽 t in the above, 4, is not a new set . It coπ 田 ponds to a reduced Y d.. In
Exhausti ve Enumerati on, general l y it is 加 empt yset . AI I of thedi scretevari abl es 紅 巳
set tosome al l owabl e val ue.
8. 2. 1 Exhaust i ve Enumer at i on
Thi s is 白 esi mpl 巴st of 白 edi screteopti mi zati ontechni qu巴S. It eval uates an opt i mum
sol uti onfor all combi nati ons of the di screte vari abl es. The best sol uti on is obtai ned
by scanni ng thelist of feasi bl esol uti ons in出 eabove i nvesti gati onfor themi ni mum
val ue. The total number of eval uati ons is
/ 1. = IIp/ (8.16)
If ei th巴r nd or Pi (or both) is l arge,自己 n出 is represents a lot of work. lt al sorepresents
an exponenti al growt h inth巴 cal cul ati ons wi th由巳 number of di screte vari abl es. I na
mi xed op也 凶 zati onprobl em,出 is woul d i nvol ve nc conti nuous opt i mumsol uti onof
thereduced model. If themathemati cal model and its ∞mput er cal cul ati ons are回 sy
to i mpl ement , this is not a seri ous burden. I f出巴 mathemati cal model 陀 qui res
extensi ve cal cul ati onsー for ex創 npl e, 自 ni te el ement or finite di fference
cal cul ati ons- then some concemmay ari se. I nsuch cases,l i mi ti ngtheel ements of the
di scr芭 tevari abl es tosmal l er c1usters may rep陀 sent a useful approach.
Exampl e 8. 1: τbe exhausti veenumer司ti onof Exampl e 8.1isavai l abl einEx8_1. m.
Tabl e 8.1a shows出巴 opt i mumval ue of theconti nuous vari abl ex I for every a110wabl e
combi nati on of 山 di scretevari abl es X2and x ) ・Tabl e8.1b shows 山 correspondi ng
val ue of theobj ecti vefuncti on for each of the cases. Si nce 出 is is an unconstrai ned
probl em, no di scussi on of feasi bi l i ty is i nvol ved. There are 20 val ues and the best
sol uti onfor Ex創 npl e8.1 is
  ~ = l. i 433; X; = 1. 5; x ; = 1. 73; r = 0. 1782 (8.5)
There is some good news though. Fi rst, each of the 20 cases is essenti al l y a
s;ngle- lI ari abl eop山 凶 zati onprobl em. The number of desi gnvari abl es in出巴 ori gi nal
IFil es 10 bedowl l l oadedfromIheweb 削 除 a四 indicalcdbyboldfuce白 肌 S S町 iflype.
Tobl e 8. 1b Opumol Val ue of Obj ecuve Fuocuoo for Dl screl eCombi ooti on of Xz aod X3
0. 2200 0. 7500 1. 7300 2. 2400 2. 7800
0.5 1. 9107 1. 3542 2. 7915 4. 8060 7. 8840
1. 5 3. 3240 1. 3542 0. 1782 0. 8327 2.4707
2.5 8. 0707 4. 6875 0. 8982 0. 1927 0.3907
3.5 16. 1507 11. 3542 4. 9515 2. 8860 1. 6440
model (3) is reduced by 出 e number of di screte vari abl es (2). Model reducti on is
i nvol ved inenumerati on techni ques.
Al gori thm: E.幼 ' austl veEnumerat i on (A8. 1)
Step 1. ! = i nf, X = [0, 0, .. • , 0]
For every a1l owabl e combi nati on of (Y.. Y2,・・・・ Y"d) =辛 ( Yb)
Sol ve Opti mi zati on Probl em(8. 11)一(8. 15) (Sol uti onX・)
I f h( X¥Y
) = [0] and
I f g( X. , Y
}三 [0] and
If j ( X¥Y
) <!
Thenf ←ft. X ¥Yb}
Y ←Yb
End If
End If
Exampl e 8. 2iII ustrates 白巴 useof al gori thmA8. 1 wi th feasi bi l i ty requi rements.
Exmnpl e 8. 2 The Aerodesi gn Cl ub has i ni ti ated fund-rai si ng activiti ,回 by sel l i ng
pi zza duri ng l unch inthe student acti vi ty center. Over the y回 rs 出 ey have col l ected
也 ta to i denti fy a mathemati cal model that wi l l r叫 uce 白 eir cost whi ch di rectl y
tr剖 sla回 to hi gher profi ts. Onl y t wo types of pi zza wi1I sel l - pepperoni (x.) 佃 d
cheese (x21. The l ocal pi zza chai nwi1I del i ver a maxi mumof 26 pi zzぉ . Factori ngi nto
account wastage and the ti me avai l abl etomake thesal e出 ey arri veat thefol1owi ng
mathemati cal model :
Mi ni rni ze f(x.. X2) = (XI - 15)2+ (X2- 15)2
Subj ect to: 8. (Xh X2): XI + X2S 26
g2(X. ・x21: x. + 2. 5x2S 37
(8. 18a)
(8. I 8b)
XI ~ 0; x2 O. and i nteger (8.18c)
The graphi cal sol uti on for conti nuous rel axal i on is avai l abl e in Fi gure 8.2and the
anal yti cal Sol ul i on (not i ncl uded) i denti fi es that 12. 86 pepperoni pi zzas and 9. 65
cheese pi zzas will provi de for themi ni rnumcost of operati on.
Ex8_2. mis 山 e m-fi l e 出 at wi l l perf ormthe exhausti ve enumerati on of the
probl em. The di scretesol uti onis
X; = 12; xi = 10; r = 34; g2is acti ve
Thi s confi rms 伽 tnei ghbori ngdi screteval ues around a di screl esol uti onworks wel l .
Programmi ng exhausl i veenumerati on is strai ghtforwardand parti cul arl y easy wi th
MATLAB. Gi ven出巴 processi ngspeed, l argememor y avai l abi l i tyon白 edesktop. 国 sy
programmi ng throughsoftware likeMATI ..AB. e対 祖 国 ti veenumerati on is probabl y a
very good i deal oday. Whal recommends it even further is 白 紙 thesol uti onis a gl obal
ゆ ti mum. FlI rthermore. unl i ketheearl i er chapt巴 rs where th巴 concepts of deri vati ves.
matri ces. l i near i ndependence. and soon were essenti a1tounderstandand i mpl ement
thetechni ques of opti mi zati on, l ack of sucb knowl edge is no di sadvan包 getosol vi ng
a di screte opti mi zati on probl emusi ng exbausti ve enumerati on. Tbere is onl y one
essenti aI skill ー 凶 Is1ati ng 恥 mathemati cal model i nto programcode. Wi t h 出 E
experi ence provi dedin出 is book. this is defi ni tel y not an issue.
‘ ー 、 g ~   O
. .
‘ - ....
・ ¥ 6 i e - 7・   、 二
10 15 20 25 30 35 40
pepperonl pl zza
Fl gure8. 2 G阻 phical solutlonExampl e 8.2.
百 esol uti oninEquati on(8.19) canbe obtai nedwi thconti nuous rel 蹴 叫 onof 山 E
probl em. In this case the di screte sol uti on has to be recovered by adj usti ng the
conti nuous sol uti ontonei ghbori ngdi screteval ues.
Exhaust#ve Enumer at l on for Cont l nuous Optl ml zatl on Probl ems: Thi s title
must be quaI ified f urther- i n lieu of gradi ent-bas巴d numeri caI techni ques. The
avai l abi l i tyof seri ous desktop computi ng power. thepromi se of gl obal opti mi zati on.
and由巳 easeof i mpl ementati on suggesl a mul ti stageexhausti ve enumerati on strategy
for conti nuous opti mi 回 ti onprobl ems. Exhausti ve enumerati on after all is a search
strategy. si mi l ar 10 the ones used before. The search for sol uti on is aki n 10 白 e
zero-order mel hods empl oyed for unconstrai nedopti mi zati on(Chapter 6).
The hi gher stages in mul ti stage enumerati on i nvol ve enumerati ng on decreased
i ntervaI s centered around 白 e sol uti on f rom白巴 previ ous stage. The vaI ues of the
desi gn vari abl es used for enumerati on wi l l i ncorporate progressi vel y i mproved
tol erance. The strategy must takei nto account thepossi bi l i ty that theiniti aI i nterval
mi ghl overl ook gl obaI opti mal sol uti ons tllat do not appear attracti ve ina parti cul ar
stage. Hence. mor e than one mi ni mumis tracked in each stage. Each of these
mi ni mumcan be consi dered姐 enl l merati oncluster. Each such c1uster is mul ti staged
until a prescri bedtol eranceis achi eved. Inengi n田 ri ng. m阻 ufacturi ngtol eranceof the
order of 0. 0001 is achi evabl e today. Tol erances bel ow出 is l i mi t probabl y need tobe
j usti fi edunl ess 白 eareais mi croel ectroni cs. Fr oma programmi ng perspecti veit jllst
adds an extra l oop to the i mpl ementa1i on. 百 le final sol uti on obtai ned represents a
conti nuous sol uti onwi thi nthefinal l ol erancei mpl emented by thetechni que.
Such an exerci seis l efl 10thestudent . As wi thall i mpl ementati on themeri t of lhis
approach lies inthetractabili1y of themathemati cal model and theneed for a gl obal
opt i mumsol uti on. Si nce themet hod is l argel yheuri sti c, thereis no guarantee that the
sol uti onis actuaI lya gl obal opt i mum(of course assumi ng thefuncti ons i nvol vedare
not convex).
8. 2. 2 Br anch and Bound
Exhausti ve or compl ete enumerati on is possi bl e onl y for a l i mi ted set of vari abl es
because of theexponenti aI growth of effort 問 qui 民 dtoexami ne aI l possi bl esol uti ons.
Usi ng parti al (or i ncompl eteor selective) enumeral i on toarri veal thesol uti ontothe
di screl e opti mi zati on probl emwoul d provi de tlt巴 necessary advantage inhandl i ng
l arge probl ems. Partial enumerati on works by maki ng deci si ons about groups of
sol uti ons rather 出 叩 every si ngl 巴 one. The 8ranch and 80und ( 88) al gori thmis one
of thesuccess向 I al gori tl1ms touse sel ecti ve/partial enumerati on. lt does soby usi ng
rel axati onmodel s i nsteadof theori gi nal compl e1e di screl ema1hemati caI model . The
8 B al gori thmuses a treestructure10represent cqmpl eti ons of parti al sol uti ons. The
町 'ees汀 uctureuses nodes and edges/li nks 10represenl l hetrail of partial sol uti ons. Thi s
is call 吋 fatl romul g出 ep釘 tial sol uti on. The termi nol ogy is borrowed f romoperati on
rese邸 h literature [3] and is expl ai nedbel owwi 山 resp削 to Exampl e 8.1 whi ch is
reproduced herefor conveni ence:
Mi ni mi ze f(Xl ・X2・X3) = (XI - 2)2+ (XI - X2)2+ (XI - X3)2+ (X2- X))2(8.1)
XI E R (8.2a)
X2ε[0. 5 1. 5 2.5 3.5] E X
X3E [0.22 0.75 l. 73 2.24 2.78]ε X
Partl al Sol utl on: A sol uti oninwhi ch some discretedesi gnvari abl es arefi xedand
o山 ers arej ト'eeor undetenni ned. Vari ab1es that arenot fi xedarerepresentedby 出 e
symbo1 γ:" For exampl e,
x = u : 0.5,刀 (8.20)
rep問 sents theopti mi zati onprob1emwi 出 X2= 0.5and XI and X3tobedetermi ned. The
pa凶 al sol uti ons ar芭a1sotermedthenodes of thetree. 百 lefreevari abl es areusual l y
sol vedusi nga conti nuous rel 回 出 onof themodel .
Compl er / ons (0' PaTt刷 Sol utl on): Ench partial sol uti onor nod巴 cangivt: rist: lu
addi ti onnl partia1 sol uti ons/nodes di rectI y under it. Thi s is called bral l chi ng or
expansi oTl of lhe node. In this newset of nodes another one of 出巴 frec discrelc
vari abl cs is fi xedand白 eremai nder slill conti nuetobefree. For exampl e, under 出巴
node representedby thepartial sol uti oninEquati on(8.20), fiveaddi ti ona1nodes are
卯 ssible:
X = [ f , 0. 5 , 0.22]; X = [ f , 0.5, 0.75]; X = [ f , 0. 5 , 1. 73];
X = [ f , 0.5, 2.241; X = [ f , 0. 5 , 2.781
Thesc compl eti ons branchout andhcnce a treelikc structure. There is a hi erarchi cal
structuri ngof thenodes. Eachbranchrepresents 1eads toa tier inwhi ch some vari abl e
is hel dat its prescri beddiscretevalue.
Fat homi ng t he Parti al Sol utl on: Expandi ng thenode all theway till theend is
termedf atl wmi ng a node. Duri ngthe fathomi ngoperati on it is possi bl e10idenlify
thosepartial sol uti ons at a node that do not needtobe i nvesti gatedfurther. Onl y the
best sol uti onat 山 e node is branched or compl eted. Thosepart討ial soωlu 凶』此川t “io 叩 n凶sll 凶I no ωoωdeωs 山 a瓜t
will not b 恥ei nvesti gatedfurther arepruned 0 町 r tenni natedi 加n白 elI出芭 C.
Act l ve Partl al Sol uti on: A node thal has not beentenni natedor prunedand has
nol bcencompl eted.
EdgeslLl nks: Thes巴 釘 ' enode-to・nodeconnecti ons ina町 田 . They identifytheval ue
_ r ・   ・ _ ~ ..n_備 L ・司'....... : 例目. . . . . . . . J
Root: Th巴 BB al gori thmsearches a10ng a II 芭 C. lt starl S al the root where a11 the
部 23t九弘 :31?お 1 J 訪問: 叩 :;;:??J22::
preferredtoj umpi ng across nodes in白 esame凶 erarchy.
I ncumbent sof tJ Mont The i ncumbent solutionisthsbGSt feasiblesolutionMown SO
far wi tha11 discretevariables assumi ngdiscretevalues. When lh巴 BBsearchfinal1ystops,
白 Ehal i nCUREbBEEt sol uti on,i fomexi sts,isa zi obal opti mumsoi tHi on-hthefol l owi ng
thei ncumbentsol uti onis denotedbyr- - 13mi ncumbaudESi gn vmabl es眠 X...
Exampl e 8. 1- Usl ng Br anch and Bound: Here 血 e BB search is appl i edω
Exampl e 8. 1. BBEX8J J n cal cul ates al l of thEdataneeded for genEraI i ng血 巴 住 民 ・
TheBB町 田 itse1f is devel opedinFi gu民 s8.3and 8.4.
Start: Node O. Al l of 出巴 vari ab1esaref民 C. The sol uti ontotheconti nuous rel axati on
ofthe obj 田 tivefuncti on(sol vedas a conti nuous probl em) is identifiedat the
Nod巴s1-4: Thi s is th巴 first tier of theBB tre巴・ Thedi scretevari abl eX2 is assi gned
Eachof thgpenni sBi bl edi screteval ues at thenudcs inthisti er-Thi s ESi ndi cated
X=[ f,f,fJ
X* = [1.74,1. 5 , 1.73]
p =0.178
p* = 0.178l ncumbenl solutlon
XoI". = [1.74, 1. 5 , 1.73J
Fl aure8.3 BranchandboundExampl e8.1.

X" = [1.57, 2.5, 0.22]
P=8. 07
X" = [1.75,2. 5 , 0.751
P=4. 68 xe=I 107,zs,1.731 x' =I 2. 26,zaz. Ml
P = 0.898 P = 0.192
F1gure 8.4 Branchandb司 undExampl e8.1・-fathomi ngNode 3.
on the edges between the nodes in Fi gure 8.3. The remai ni ng v釘 i abl es 鉱 B
fr,田 I conti nuous. The opt i mumsol uti onat thevari ous nodes is di spl ayedon出 E
tree. The mi ni mumsol uti onis theone出 at will be f athomed at thenext step.
Here Nodes 2 and 3創 - e candi dates for expandi ng thebranches. Nodes 1 and 4
will not be f athomed (unl css their opt i mumval ueis better 出 anIhei ncumbent
sol uti on)一 山 ey will be termi nated or pruned 脳 血 e obj ecti ve functi on has a
hi gher val uc. ηl e conti nuous rel axati onensures a l ower bound on theopt i mum
val ueof theobj ecti veat thenode. Node 2 is takenup for f athomi ng based on
node orderi ng
Compl eti ons of Node 2: Thi s expands Node 2 by setti ng血 edi scretevari abl eX3 10
its permi ssi bl eset of di screteval ues. It creates branches Nodes 5-9. lt is al so
thesecond tier of thetree. The sol uti onis i ndi catedat 出 evari ous nodes. Inall
of themthe t wo di sc問 te vari abl es areset 10i ndi cal edval ues whi l e Xl is free.
whi ch is a conti nuous vari abl eby defi ni ti on. Si nce all of thedi screl evari abl es
bave di screte val ues, this node is al so fathomed. Node 7 i ndi cates the best
sol uti on. Si nce all of 出 e di sc民 te vari abl es arち assi gned di sc悶 le val ues,出 E
sol uti onat lhisnode is thecurrent i ncumbent sol uti on・Theremai ni ng nodes will
be pruncd. The currenl i ncumbent sol uti on (f. = 0. 1782) is hi gher 白 叩 由 巳
sol uti onal node 3 (f = 0.15). Thi s requi res Nod巴 310be fathomed.
Compl eti ons of Node 3: Fi gure8.4iIl ustrates thebranchi ngat Node 3. The process
is si mi l ar 10compl eti ons of Node 2 above. The best sol uti onis at Node 13wi th
a val ueof f = 0. 1927. This is l argerthantheCurrent i ncumbent sol uti on. Al I of
出 enodes can be l ermi nal ed.
No further nodes areavai l abl e for expansi on. The current i ncumbent sol uti onis
al so thegl obal opti mal solution. Thi s is thesol uti onto山 eopti mi zati onprobl em. In
出 is exampl e, 15 partial sol uti ons were n民 essary. Each partial sol uti on can be
consi dered an enumerati on. In exhausti ve enumerati on, 20 巴 numerati ons were
requi red for Exampl e 8.1. There is a nel gai n of 5 enumeral i ons. Th巴 BB search
redu田 sthenumber of total enumerati ons need巴d 10sol ve出 eprobl em.
Al gorl thm: Br anch and Bound伺 8.2): For even a smal l set of di screte
vari abl es actual l y drawi ng th巴 BBtreestrucl ureis most l i kel y a difficult enterpri se.
The 11' ・ee structure merel y i denl i fi es the search process. Exampl e 8.1 di d not have
constrai nts so feasibility was nol exami ned. Wi t h constrai nts present , choosi ng the
p釘 tial sol uti ontocompl ete based onl y on mi ni mumval ue of theobj ecti vefuncti onis
not a j usti fi abl e s回 tegy. In this case the besl feasi bl esol uti on is 山 e candi date for
compl eti on. Al so, an acti ve feasi bl epartial sol uti on 白 紙 has a better val ue 出 朗 自 己
current i ncumbent sol uti on is al so a candi date for compl eti on. The fol l owi ng
al gori thmcapωres 白巴 essenceof theBranch and Bound al gori thm.
StepO. Establ i shtheroot node wi thall di scretev訂 i abl es 合 問 .
lnitializesol uti oni ndex s toO.
. .
Sel current i ncumbent soI uti on tof -- = "", X- - = [∞], (mi ni rni zati on
probl em).
Stcp 1. If nny acti vefeasi bl eparua1sol uti ons remai n ( 白紙 is beuer thanthecurrent
i ncumbenl sol uti on) , sel ecl one as X<J) and go toStep2.
If none exi sts and thereexi sts an i ncumbent sol uti onil is opti ma1.
If none exi sts and the陀 is no i ncumbent sol uti onthemodel is i nfeasi bl e.
Step 2. Obtai n 白 e compl eti on of the partial sol uti on Xωusi ng a conti nuous
rel axati onof theori gi nal model.
Step3. If there釘 e no feasi bl ecompl eti ons of 白巴 partial sol uti onXω, termi nate
X<I), i ncrement s ←s + 1, and retumtoStep1.
Step4. If thebest feasi bl ecompl eti on of partial sol uti onX<I) cannot i mprove the
i ncumbent sol uti on, thenterrni nateXω, i ncrement s ←s + 1, and retumtoStep
Step 5. If白 e best feasi bl e compl eti on is better 由 加 the i ncumbent , update the
i ncumbent sol uti on
r. ←!<I), x.. ←x ω
termi nateX<s>, i ncrement s ←s + 1, and retumtoStep 1.
Exampl e 8. 2 The Branch and Bound al gori thmis appl i ed to Exampl e 8.2. The
S国 民 h is appl i ed through the BB住 民 . 百 le mathemati ca1 model for Exampl e 8.2,
whereXl ・andX2 arei ntegers, is
Mi ni mi ze !(x" x v = (Xl - 15)2+ (x2- 15)2 (8.17)
Subj ect 10: 81(X" X2) : X( + x2 :!>26
82(XI ・X2) : X( + 2.5x2 :!>37
XI   0; X2 2! 0
The sol uti onis
X;=12; X;=10; f ・=34; g2is acti ve (8.19)
ヱi z : ; 2 2 J 3 1 f 尽な e? L i t zは 522e i f z : : i : : z z t s z
fol l owi ngdevel opment
E [10 Il 12 13]
ろ E [9 10 11 ]2)
Usl ng Br anch and Bound: 百 eBB田 eis shown inFi gure8.5. The datafor 曲 e
町 田 町 C必 cuJat吋 through BBExι 2. mwhere theconti nuous 陀 l蹴 ation is sol v吋
anal yti caIl y usi ngthc K T conditions.τ 'he treeis brnnchedns fol l ows:
1'* =40
X. = [12. 86, 9.951
1'* :::33.13
1nf 伺 slble
Rgure 8.5 BranchandboundIr自白 for Example8.2.
NodeO: 百 出 is theroot node. Al l thevari abl es 釘 efr,田. 百 econti nuous rel axati on
of theprobl emis sol ved. The sol uti onis i ndi catedon由巳 tree. Currentl ythis is
theonl y avai l abl eactivepartial sol uti onand hence needs tobecompl eted.
Nodes ] - 4: 百 epartial sol uti onat Node 0 is compl etedusi ngvari abl eXI f romthe
reduced discrete set identified above. Thi s l eads to four branches and the
opti mal val ues ar芭 i ndi catedon the紅 白 . The best val ues are i ndi catedat 出 e
nodes. For 血 i sprobl em出巴. y arealsofeasible. There紅 enowfour activepartial
Node 3: The so]uti onat Node 3 is alsoa feasiblesol uti onof theoriginal discrete
opti mi zati onprobl emsi ncethedesi gnvari abl es aredi screl eand bel ongto白 e
permi ssi bl ediscreteset. Node 3 i sfatJ l Omed. 百 is provi des thefirst i ncumbent
Th巴 rearethreeavai l abl eacti vefeasiblepartial sol uti ons ( Nodes 1,2, and 4)
for compl eti on. Si ncethesol uti ons at 白 enodes areall feasible, onl y thebest
activefeasiblepartial sol uti onis pi ckedup for compl eti on- Node 4. Nodes 1
and2 canbetermi nated.
Compl eti on Node 4: Node 4 has a feasible sol uti on better 曲 朗 自 e current
i ncumbentsol uti on. l ti s compl etedby branchi ng10Nodes 5, 6, 7,8. Onl y Node
5 has a feasib]esolution. Tbe besl sol uti onal Node 5 isnot better thanthecurrent
i ncumbent solution. Nodes 5, 6, 7, 8 canbe termi nnted.
There areno more acti vepa凶 al sol uti ons and出 巴 加 ehas becnfathomed. The
current i ncumbent sol uti onis theopti mal solution.
百 lis compl etes the BB search appl i ed to Exampl e 8.2. The 陀 ducti on in
enumerati on, compared to由巴 exhausti veenumerati on, is difficult 10comment on for
two reasons.
1. The del i beratechoi cetorestrlct 白 ese1of discreteval ues used10branchthe紅 白
seemed arbitrary. Whi l e白 is was done for conveni ence, a seri ous justification
伺 n be advanced based on the conti nuous sol uti on at Node O. The reduced
discrete set is chosen to i ncl ude a l i mi 1ed nei ghborhood/ wi ndowof the
conti nuous 叩 1凶 on.
2. The exhausti veenumerati on, basedon disc陀 teval ues, was easi ertoeval uatefor
a purediscreteprobl em. Appl yi ng KT condi ti ons toobtai nthepossi b]ep紅 tial
sol uti ons to出 econti nuous rel axati on, and choosi ngtheacceptabl esol uti onis
an i nvol ved exercise. Usi ng numeri cal l echni ques Ii ke SQP 10 sol ve the
probl em, whi ch wil1 mosl Ii kel y be done in practi ce, has its own probl ems.
Dependence on good startingval ues wi 出 noguaranteeof convergence requi res
user i nl erventi on 10identify 血 eparticular solution. It woul d requir芭 alot of
effort 10automate thepro白 5Ssoas 10appl y山 eBranch and Bound al gori thm
10general nonl i near mi xed opti mi zati onprobl ems. Li near mi xed opti mi zati on
probl ems do not possess 血 is limitation, at least to thesame extent.
In山 is exampl e,白 eacti ve partial sol uti ons at the root and in山巴 first tier were all
feasible. What woul d be thestrategyif theroot nod巴 or other nodes have ani nfeasi bl e
acti ve partial sol uti on? In白 紙 case山 e88 searchprocess must compl ete all acti ve
partial sol uti ons and not onl y feasibleacti vepartial sol uti ons. That is a modi fi cati on
山 at canbe easilymade tothe8ranch and 80und Al gori thmabove.
Ot her Sear ch Techni ques: ηl e two search techni ques in山 is secti on, n創 nel y,
exhausti ve enumer冨 tion,and branch and bound, provi de an addi ti onal feature of
di scoveri nggl obal opt i mumsolutions.τ 'hey al sobel ongtothec1ass of methods used
to address gl obal opt i mum. τbe 回 sy availability of l arge and faster computi ng
resources has brought renewed emphasi s to gl obal l y opti mal desi gns. Si muUl ted
anneal i ng and geneti c al gori thms ar芭 among the l eadi ng candi dates for gl obal
opti mi 祖 tion. 百 ley 釘 ecoveredinthenext chapter and canal sobe characteri zedas
enumer混 i on飽 chni ques,like出巴 ones in白 is section. 80th of themcanand areused
for discreteopti mi zati on. They 釘 'enot di scussedinthis chapter. The appl i cati onof
most of thesetechni ques is still heuristic and oftendepends on experi encebasedon
numeri cal experi ments among c1ass巴s of probl ems. Some appl i cati ons are probl em
specificand requi reuser i nterventi onand l earni ng. Standardi mpl ementati ons in由 ese
methods. especi al l yfor nonl i near probl ems, arenol yet thenorm.
8.2.3 Dynami c Pr ogr amml ng
Di scretedynami c programmi ng (DDP) is anexci ti ngtechni que for handl i nga special
c¥ass of probl ems. Exampl es 8.1 叩 d8.2arenot direct members of this c¥ass. 則 chard
8el l man[4) was responsi bl efor 伽 ti n加 duci ng出 econcept andthealgori 山 m. It is an
op白 凶 zi ngproαdure basedon Bel l man・sprincipleof optimality. The principleis based
on a sequen田 of deci si ons on partial solutions. so曲 at when ccrtainvariables have been
determi ned. theremai ni ngvariables establishanopti mumcompl eti onof theprobl em[5).
Ano出 er expressi onof this戸 ncipleis ob包 i nedfromReference6. An opti mal sequenα
of deci si ons h脳 血 epropertythat at 出 ec町 rent timc. whatever thc initial stateand白 e
previ ous decisions. 山 E陀 mai 凶 ngdeci si ons must be anopti mal set wi thregardto血 estate
m 刷 ngfromthefirst d出 sion. DDP isabout a sequence of deci si ons (ofteninti me) and
is someti mes term吋 asequenti al ,たdsi onprobl em. Probl ems I i keExampl es 8.1叩 d8.2
do not directlyfit 白 iscl 俗 sifi 回 tionbut they回 nb山 田 sformedtofit 白 isr珂 ui rement (5).
Very often. DDP probl erns canbe descri bedas 卸 opti mal pa出 determi nati onprobl em.
Such P瓜 hprobl ems aresol vedusi ngdi rectedgraphs (di graphs). 百 leseprobl erns may
actual1y have no connecti ontophysi 回 J paths or distances. Li ke the88 tree,出巴 ycnabl e
better appreci ati onl appl i cati onof theal gori thmi c proωdure andarenot s汀ictly陀 quired.
DDPr珂 u田 sdefi 凶 tionof s. ωtes andstages. andonl y theformer is di scussedhe隠. 百 le
equati onsωed toes包 blishtheprinciple of opti mal i ty are回 ))edρ Ul ctiollal equati ons.
Exampl e 8.3will be usedtodefi ne血 enewterms.
Exampl e 8.3 An establ i sheduni versi tyis inl 巴 胞 stedindevel opi nga newCo))巴gefor
I nformati on Technol ogy duri ng the currenl ye釘 . Tabl e 8.2 ilI ustrates 恥 data
Tabl e8.2 Facul tyH1ri ngPollcyData for Exampl e 8.3
l tem Fall Wi nter Spri ng Summer
R明 uiredFaculty 5 10 8 2
Recrui 回 entCost 10 10 10 10
Uni t Facu1tyCost 2 2
Uni tRe凶 ni ngCost 2 2 2 2
necessary tocal cul al ethecost of recrui ti ngfacul tyfor 山 is enterpri sefor thefirst ye釘
of oper富 山 n. The col l egewill operateduri ngfour quarters (FaU. Wi nter. Sp由 g. 組 d
Summer) i ndi catedby出 ecol umns. Deci si ons areassoci atedwi ththebegi nni ngof the
quarters. The first rowi ndi cates thenewfacultyrequi redtoi mpl ement thecours白 血 at
will be offered. The secondrowis 山 erecrui tment cost for newfacul tyeachquarter.
whi ch is si gni fi cant because of 山 ecompeti ti ved巴mand for qual i fi edpeopl e in this
釘 回 . 百 le白 irdrowis 出 eunit cost of newfacul ty interms of thestandardinstitut巴
facul ty cost . Course speciali 回凶 on and availability makes this cost different in
different quarters. The last rowis 出巴 unit cosl for retai ni ngextrafacul tyhi redin出 E
earlier quarter (tomi ni mi ze cost over theyear). What shoul dbe thehi ri ngpol i cyfor
由巳 uni versi l ylomi ni mi ze cosl ?
百 lis exampl e is c1early il1ustrative of sequenti al deci si on maki ng. Deci si ons are
requ耐 dtobe made every qu副 er. For exampl e, it ispossi bl e10早 cruit all neededfaculty
duri ng出 efirst quarter andpay出 B 問 凶 ni ngcost. or it may cost less tohireduri ng回 ch
  , 血     quarters andhireagai nin白 ef o町 出 . 百 leprobl emwill be
sol vedusi ng加 opti maUshortest pathmethod even曲 oughthe隠 arenorou回 i nvol vedi n
出 . eprobl em. τbebest ways tounderstand白 eshortestp砥 hal gor抽 mistodrawa di rected
graph(di graph) betw田 nthenodesl states intheprobl em.
S旬 tes: States 問 present poi ntsl condi ti ons in DP where deci si ons have to be
consi dered. Thesearesometi mes 児 島 π'edtoas pol i cy deci si ons or just pol i ci ω. This
has a s叫 uenti aJ connecti on as current deci si ons cannot be ∞nsi de陀 d UD世 出 e
previ ous deci si ons have been made. Cl earl y in Exampl e 8. 3 出 町 民 present 白 e
begi nni ngof eachquarter. 明 lenumber of states is usual l yone more由 加 thenumber
of d巴ci si ons toi ncorporatetheeffect of thelast pol i cy/deci si on. Nodes inthedi graph
rep陀 sent states intheDP probl em. The di graphrepresents a connecti onbetween each
pair of states 出 at is permi ssi bl e. Thi s canbe made c1earer by stating伽 t if a node is
goi ng to be affected by a deci si on at any previ ous nod巴. then there is a direct
connecti on(I inelarc) between山 enode喧. The cost of theconnecti onis i ndi catedon由 E
connecti onitsel f. An arrowcanbe usedtoi ndi catethedi recti onof traverseal ong血 is
linelarc segment. TheseIi n巴s/ arcs 由 ereforerepresent decisiofls. Once thedi graphfor
theDP probl emis available. theopt i mumsol uti oncorrespofl ds to theshortest path
f r omIhebegi nni l l g10 theendi ng stateof the di graph [3).
Fl gure8.6 Di graphfor Exampl e 6.3.
Fi gur巴 8.6is 由 巴 di graphfor thehi ri ngpol i cy. There arefi vestates wi ththefirst
four representi ng出 equ紅 白 隠 wherethedeci si ons must be made. Node 1問 P陀 sents
theFall quarter and 50on. Al ong thearcs on山 edi graphis th巴 C05t of thedeci 5i onor
pol i cy 出 at wiJI ori gi nateat thecurrent node and meet a11 of therequi rements at 曲 e
l刻 印 r node. Inorder 10us巴 thei nformati oninthedi grapheffi巴cl i vel y,and 10devel op
Ihemathemati cal model , thequarters areassi gnedi ndi ces f rom1 to4. The fol l owi ng
abbrevi ati ons areused: requi redfacul l Y, RF; 陀 crui tment COSI , RC; uni t facul l y cost,
FC; uni l rel ai ni ngcost - hol di ngcosl , HC. The pol i cy deci si ons or cost between the
start node i and theend nodej is C(i,j). For exampl e. C( l , 4) rep問 sents 出 epol i cyめ
副 民 泊 theFal l q凶 rter,whi ch will meet 出 estaffi ngreq凶 rements for 出 eWi nl erand
Spri ngquarters a150thus payi ng曲 eappropri ateretai ni ngpena1ties:
C(l, 4) = RC( l) + [ RF(l) + RF( 2) + RF( 3) ) . FC(l )
+ ([RF(2) + RF(3)) 巾 HC( l) + [RF(3)] 吻 HC( 2) )
Ex8_3. mis them-fi l e白 at generates thedatafor thedi graphfor Exarnpl e 8.3 釦 d
i denti fi es 山 e50l uti onby sol vi ngthefuncl i onal eq凶 l i ons and backtracki ng山 epa白
to i denti fy the opti ma1 s巴quence. Note 出 at for differ芭 nt probl ems on1y the cost
cal cul ati onwiJI differ. The functi onal equal i ons and backtracki ng紅 巴 el ements of the
DDP met hod and shoul dbe thesame.
Functi onal Equatl on for Dynaml c Pr ogr amml ng: The functi ona1equati onis
theessenti al part of theDDP a1gori thmor method. It empl oys a recursi vepro印 du児
10i denti fythebesl sol uti onal eachstate. Lel Val ue (i) be th巴 best val ueof thedeci si on
al node i. l aki ngi nl oaccount a11 possi bl earcs/ paths thal l eadtonode i. Let Node (i)
be thenode白 紙 l edtothebest pol i cyat 白 ecurrent state. The recursi onrel ati ons for
IJ states canbe sel up as fol l ows:
Val ue (1) = 0
Val ue (k) = mi n (Val ue (の+C(/, k); 1 S 1 < k}, k = 2, . . . , 11+ 1
Forexampl 巴, inExampl e 8.3
VaLue (4) = mi n (Val ue (1) + C(l , 4); Val ue (2) + C(2, 4); Val ue (3) + C( 3, 4)}
Val ue (4) = mi n ([0+85]; [15+ 62]; [45+ 26]) = 71
Node (4) = 3 (whi ch establ i sh巴dthemi ni mumval ue)
Theseca1cul ati ons areembedded inEx8_3. m. The output f romtheprogramIS:
E xampl e 8. 3-Dynami c pr ogr ammi ng
Cost bet ween pai r s of nodes/ st at es C (i ,j )
o 15 45 85 99
o 0 30 62 74
o 0 0 26 34
o 0 0 0 12
0 0 0 0 0
odes - Val ues ( i )
opt i mal pat h val ues at v   r ~ o
o 15 45 71 79
Best pr evi ous node ヒo r each cur r ent node - Node( i )
0 1 2 3 3
The Val ue (4) is 71 alldNode (4) is 3. The opt i mumval ue for theprobl emis Val ue
(5) , whi ch is 叫1Ua11079.
Shortest Path: The shortest path is obtai nedby backtracki ng f romtlle end stat巴
凶 i ngtheNode (i )i nf onnati on- whi ch is ther owof dataabove.
At NodB5t hEbest nodEf r omwhi ch toreachNode 5i s Node3
o 2 3 3
AI Nod巴 3thebest node f romwhi ch toreachNode 3 is Node 2
o 2 3 3
At Node 2 thebest node f romwhi ch toreachNode 2 is Node 1 (this is 山 estartmg
o 1 2 3 3
Thi s sol uti onis i ndi catedby出 ethi cksoIi dlineinFi gure 8.6.
τ百ne 臼 ac伽 加 a叫I h 刷li 山耐 i討凶ri 巾t訂加n略I gp 卯ol i c 必cya 釦.cco ∞orl凶d伽 n昭gtωo白 iぬss叫ol u 凶tio ∞niおstωohi 甘i町r 隠e 叩 s抗tt白her問巴叫q伊u 凶山iI1凶 edf:批 a配c叩u凶山It旬y
for 血巴 FaJ l qu釘 t総er 泊 the FaJ l qu釘 t匂er. Do thesame for the Wi nter quarter. For the
Spri ngquarter, however, hi 陀 for boththeSpri ngand Summer quarters.
Al ternate Pat hs: There is an al ternate sol uti on for thc same opti mal val ue. The
output f romEx8_3. mi ncl udes 出 is i nformati onthough it was not i ncl udedabove. In
曲 eoutput inthecommand wi ndow, each r owcontai ns addi ti onal nodes出 at wi l l yi el d
same opti mal vaJ ue. 百 l eonl yen町 y is 1 inthi rdr owand second col urnn. Thi s suggests
that 鉱 山 e thi I1dnode, Node (3) can 山 o have the val ue of l - t hat is, Node 1 al so
provi des thesame vaJ ue toreachNode 3. This wi l l cause出 eNode (i) i nformati onto
change as

3 3
The shortest pa出 C朗 自 enbe i nterpretedas fol l ows:
At Node 5 thebest node f romwhi ch toreachNode 5 is Node 3

3 3
At Node 3 thebest node f romwhi ch toreachNode 3 is Node 1
o 1 3 3
I nFi gure 8.6this is i ndi catedby thethi ckdashed arcs. The hi ri ngpol i cy now woul d
be tohi refor FaU and Wi nter inFal1. Then hi refor Spri ngand Summer inSpri ng.
  Dynaml c Pr o, 吻 r amml ng伺 a勾
The fol1owi ng al gori thmis i denti fi cdi n-Refcrencc 3 for pa山 s that do not cycl e. The
ori gi nal 8el1man al gori thmincJ udes an i terati ve 陀 f i nement that al l ows for cycl i c
pa血 s. I n Exampl e 8.3 血 ere are no cycl i c pa白 s. Thi s al gori thmis i mpl emented in
Ex8_3. m.
Step 0; Number nodes insequenti al order. Arcs ar芭 defi nedfor j > i. Set thestart
node S opti mal val ue as
Val ue (s) =0
Step 1: Termi nate if all Val ue (k) have been cal cul ated. If p represents the last
unprocess巴dnode. then
Vallle(P) = mi n{ Va/ ue (i) + C (i. p): if arc (i , p) exi sts 1 S i < p}
Node (p) = thenumber of Node i that achi evedthemi ni mumabove
Return toStep 1
Thi s was a si mpl e exampl e of DDP. It is i mportant to note白 at sol uti oninfinal
均 l ue (り incJ udes on1y the best sol uti on at all states/nodes. Li ke other techni ques,
the偲 創 ' c several di fferent categori es of probl ems addressedby DDP. incJ udi ng speci al
di screteprobl ems liketheKnapsack probl ems. It is al so possi bl 巴 to bri ngExampl es
8.1 and 8.2under thef ramework of DDP. though toi mpart a sequenti 凶 naturl 巴 inthe
process of sel ecti ngdesi gnvari abl es inthis casemay i nvol vea to凶 transformati onof
白 eprobl em. Even thenit may not be effi ci ent tosol veit usi ngDDP. Inpracti ce, DDP
has al ways becn regardedas computati onal l y compl ex and prohi bi ti ve. It is sel dom
used for probl ems wi 出 a l arge number of states, or if the esti mati on of cost is
i ntensi ve. Another cri ti ci smof DDP is 白 at it ca而 es out a lot of redundant
computati on for conti nuous probl ems. Today. these are less of an i ssue 山 肌 in the
past. For stochasti c and di screteprobl ems DDP may sti11be an effecti vetool . 8el1man
has al soi ndi catedthat it c朗 beuscd tosol vea set of unstabl esystems of equati ons.
DDP al so compu也 s desi gn sensi ti vi ty as part of its sol uti on. This secti on was
pri m訂日 y meant to i ntroduceDDP as a useful tool for some categori es of probl ems.
Readers arc encouraged toconsul t 山 ereferences for mor e i nformati onon白 esu句ect .
Inthis secti on. t wo addi ti onal exampl es arepresented. The first one is an engi neeri ng
cx創 npl e f rommechani cal desi gn und illustrates how the di sc問 te versi on of a
four-vari abl eprobl emvery effecti vel y reduces toa si ngl e-vari abl eprobl em白 atcan
be easi l y ad也 国 sed by enumemti on. An unbel i evabl e reducti on in compl exi ty
yi el di ng gl obal opti mi zati on is obtai ned by el ementary cal cul ati ons. It defi ni tel y
i nvi tes scruti ny regaroi ng 出巴 E仔ort in sol vi ng conti nuous nonl i near opti mi zati on
probl ems. The second one is 出 eappl i cati onof thebmnch and bound searchtoa 0- 1
i ntcgcr pr ogmmmi ng probl em. Much of theappl i cati onof di screteopti mi zati onis in
也 e釘 'eaof deci si onma話 ng. and deci si ons likefor or agai nst. yes or no, arepossi bl c
usi nga 0- 1 l i near i nteger programmi ng model.
8. 3. 1 Exampl e 8. 牛 ー I BeamDesl gn
τhi s cxampl e is a structura1desi gnprobl em. Infacし it is another vari ati onl appl i cati on
of the I - bcamstructure seen 回 rlier. The conti nuous probl cmwas devel oped and
sol vedas part of a student proj ect . In山 is exampl e出 ebeamis used as a hori zontal
member ina fi xed“E" seri es gan町 cmne. The hoi st , wi th a capaci tyof W = 2000 Ib.
is at thecenter of thebe阻 1. The obj ecti vei sωdesi gn a mi ni mummass be創 nthat wi l l
not fai1under bendi ng. shcar, and spcci fi eddetl ecti on. The l engthL of the beamis
speci fi cd as 96 in. Steel was choscn for thc materi al of 出 e be創 n. Thc modul us of
elasticity of steel is E = 29 E+06 psi . The wei ght densi ty of s同 I is P = 0.284Ib/in

The yi cJ d s田 ng山 in健 闘 onis σ'y= 36 E+03 psi. The yi el dstreng出 inshear is τy=
21 E+03 psi .百 esp配 ifiedal l owabl e detl 民 ti ODis 0. 25in. A factor of safetyof 1.4on
thel oadis assumed. Fi gure 8.7iII us回 tes thecen回 c l oadi ngon thebeamas wel l as
thedesi gnvari abl es. (A11 cal cul ati ons areini nches and pounds. )

The Dl screteProbl em: The di screte probl emshares the same mathemati cal
model except the val ues for thedesi gn vari abl 巴s wi l l bel ong toa set of di screte
val ues. It is possi bl e to use巴ith紅 白 巴 compl eteenumerati on or the BB search to
sol ve 由巳 probl em. Bef ore proceedi ng further in this di recti on, it is useful to
recogni ze 白 紙 the reason for di screte opti mi zati on is to choose an off-the-shel f
I - beamwhi ch wi l l keep thecost and producti onti me down. Several mi l l s provi de
i nformati on on standardrol l i ngstock 曲 ey manufacture. Ref erence [7] is used to
col l ect data for this exampl e. lf a standard S- beamis needed,出 e reference lists
several beams wi th acωal val ues for thedesi gn vari abl es XI ・X2・X3・X4・Choosi ng
an I - beamf romthelist al soi denti fi es all of thedesi gnvari abl es. Thi s becomes a
si ngl e-vari abl e probl em, the vari abl e bei ng the parti cul ar i temf romthe list of
beams. I nthis case thereareno geometri cal constrai nts necessary and nei ther are
出 esi deconstrai nts. A compl ete enumerati on can be perf ormed on thecandi dal e
beams f rom出巴 stocklist, and thebest beamcan easi l ybe i denti fi ed. Compar e this
enumerati on to sol vi ngthe nonl i near mathemati cal model wi thscven functi onal
constrai nts. When such an opportuni ty ari ses, expl oi ti ngthedi scretc opti mi zati on
probl cmis hi ghl y rccommendcd. On theother hand, consi der a di fferent probl em
where a pl asti c I - beamwi1I be mol ded in the manuf acturi ng faci1i ty. Hcre the
conti nuous probl emneeds tobe sol vedfor themi ni mumcost.
Ex8_ 4. mwiIl perf orma compl ete enumerati on of this exampl e. It i ncl udes a
truncated stock list f romwhi ch thefmal sol uti on is obtai ned. The programmi ng is
strai ghtforward. The output intheCommand wi ndowis
X: =Oお 1" =48. 239 x; =0. 1563; X; =2. ∞; =5泊 28;
Opt i mumBeami s : W6X9
Mass of t he Beam 71. 543 (l b)
Dept h, Wi dt h, Web Thi ckness , Fl ange Thi ckness
5. 900 3. 940 0. 170 0. 215
Const r ai nt s: Def l ect i on, Nor mal
( <= 0)
- 2422048221. 600
Sol ut i on f ound
( i nches)
St r ess Shear St r ess ,
- 86086. 261
Note that 山 esol uti onis si gni fi cantl y hi gher 白 antheone obtai nedintheconti nuous
rel axati on, once agai n illus凶 ting 白 紙 the conti nuous rel axati on provi des a l ower
bound on thesol uti on.
(normal slress)

(defl 配 tion)

The mathemati cal model for this probl emcan be exp陀 ssedas:
Mi ni mi ze
Subj ect to:
Desi gnvariables: Exampl e8.4.
f. i . X) =pLA . ,
gl (X): WU - 0
g2(X): Wl x
- ,
g3(X): W仏 - 21X3τyS0
Fi gure8. 7
(shear s悦 5S)
Some geomctri c constrai ms torelatc thevari ous desi gnvari abl es ar巴
g4(X): xl - 3X2::;; 0
g5(X): 2x
- xl ::;; 0
g6(X): x3- 1.5x4 0
g7(X): 0. 5X4- x3 0
The si deconstrai nts on由 edesi gnvari abl es are
0. 25::;X
::; 1.25 0.125 ::;x
S 0. 75,
::;15, 3::;x
- 2468680. 570
The fol l owi ngrel ati ons def me出 ecross-secti onal a悶 a(Ac) , moment of inertiaの , and
first moment of areaabout thecen回 id(Qc):
Ac =2x
r 4
+X1X) - 2勾X
1 = _ (x
- X
- 2x3)3
= - - - - =- - - ' - - "' - - - - - - : :
12 12
Zerか 心 neI nteger Pr ogr amml ng
Many L P programs/model s have speci al features whi ch areusual l yexpl oi tedtbrough
speci al al gori thms. An i mportant cl ass of LP probl ems requi re出 evari abl es be bi nary,
that is,出巴 ycan have one of two val ues. Theseprobl ems can be set up so白 紙 val ues
Qc ザ  
Thc sol uti ontotheconti nuous rel axati onmodel is
Capi 組 1Budgeti ngExampl e Tabl e8.3
VaJ ue Year3 Year宮 Yearl Group Item







Mi crosystems
Model i ng
M脂 血 Is
Photoni cs


X" • • • ,X6ε[ Oor 1]










8 10 Budget
i = 1, 2,. . . ,m ι
u g




Mi ni mi ze
Subj ect to:
Note白 is is not a mi ni mi zati onprobl emas requi 問 dby thestandardf ormat .
A句 ori thms: 百 .erearesevera1al go抽 ms 伽. 1have beendevel opedfor sol vi ngZI P
model s. They all usei mpl i ci tJ selectiveenumerati on白 roughBranch andBound s朗 氏 h,
together wi th conti nuous relaxation. However, the bi nary nature of 出 e variables is
expl oi ted when compl eti ng the partial sol uti ons or 回 versi ng the trl田 . 百 le Bal as
Zero- One Addi ti veAl gori t hml Methodis one of 脈 問 commend吋 臨 加 i ques for 出 s
c1ass of probl ems [6]. The al gori thmfrom出巴 samereferenceis inc1udedbel ow.
Xj = 0 or 1, j = 1, 2, . . . ,n
Except for the bi nary sid巴 constrai nts on 出 e desi gn vari abl es,出 is represents 出 e
mathemati cal model for an LP probl em(the standard model [1叫凶 res equal i ty
cons回 intsonly). Most al gori thms assume出 巴 今 M芭 positive. If any cj is negati ve, then
1一巧 is sllbstitlltedfor xJ in出 eprobl em.
Several varieties of opti mi zati onJ deci si onprobl ems i nvol vebi naryvari abl es. They
inc1ude theknapsack (apure I LP wi tha si ngl emai n constrai nt), capital bl l dgeti ng(a
mul ti di mensi onal knapsack probl em) , assembl y line bal anci ng, matchi ng, set
coveri ngandfacilityl ocati on, and soon. Some exampl es aredetai l edbel ow.
for 白 e vari abl es are zero or one- - defi ni ng a Zero- One Integer Programmi ng
Probl em(ZIP). The general f ormof a ZI P is
f(X): :2,明
Exampl e 0 ' a Knapsack Probl em: Mi ni mi ze 血 e nl l mber of penni es (X1) ,
ni ckel s (x21, di mes (X3) , and quarters (X4) toprovi decorrect change for b cents:
X1+x2 +x3 +X4
X1 + 5X2+ I Ox3+ 25x4 = b
創 'emtegers
Mi ni mi ze
Sl l bj ectω:
X" X2,X3,X4
In白 eabove probl em, theunknown vari abl es arenot bi nary vari abl es but 釘 ei ntegers
none白 eless.
A匂 orl肋 m: Bal as Zero- One Addi ti ve 匂     , (A8. 4)
The enumerati onis 佐 ackedby two vectors U aitdX. Both紅 巳 of l engthn. lf a partial
sol uti onconsi sts of 巧 1,ろも・. . ,巧 sassi gnedinorder, theval ues of X correspondi ng
to白 eassi gnedvari abl 巴sareOor 1. τ'he remai ni ngv紅 iabl 回 紅 e合 'eeand have thevallle
-1. The componenωof U are
λ I i 江f 巧k =J a 叩ndi出tωsc∞om 岬 pμIlem巴叩n 飢Itけhasno 悦 t b巴民c叩nc∞o叩 nω ere
II 句l l k=イ古叫:k ぅ = 1 or 0 and its 叩 pJ em側 h 叫 eenconsi d町 d
o I i f k>s
Step 1. If b :2: 0, opti mal sol 凶 onis X = [0]. Otherwi s巴 U= [0] , X = [ ー J] .f. = 0 0,
X. = X (i ncumbent sol uti on)
Step2. Calclllate
Exampl e 0 ' Capi tal Budgeti ng: 百 le Col l ege of Engi neeri ng is consi deri ng
devel opi ng a uni que Ph. D. prograrn in Mi crosystems in the next thrl田 years. Thi s
expansi oncallsfor si gni fi cant i nvestment inresources and personnel. Si x i ndependent
focus areas have been i denti fi edfor consi derati on, a11of whi ch c佃 not be devel oped.
Tabl e 8.3pI 1CSents theyearl ycost associ atedwi thcreati ngthevari ous grOllpSover the
由 民 ey回 rs. 百 lelast linein出 etablerepresents theavai l abl ebl l dget (inmi l l i ons of
dollars) per year allotted to this devel opment. 百 le last col umn is 出 e vallle (in
$100,000) thegroupis expectedtogenerateinthenext 5 years.
The capital budgeti ngprobl emllses sixbi naryvari abl es ( Xh . • • , X6) , one for each
i temin出 etable. The ZI P probl emcanbe fo口nul atedas
J is theset of assi gnedvari abl es
y,=bj - : 2 , a内,
20X1 + 3X2 + 30X3 + 5X4 + I Oxs + 5X6
3X1 + 2x2 +x3 + 2x4 + 5xs S 10
Maxi mi ze
Subj ect to:
y = mi n Yi' i = 1, 2, • . . , m
I =L
I f y注 O加 d 1<1' then I =J , and X・=X Go toStep6
0出 erwi seconti nue
Step3. Createa subset T of 合- eevari abl es巧 defi nedas
T=i j : f +今 <1,aij < 0 for i such曲 at y/ <O)
1f T = o (empty set) thenGo toStep6, EI seconti nue
Step4. Infeasi bi l i tytest:
If i suchthat Yi < 0 and
Yi - L mi n(O, aij) < 0
j GT
Go toStep6, EI seconti nue
Step5. BaI as branchi ngtest: For each freevari abl e巧 createtheset 叫
叫 =(i: Yj - aij < 0)
If aI I sets Mj areempt y Go toStep6. Otherwi se CaI cul atefor 回 仙 台 ee
vari abl e巧
巧 =ヱ 仇 -ai j )
i eM}
where竹 =0 i ftheset 叫 is empty. Add tothecurrent parti aI sol uti onthe
V紅 i abl eXj 由 at maxi mi zes 巧・ GotoStep2
Step 6. Modi 勿 U by setti ng the si gn of theri gh加 ost posi ti ve component . AlI
el ements totheri ght areO. Ret umtoStep2
If there are no posi ti ve el ements in U,血 e enumerati on is compl ete. The
・ ; _ _ . dI
opti maI sol uti onis inthei ncumbent vector X-. The obj ecti veis in.r. Jf.r is伺 ,
thereis no feasi bl esol uti on.
Bal as. mis 出 em-fi l e白 紙 i mpl ements theBaI as al gori thm. It was transl atedf roma
PascaI programavai l abl einReference 6. It is a programand prompts theuser for the
probl emi nf ormati on- n, m,A, b, c. 百 letest probl emgi veninReference 6 is i ncl uded
bel ow.
Exampk8. 5
Mi ni mi ze f (X): l Ox. + 14x2+ 2l x3+42x
l ~
~ ~

/ 信

Subj ectto: - 8x 1 - l l X2- 9X3- 18x4S - 12
- 2x. - 2x
一 7x3- 14x4S- 14
-9x. - 6x2 - 3X3 - 6x4 s- l O
X.. X2' X3. X4E [0 1)

i 川


Theoutput 合 omBal as. m:
Sol ut i on exi st s

ーーー・・ーーーー- - - ーー- - - -
The var i ab1es ar e:
1 0 0 1
The obj ect i ve f unct i on 52. 00
const r ai nt s - Va1ue and Ri ght hand si de
- 26 - 12
- 16 - 14
- 15 - 10
This corresponds tothesol uti oni denti fi edinReference 6.
Compl et e Enumer at l on: The Bal as aI gori thmperforms ~   ~ enumerati on. The
compl ete or exhausti veenumerati on (n.) for n bi nary vari abl es is gi venby therel ati on
Typi cal l y,erIteEprisedeci si onmaki ng i nvol ves several hundred vari abl es and l i near
model s. On the0出 erh叩 d,engi n回 ri ngprobl ems characteri zedby a nonl i near model
may have about 20 bi nary vari abl es mqui ri ng ne=l ,O48,576enumBEaI i ons- Thi s i s-no
bi ' deal wi th desktop resources today. KEep in mi nd 白 紙 compl 巴te enumeratl on
avoi ds any rel axati onsol uti on(purel ydi screteprobl ems onl y).

CaI l Si mpl ex. m: 明 白 is a functi onm-fi l ethat c叩 beused tosol veL P programs. It
c m beusdt oobt ai n sol uti ons toLP ml axati onmodel s duri ngtheBB SEarch.I t is
basedon theaI gori thmavai l abl einReference 6. It is transl atedf roma Pascal program.
百 lemathemati caI model is
Mi ni mi ze c
Subj ect to: [A] 匹 ] = [b]
x ~ o
Sl ack vari abl es are incJ uded in出 ecoeffi ci ent matri x [A]. Greater 出 制 or equal to
constr凶 nts c佃 be set up wi th negati ve sl ack vari abl es as the programwill
automati cal l y execute a dual -phase si mpl ex method. Another al temati veis to出 巴 白 e
L P programs provi ded by MATLAB. The functi on stal ement for usi ng Cal l
Sl mpl ex. mis
f unct i on [ xsol , f] Cal l Si mpl ex( A,B,C,M,N)
To use thefuncti on. A. B. C. M. N have tobe defi ned. M is 血巴 numbcr of constrai nts
and N is thenumber of desi gnv釘 iables.
1. Fletcher. R., Pracl i cal Mel hods ol Op' i mi zal i on, Wi l ey, NewYork, 1987.
2. Papa加 nbros,P. Y. , and Wi l de, J. D. , Pゆ l Ciples 01 p l i ~ 聞   Desi gn- Model i ng and
ConψUlalion. Cambri dge Uni versi tyPress, New York, 1988.
3. Rardi n, R. L. , Opl i mi l al i oninOperal i ons Researcl J , Prcnl i ce-Hal l , Engl ewood CI 江 fs,
NJ ,1998.
4. Bel l man, R. , Dynami c Progranun加 g,Pri ncetonUni versiLy Press, Pri nceton, NJ , 1957.
5. Whi te, D. J. Dynami c Programmi ng, Hol den- Day, SanFranci sco, 1969.
6. Sysl o, M. M. , Deo, N. , and Kowal i k, J. S. , Dl screl e Opl i mi zati on Al gorl l hms,
町宮 nti ce-Hal l ,Engl ewood Cliffs, NJ . 1983.
7. RyersonSl ockSleel Lisl. J osephT. Ryerson, Inc. , 1995.
8.1 Transl ate Exampl e 1.1 i nto a di screte opti mi zati on probl emand sol ve by
exhausti ve enumerati on.
8.2 Transl ate Exampl e 1. 2 泊 10 a di screte opti mi zati on probl emand sol ve by
exhausti ve enumerati on.
8.3 Transl ate Exampl e 1. 3 i nto a di screte opti rni zati on probl emand sol ve by
exhausti ve enumerati on.
8. 4 I mpl ement a mul ti stage exhausti ve enumerati on for thc conti nuous
opti mi 却 ti onof Exampl e 1.1.
8.5 Transl a飽 Exampl e1.1i ntoa di screteopti rni zati onprobl emand sol veby branch
叩 dbound method.
8. 6 Transl ateExampl e 1.2i ntoa di screteopti mi 四 ti onprobl emand sol veby branch
and bound method.
8.7 Transl ateExampl e 1.3i ntoa di screteopti mi zati onprobl emand sol veby branch
and bound method.
8.8 I denti fyappropri atest∞k avai l abi l i tyfor thefl agpol eprobl eminChapter 4 and
sol vethecorrespondi ng di screteopti mi zati onprobl em.
8. 9 Sol ve Exampl e 1.3usi ng山巴 stocki nformati onavai l abl einEx8_ 4. m.
8. 10 You aretheowner of an automobi l e deal ershi pfor Brand X cars. Set up a DP
for monthl y i nventory control based on cars expected 10be sold. hi ghorderi ng
costs. uni t costs for thecar. and i nventory costs for excess cars. Sol ve it usi ng
thecode inEx8_3. m.
8. 11 Sol ve thc fol l owi ngusi ngBal as al gori thm- usi ng compl etc cnumerati on
Mi ni rni ze f ( X) = XI + X2 + X3
Subj ect ω 3 x I + 4X2 + 5x3ミ 8
+ 5x2+ 3X3 3
X¥. X2. X3 ε[ 0 1]
8. 12 Progr盲目 1an exhausti veenumerati on techni quc for zero- one I P usi ngMATLAB.
Sol ve Probl em8.10.
8. 13 Is it possi bl etosol vethecapi tal budgcti ng ex創 npl e? If so, what is thesol uti on
usi ngcompl ete enumerati on?
8. 14 Sol ve capi tal budgeti ng exampl e usi ngBal as program.
8. 15 Create an exampl e whi ch i nvol ves maki ng bi nary choi ccs regardi ng your
parti ci pati on incourse and extracurri cul ar activitics 10 maxi mi ze your grades.
Sol ve it .
Thi s chapter i ntroduces two of themost popul ar techni ques for fi ndi ngthebest val ue
for the0均ecti vefuncti on, al so referredtoas thegI obal opti mumsolution. They are
Si mul ated Anneal i ng (SA) and the Geneti c Al gori thm(GA). The names are not
acci denta1si ncethei deas behi ndthemarederi ved合 o mnatural processes. SA is based
on cool i ng of metal s to ob組 in defi ned crysta1line structures based on mi ni mum
potenti a1energy. GA is basedon thecombi nati onand recombi nati onof 白 egenes in
thebi ol ogi ca1 systeml eadi ng to i mproved DNA sequences or speci es selection. In
actual appl i cati on, they ar官 more like a conti nuati on of Chapter 8 (Di screte
Opti mi zati on) , because these methods can be used for conti nuous and di screte
categori es of theopti mi zati onprobl ems. They sharean i mportant characteri sti c wi th
other opti mi zati on techni ques in 白 紙 出 ey are pri m釘 ily search techni ques-they
identify 白 e opt i mumby searchi ng the desi gn space for the solution. Unl i ke 出 e
gradi ent-basedmethods, thesetechni ques arel argel yheuristic, genera11yi nvol vel arge
細 ounts of comput姐 on,and i nvol vesome statistical rea50ni ng. Gl oba1opti mi zati on
is bei ngpursuedseri ousl ytoday at回 cti nga si gni fi cant amount of newresearchto白 e
areaof appl i edopti mi zati onafter a l onghiatus. A si gni fi cant drawback tomany of
thesetechni quesi s 白 鉱 山 eyrequi reempi ri cal tuni ngbasedon theclass of probl ems
bei ng i nvesti gated. There is also no 回 sy way to detecmi ne techni quel
probl em-sensi ti veparヨmeters toi mpl ement 岨 automati copti mi zati ontechni que.
The gl obal opti mumcannot reall y be characteri zedωbei ngdifferent f romthel ocal
opti mum. The latter is referencedwhen onl y a part of thedesi gnspace(as op抑 sedto
出 ecompl e総 space) is vi ewed inthenei ghborhoodof a parti cul ar desi gnpoi nt . The
standard Kuhn- Tucker condi ti ons, establ i shed and used in earlier chapters (for
conti nuous probl ems), wi l l i denti fy 出 e loca1 opt i mumonl y. Thi s is a1so more
emphati c innumeri ca1i nvesti gati ons of conti nuous probl ems where theal gori thms are
expectedtomove towardtheop“mumval ues cl osetowhere th巴:yarestarted(i nitial
, Whi leit makes sensetol ook for thegl oba1opti mum,也巴飽 i包5no method to
白LEn6ιff v 伽 J char叫 帥
c側 in ふuo1u 凶1路sc. ω伽' 01 仰 m附v. 附Fぽe町x; Em いroωo 油bl ems蹴 g伊u凶 aran附 帥 附 a刊岬E 以l伽 州Is仙o ω仙山lu蜘 I仙ti 山is削 d雌i江 iffi 伍fic 伽刷帥
c ω叫州
tωoa 釘 閉 剛 E卸 t路削t 由伽 ha “tSl帥 a d 尚 蹴 re叩pres鈴en附tωson 均l yami加nis印叫削 c α凶 u 山』
r叫 : みP帥 削i捌 o叩 np 仰ωb 凶l em 附 s 紅 御 巴 日n削ot n 附巴郎ces 郎 s叩 州ly c ∞伽 o叩 n附 v巴 此 Therefore, i 加nt 蜘h恥el蜘it 加t匂臨 e釘 r 酬 眠
bothoft血he鈴semethods areexp巴 ct飽, ed ω sol uti ons c10se ω 白 巴 opti mum
sol uti on, rather thegl oba1sol uti on血 emsel ves-' I11i sisnotto suggestthat i nvesti gati on
of thegl ob山 ol uuOni s a wa蜘 of 幽 eif thepossibilityof one叩 notb巴 倒 油 lished
; 立かお古 222221;ti::;fr;;LP222ぷ JZ;21
0pti IIl i zaGon techni ques are si mpl e to i mpl ement whi l e mquut ng exhausti VE
cal cul ati ons.Thi s canpotenudl y soak up all of 山 eidleti m巴 onthepersona1desktop
computi ng resource, le叫 ngit 回 medi atea町 acti veness.
23e techni ques and their ascussI on axeVEry basI Cfol l owi ng血 E白 emeespoused
in円 apter 8. 百 e問 der is encouraged to由 it j ournal publ i cati ons ω moni tor 血臨

ma 酬 aU 帥1IよEzzE odf 血伽帥白悶巴 慨帥帥 h加ni榊q中仰脚 u田附巴鈴s. l n 白刷帥i泊s c 伽h加apt飽e肌 白耐伽伽 r悶附 sはt s 叩o佃 叩 m鵬 e i …s騎 捌 叫 叩 s叩 削 sues 凶附巴郎s r 時e 曙例判 g阿凶凶副叫 i 加均 n昭g g 脚10凶
刑 同 阻 u ω帥o叩 n紅 蹴' ep附 n】t叫 f刷011 伽1
τ'h巴 fol l owi ngdcvcl opmentor-gl obal opti mi zadonisbas do n unconstrai nedconti nuous
op自 国 zation. Sol vi ngcons回 i nEdpmbl emsdGemotappeartohaveevol vedt oacommon
tec加 向 田 血 d is not illus回 .ted in this book. Note 也 at cons回 i ned probl ems can be
convertedto anuncons佐官 i nedprobl emusi ng a penal ty formul ati on(Iαl ap町 7)・明 le
probl emcanbe defi nedas
Mi ni mi ze f [X); [X)n
Subj ectto:   三
百 刷 sprobl emwere削 抑 制 (onl y one捌 加 um) ,白 eop削 捌 oncoul d切手 ly be
回 tabl i shedby any of thea卯i f Opriat巴 techni quesusedintheb∞k si ncetheloca1mi ni mum
is also也 , egl obal mi ni mum. To也 , vesti ga胞 thegl obal mi n田 町 nit is thereforeexpected
伽 巾 帥 附 ul dbem叩 yl o仙 ninirr剛 15,several of 伽 nhavi ngd臨 rent va1附お r批
objective. 宮 崎 least among themisthen出 egl obal solution. Loca1mi ni mums, in血 eωse
of conti nuous probl ems, are血 ose白 紙 sa出 fy血 eKT condi ti ons,佃 dwill be numeri cal l y
determi 耐 j f 白estarting仰 s w俗 in血 enei 酔 bo出 oodof 胞 p副 叫 mi ni mum. Thi s
definitionmay be appl i edtodiscre島eop自 国 zati onprobl ems that us巴 conti nuousrelaxation
ωo b阻 in 曲 esoMUon- For di scmteprobl ems usi ngcompl e飽 enumerati on,the gl obal
ooti mi muon Shoul dhdi scoverEd automati cal l y.FIT1eEEfGEe-the di scussi on applies to
di scretep帥 l ems thatu鴎 onl yi ncompl eteenumerati on.
9.1.1 Gl obal Mi ni mum
The gl obal rni ni mumfor the exists if ぬ efuncti onis conti nuous on a
nonempty feasibleset S出 at is cl osedand bounded [1]・
百 leset S defi nes a regi onof 出 edesi gnspace白 紙 is feasi bl e. Any poi nt inS is a
candi date desi gn poi nt X. In the n-di mensi onal Cartesi an spa田 , the poi nt X
co町 田 ponds toa vector of l cngthIIXII. The boundari es of theregi onarepart of thes巴t
S (c1ose, の.官 le l ength of the v民 tor X shoul d be a finite number (bounded). 百 1巴
defi ni ti onfor thegl obal mi ni mumX. can be statedω
f(X. ) 壬f(X); X e S (9.3)
The condi ti ons for theexi stenceof thegl obal mi ni mumas gi venabove areattri buted
toWei erstrass. Whi l e白 egl obal mi ni mumis guar叩 t田 dif thecondi ti ons aresati sfi cd,
出 ey do not negate出 eexi stenceof thegl obal sol uti onif condi ti ons arenot satisfied.
Consi deri ng thebroad range佃 d typeof opti mi zati onprobl ems, thecondi ti ons ar芭
fairly l i mi tedin白 血 use釦 l ness. The condi ti ons for 血巴 i田 al m泊 回 umar e白 esame
as constrai nt (9.3) except theset S is l i mi tedtoa smal l regi onaround X. .
Convexl t y: If an opti mi zati onprobl emcan be shown ωbe convex, then山 el ocal
mi ni mumwill al sobe a gl obal mi ni mum. A convex opti mi zati onprobl emrequi r, ωt hc
obj ccti ve functi onωbe convex. If constrai nts arepresent, thenal1of 由巳 i nequal i ty
constrai nts must be convex, whi l e the叫 ual i tyconstrai nts must be linear.
ConvexSet : 百l I s usual l y rel ates to 由巳 desi gn vector (or desi gn poi nts) of 出 e
probl em. Consi der t wo desi gnpoi nts X. and X
As before, theset S defi nes a desi gn
regi on and by i mpl i cati onbo白 X. and X
bel ong toS. The linej oi ni ngX. and X
contai ns a whol e seri es of newdesi gnpoi nts. If al1of thesepoi nts al sobcl ong toS,
山 enS is a convex set. A cI osed set S r l叫 ui 陀 s that thepoi nts X
姐 dX
can al sobe on
theboundary. I f , however. thelinej oi ni ngX. and X
contai ns some poi nts that do not
bel ongtoS. 出 enS is not a convex set . For a two-di mensi onal desi gnprobl em. convex
叩 dnonconvex sets arc i l l ustratedinFi gure 9. 1.
Convex Funct / ons: A convex functi on is defi ned onl y for a convex set. Once
agai nlet S be a convex set f romwhi ch t wo desi gnpoi nts X. and X
arc sel ccted.
Let f be thefuncti onwhose convexi ty is tobe cstabl i shed. Consi der any poi nt X"
on a l i nej oi ni ngX. 佃 dX
. The functi onfi s convex ifj {XJ is less 白 畑 山 va1ueof
白 ecorrespondi ng poi nt on白巴 l i nej oi ni ngftX. ) andf t Xu. l nn di mensi ons 山 is is
not easy tose巴 but a one-di mensi onal representati onis prωented inFi gure 9.2. 百 le
condi ti onfor convexi ty in山 efi gureis
f(x, , ) S r (9.4)
A mathemati cal condi ti onof convexi ty is that th巴 Hessi anma凶 x (matri x of second
deri vati vcs) of thef uncti onf musl bc posi ti vesemi dcfi ni tc at al 1poi nts intheset S.
Note thal this corresponds to 出 e K T condi ti ons for unconstrai ned mi ni mi zuti on
守 す で   寝 間 ヨ
一「 寸d

s NonconvexSe.
I X1 十

円 gure9.1 Convexandnonconvex se!.
4 支


probl ems. Fr oma practi cal vi ewpoi nt. theei genval ues of 自己 functi onmust be greater
山 anor equal tozeroat al 1poi nts inS.
lt is i mportant toreal i ze白 紙 agl obal opt i mummay sti1l exi st even if theconvexi l y
condi ti ons arenot met. l nreal opti mi zati onprobl ems. it may be difficult 10estubl i sh
convexi ty. Inpracti ce. convexi ty checks arerarel yperformed. An assumpti on is made
白 紙 agl obal opt i mumexi sts for theprobl em. 明 白 is a subj ecti veel ement 白 血 forced
by experi ence. i nsi ght. and intuition. Thi s is al sooftenj usti fi edby白 epossi bi l i ty白 紙
a better sol uti onthan山 ecurrent one can be found.
X] X
Fl gure9.2 Convex functl ロn.
9. 1. 2 Nat ure of the Sol utl on
It is assumed出 創 出 egl obal opti mumsol uti onwiIl be obtai nednumeri 心a11y. A good
understandi ngof thenatureof thesol uti onis possi bl ewi threferenc巴 ωaconti nuo叫
desi gnexampl e. Exωnsi on todi 鈎 re胞 probl ems is possi bl eby consi deri ngthat any
conti nuous sol uti onco即 ' esponds toa speci fi ccombi nati onof theset of disc陀 teval ues
for thedesi gnvari abl es. Fi gure9.3is an i1I us町 出 。 nof a fi gurewi thscveral mi ni ma
and maxi ma wi th a s位 onggl obal mi ni mumat x, = 4, X2 = 4. 百 le fig町 e,avai l abl e
血 roughf l g9_3. m,1 is theplot of 白 efuncti on(aI soExampl e 9.1)
子、 sin(0.1+ぜ ( X. - 4) l +( X2- 4)勺 (9.5)
f(x. ・Xz) =- 20
..     ~ , 0.1 +ぜか 1- 4) z+( x
- 4)Z
Fi gure9.4iIl ustrates 白 econtour pl ot The contour pl ot canbe bet旬 r app児 ci atedin
col orusi ng出直 col orbar for interpretation. Runni ng them- fiI e will createbothfigures.
Fr omrelation(9.5) it is cI ear theripples wiIl subsi di zeasXI and x2 move much farthcr
away f romwhere the gl obal mi ni mumoccurs, be氾 aus巴 of th陪 i nfl ucnce of the
denomi nator. The numerator isconstrai ncdtoIi c between -1 and + 1 mul ti pl i edby thc
Several ml nl m目 白 ndmaxl ma wlthglobal solullon


- 20
-10 -10
Fi gure9.3 Surfaceplot of Exampl 由 9.1.
Ipiles10be downl oaded針。 mthewebsi!ear芭 indicnllldbyboldfacesnnsserif typc.


Several ml ni ma and maxl ma withglobal solution- contour plot

~ ~ ~ ~ 0 2 4 6
x, val ues
Fl gure9.4 Conl our plot of Exampl e9.1.
scal i ngfactor of 20. Inthelimit theri ppl es shoul dstart tobecome a plane. This can
be confi rmcdby executi ngf l g9_3. mwi thdifferent ranges for 山 variables.
Fr omFi gure9.3it is easy tospot 白巴 gl obaI sol uti onby observati on. Surroundi ng
出 e gl obaI mi ni mumare ri ngs of 10四 1extrち ma. Numeri cal l y it is a chaI l engc 10
negoti ate a1l of the bumps and dips. A si mpl e way to charactc白 .e numeri cal
techni Ques for a mi ni mi zati onprobl emis that 白 eyattempt tofinda di tch(or val l ey)
tofaI l into. If this is胸 凶 as anunconstrai ncdmi ni mumprobl em, theK1・ condi ti ons
wi l l l eadthesol uti ontobe回 ppcd泊 四 yoneof 也 el ocaI mi ni mumri ngs, dependi ng
on thel ocati onof 出 cstartingpoi nt for thciterations. lf theiterations 創 ' CstartedcI ose
toX( = 4, x2 = 4,出 enit is more白 叩 I i kel y出 c gl obal opt i mumwil1be di scovered.
Stari ngatxl = - 5, X2 = - 5, usi ngstandardnumeri cal t田 hni ques will probabl y not l ead
to出巴 gl obaI mi ni mum.
Thi s is a si mpl e illustration but not Gt hemareMani tBl ocal mi ni mums and
m蹴 加 ums 制 daEi ngl egl obal mmm. I n 則 的 i gnprobl ems wi 白 morc 白 叩 2
V紅 iables,it is not readi l yapparent howmany locaI mi ni mums exist , and if 出 egl obaI
mi ni mumis rcally that distinct, even if it is postul at吋 to cxist. What about the
compl cte enumerati ontechni que? Thcreisno way thcgl obaI mi ni mumcanescapethc
siftingdue to血 is techni que. There is a1sono techni que山 at is si mpl er 出 佃 compl ete
enumerati on. 百 le probl emis Ihe ti me to finishsearchi ng Ihe infinitepoi nts of Ihe
desi gnspacetodi scover Ihe solution.
The sol uti on to gl obal opti mummust Iherefore lie betweenstandard numeri cal
techni ques andcompl ete enumerati on. The former will not findIhe sol uti onunl ess it
is fortunate. The 1atter carmot find Ihe sol uti on in ti me to be useful. Gl obal
op也 凶 zati on飽 chni ques tendtofavor thelatter approachbecauseof 出 esi mpl i ci tyof
i mp1ementati on. To be凶 eful l hesol uti onmust be di scoveredina reasonabl etime.
9.1.3 El ement s of a Numerl cal Technl que
百 leprob1ems facedby a numeri cal techni quesearchi ngfor Ihe gl obal mi ni mumhave
b田 ntouchedon in血 eprevi ous section. Inmost standardnumeri cal techni ques 白 E
loca1rni ni mumserves as a strongma伊 et or a beaconfor Ihe solution. Inmost cases,
desi gnchanges arenot acceptedif Ihereis no i mprovement inthesolution. If standard
numcri cal techni ques have tobe adaptedtol ook for gl obal rni ni mum, IhenIhey must
encourage sol uti ons 出 ati n曲 eshort runmay i ncreaseIhe obj ecti veval ueso白 創 出 ey
canfindanol her val l eyat whose bottomis 山 egl obal mi ni mum. Theymust achi eve
Ihis wi l hout sacrificing由巳 signific叩 tpotency of Ihe local rni ni mum. becausea local
mi ni mumsol uti onis better thanno sol uti onat a11. Fi gure9.5illustrates Iheseideas
usi nga singlevariable.
InFi gu民 9. 5出 eiterations sta氏 at poi nt A. Ina standardn山 neri 回 1techni que. a
S田 氏 hdir田 tion is foundwhi ch leads to poi nt B. At B. if ∞nventi onal numeri cal
techni ques were used, a searchdirectionl eadi ngtopoi nt C will be accepted, whi l ea
searchdirectionl eadi ngtopoi nt D will be strongl ydi scouraged. But, if poi nt D was
somehowreachabl e,出 enit cou1dmove on toE and出 enF and conti nuetoIhe g1oba1
F1gure9. 6 Strategyfor global opti mumsearch.


mi ni mum. The mai n i deainfi ndi ngthegl obal mi ni mumis toencouragesolutions山 at
will escape the al 回 ction of Ihe local mi ni mum. Thi s means al poi nt E it is quite
possi bl e白 紙 白 esearchdi recti onmay 1eadtopoi nt G. 11is 血 enr l叫 ui red出 at a 1arge
number of iterations bepermi ttedso白 at Ihe va11eyho1di ngIhe g1oba1mi ni mumwi1l
berevisited. From白 isbrief i1l ustrationIhe characteristics for a techni que凶 edtofind
a gl obal mi ni mumcanbesummari zed as:
• Short-termi ncreases intheobj ecti vefuncti onmust be encouraged.
• Tradi ti onal gradi ent-basedtechni ques will not be particu1ar1y useful b巴cause
出 eyarebasedon searchdir芭 ctions 曲 at d民 間 槌 eIhe obj ecti vefuncti onat Ihe
current itcration. Theywi11not pcrmi t incr回 路 of Ihe obj ecti vefuncti onvalue.
・If descent-based search directions are not part of Ihe techni que, 山 叩 any
direction白 紙 decre酪 es 出 eobj ecti veshouI d be acceptedtopromote attraction
by Ihe 10cal rni ni mum.
• As 批 l ocati onof Ihe gl obal mi ni mumis not known, searchmust be permi tted
over a 1argcregi onwi l hno bias, and for a 1argenumber of iterations.
• Si nce acceptabl e di recti ons can decrease or i ncrease 山 obj 削 ve functi on
val ue, a hcuristicdetermi nati onof 山 S同 町 hdi recti on(or desi gnchange) must
bep訂 t ofthe旬 chni que.
• These features resemb1e Ihe zero-order techni que for unconstrai ned
op白 瓜 zati on 叩 d Ihercfore will rcqui re a large number of iterations to be
effective. 百 lelargenumber of iterations wi1l forceIhe techni que10be si mp1e.
・ T he sol uti on can onl y be concl uded based on Ihe number of iterations, as
necessary condi ti ons (availab1e for conti nuous probl ems onl y) cannot
di sti ngui shbetweenloca1and gl obal opti mum.
Wi l hout a doubt, Ihe best techni que is compl ete or exbausti ve enumerati on. It is
Ihereforenecessarytoassume that compl ete enumerati onis Il Ot all opti ol l infurther
di scussi on.
The prob1emdescri pti on, illus岡 山 n,anddi sc凶 sionhave dealt wi l huncons町 ai ned
rni ni mi zati on. Cons佐 aints canbe i ncl udedby addressi ngIhe above characteristics to
appl y tofeasiblesol uti ons only. It is 1eft to Ihe reader tocont巴 mpl ate曲 eability to
he凶 stica1lygeneratea largenumber of feasibledesi gnchanges for consi derati on.
Thissecti oni ntroduces two popul ar mel hods for gl obal optirnization. Bol h arederi ved
fromnaturaI physi cal process自 由 at hel p identify opti mumstates. 百 le first one,
Si mul ated Anneal i ng (SA). refers ω a physi cal process 曲 at establishes differcnt
crysta1line structures, each of whi ch can be associ ated wi 出 different mi ni mum
potential energy. Thi s is attemptedby reheati ngandcool i ngof 白巴 materi al s/meta1s
whi l econtrol l i ng由巳 raleof cool i ng. 百 lesecond, Ihe Geneti c Al gori l hm(GA) , works
throughmutal i onof candi datesol uti ons unti l l hebest sol uti oncanbe esl abl i shed, a
process 曲 at can be rel ated to natur百1 sol uti on or bi ol ogi cal evol uti on. 百 lese
techni ques areiII ustratedwi threspect tounconstrai nedprobl ems inthis secti on.
9.2.1 Sl mul at ed AnneaJl ng ( SA)
Thc first appl i cati onusi ngSA for opti mi zati onwas presentcdby Ki rkpatri ck et al.
[ 21. The exampl es covered probl ems f romel ectroni c systems,i n puti cul ar,
component pl acement 命 的 en by theneed for mi ni mumwi ri ngconnecti ons. The
al gori thmitself was based on the use of statistical mechani cs, demonstratcd by
Metropol i s et al. [3] , toestabl i shthermal cqui l i bri umina col1ccti onof atoms. The
termitself is mor e reprcsentati veof theprocess of cool i ngmatcri al s, parti cul arl y
metal s, after rai si ng thei r temperature to achi eve a defi ni tc crystal l i ne statc. As
menti oned earl i er, the extensi on to opti mi zati onis mai nl y hcuri sti c. Thc si mpl e
i deabehi nd S A can bEi l l ustratedthrough theI nodi aed basi c al gori thmof S A due
toBochavesky ct al. (4).
Al gorl thm: Basl c Sl mul at ed Anneal l ng (A9. 1)
Stcp 1. Choosc starti ngdesi gn"0. Cal cul atefo= J ("o)
( Necd stoppi ngcritcria)
Step2. Choos巴 arandompoi nt on thcsurfaceof a uni t n-dIl III : Rsi onal hypcrsphere
(it is a sphere inn-di mensi ons) toestabl i sha searchdi recti onS
Step3. Usi ng a stepsi zeα , cal cul ate
ft =f ( X
+ω) ; Af =ん一 λ
Step4. If Af S 0, thenp = 1
El se p = e-fJt.f
Step5. A randomnumber r (0S r < 1) is generated.
If r S p, thenthestepis acceptedand thedesi gnvector is up由 ted
回 se,no ch佃 geis made to出 edesi gn
Go toSt ep2
If i mpl emented,出巴 al gori thmhas to executed for a reasonabl y l arge number of
i terati ons for thesol uti onto命 的 toa gl obal mi ni mum. The si mpl i ci tyof theal gori thm
is very appeal i ng. A ca 蹴r陀efi “削110 伽bsc仰 rva叫6ωo 叩 n wi l l 白 伽 α 副 蹴 卯 巾
pa 創 ra 混加 a
町 叩 叩 berl叫el a飢t吋 tωot山heBol t zma聞nnp 戸ro 曲ba油.bi 出litydiおst凶rib 加utωi ぬo叩 n. Ins “ta凶tistical
mechani cs it isexp陀 ssedas -kl T; T is consi dered出 canneal i ngtemperatu問, andk出 e
Bol tzmann constant. It di ctates thecondi ti onal probab出 守 山 at a worse sol uti oncan
beac四 pted. For theal gori thmtobe effecti ve, it is陀 commendedthat p be intherange
0.5S p S 0.9. Informal S A termi nol ogy, repr芭 sents 出 eann叫 i ngtemperature. The
foIl owi ng i deas intheal gori thmarc apparent :

• The al gori 血 mis heuristic.
• The al gori thmis suggesti vc of a bi asedrandomwal k.
• Descent of thefuncti onis di rectl yaccommodal ed.
• Dirl回 ti ons 出 at i ncrease出 eval ueof thefuncti onaresometi mωpermi tted. Thi s
is i mportant toescape l ocal opti mum. Thi s is thepotenti al todi scover th巴 gl obal
opt i mumthrough出 eal gori thm.
This basi c al gori thmcan defmi tel y be i mproved. Duri ng theinitial iterations it is
necessary toencourage sol uti ons toescape出巴 l ocal opti mum. After a reasonabl y l arge
number of itcrations, the opportuni ty to cscapc l ocal opt i mum( wi ndow of
opportuni ty) must be conti nuousl y rcduced sothat thc solutioll docs not wandcr all
over thepl ace, and canccl thei mprovements tothesol uti on出 at have al ready taken
p1ace. Thi s can be i mp1emen凶 by schedul i ng changes 泊 白 e p鉱 ameter ぉ 肱
i terati ons i ncrease. InS A tenni nol ogy,出 is is referredtoぉ 白 eanneal i ngschedul e.
百 l esc modi fi cati ons do not inany way decreasetheprobl em!user dependence of the
al gori thm. Thi s probl emdependenc巴 is usual l yunattracti vefor automati c or standard
i mpl ementati on of the SA mcthod. Rcsearchers are engagcd in constructi on of
al gori thms 山 at do not depend on uscr-speci fi edal gori thmcontro1paramete路 [5J.
The SA al gori thmfor conti nuous vari abl es i mpl cmented inthebook i ncl udes many
of 自己 basi c i deas f romthe ori gi nal al gori thm. There are many addi ti onal changes
no凶 he陀 . First 恥 negati ve si gn in the prob油 ility 旬 r mis fol dedinto itscl f. A
si 町 leprocedure is i ncl udedfor obtai ni ngtheval ue   It isbased on theinitial val uc
of "0 and fo. is ca1cul a凶 ωyi cl da val ueof p = 0.7for a Af 叫 ual to50%of 恥
ini 叫 ん 百 eal gori thmfavors a negati veval ueof by d凶 i伊 . Si ncc theprocess for
establ i shi nganinitial i1I us回 tedabove can detcrmi nc a po副 veval ue, theal gori thm
corrects this occurrencc. It is latcr illustrated through numeri cal experi ments that a
smal 1magni tude of thc pararncter al l ows thesol ut i onωwander all over thep1acc,
whi l e a 1argeval ue provi des a s釘' Ong 10cal at位 acti on. Insteadof theSA stepbei ng
i ncorporatedwhen出巴 obj ecti vei ncreases, it is tri ggeredwhen出巴 obj ecti veis 1ikely
toi ncreasc bascd on胸 sl ope. In白 ea1gori thmbel ow, theco口紅 01parameter is kcpt
constant at its initial val ue. The cal cul ati on of αi s borrowed f romthe tradi ti onal
one-di mensi onal stepsi zecal cul ati onusi ngtheGol den Secti onmethod (Chapter 5) if
theobj ecti veis like1y todecreぉ e. If thesl opeof thc obj ecti veis posi ti veor zero, a
fi xedαi s chosen. Thi s val ueis a maxi mumof one of theprevi ous
Al gorl thm: Si mul at ed Anneal l ng伺 9. 勾
Step 1. Choose "0.ω叫蹴J ( "o) , p = 0.7, = -l n(p)/0.5xJ ("o)
百刷 < l ,s = 1, If s > O,s =ー l xs
Step2. Choose a randompoi nt on thesurfaceof a uni t n-di mensi onal hypersphe陀
toestabl i sha searchdi recti onS
Step3. Cal cul ateVj {Xo);
If Vj{X
) < 0, thencomput e I・Dα.
AX=α ・S; 文 =Xo+AX; X
o← 笠 ← i + 1
If Vj {X
) 0,
4f =f (為 +尚 一 f (丸) ; a =m出 (α ・ , 1)
p =
A r andomnumber r; (0S r < 1) is gencrated.
If r S p,白 en山 estepis accep包 dand出 edesi gnvector is updated
AX=aS; 文 =Xo+d. X; 為 ←文 i ← i + 1
El se conti nue i ←i + 1
00 toSt ep2
The al gori thmis avai l abl c in Si mul at edAnneal l ng. m. It is cal l ed through
Cal l _SA. m. The al gori thmis appl i edtoExampl e 9.1, whi ch was usedfor iIl ustrati on
inSccti on9.1. To executc theal gori thmthefol l owi ngm-fi l es (i ntroducedinChapter
6) are ncccssary: Upper Bound_nVar, Gol dSect i on_nVar . m. Fol l owi ng the
practi ceof drawi ng contours for tW(ト vari abl eprobl crns, t wo files arenecessary for
descri bi ng 出 .eexampl e, the second pri m釘 ily to qui ckl y drawa contour pl ot usi ng
ma佐 ix cal cul ati ons. In this case the files are Exampl e9_1. m and
Exampl e9_1 pl ot. m. The latter filename must be set inSi mul at edAnneal i ng. m.
The cal l i ngstatement and vari abl es arecxpl ai nedinSl mul at edAnneal l ng. m. Th巴 y
can be readby typi ng
>> hel p Si mul at edAnneal i ng
inthecommand wi ndow(thecurrent di rectorymust contai nthefiles)
The隠 ader is strongl y encouraged torun theexampl e 00 hi s computer. It woul d be
a si ngul ar coi nci dence if any of the 問 sults presented here are dupl i cated. Runni ng
numeri cal experi ments issi mpl y a mattcr of calIi ng theS A al gori thmand taki nga l ook
at therωul ts (especi al l y 出 e pl ot). The desi gnchanges are i l l ustrated usi ngcol ors:
red- represen出 g d民 間 asi ng sl opes- and bl ue- f or posi ti ve sl opes 伽 t l ead to
desi gn changes. The 巴 n回 pment in the l ocal mi ni mumand the es回 pear芭c1earl y
iIl ustratedon thecontour plots. The contour pl ots紅 efilledcol or pl ots toi denti fyareas
on the fi gure where the functi onhas a l ocal mi ni mumor maxi mum. Darker col ors
陀 present mi ni mums and l i ghter col ors themaxi mum. The exampl es were run usual l y
for 1000 iterations. It is concei vabl e白 紙 they coul dhave becn stoppedearlier. On a
200・ Hz PII l aptop it requi red less 由 阻 a mi nute for each run. A l arge number of
i terati ons is useful toi ndi cateif thesol uti onescapes thegl obal mi ni mumonce havi ng
obtai nedit . Whi leit was not cvi dcnt in出 is cx創 npl e,it is l i kel y probl emdcpendenl
si nce in 山 is cxampl e I he gl obal mi ni mumwas si gni fi cantl y l arger 山 an thc l ocal

" 宮

mi ni murns. Several features of theS A al gori thmareiIl ustratedusi ngsome numeri cal
Exampl e 9.1 Thi s is thefuncti onusedinSecl i on9.1. The gl obal mi ni mumis at X・=
[44] wi th an 0附田 tive val uej " = -19. 6683. In出 e fi gur民 thecontours suggest a
stronggl obal mi ni mumenci rcl edby al tematel ocal maxi mums and mi ni mums. For
most of theruns 血 estarl i ngval uewas far f romthegl obal mi ni mum.
、 si n(0.1+吋 (XI - 4) :L + (-'1. - 4)勺
f(x. , x
) = - 20
l'   - 0.1+ぜ (x
- 4) :L + (x
- 4) l
Mi ni mi ze





Fi ve numeri cal experi ments arepresentedbel ow. It shoul dbe emph鎚 i zedthat 由 ese
runs cannot be dupl i cated. Overal l , theal gori thmworked very well. For exactl y出巴
same starti ng val ues and number of i terati ons (cal l i ng Cal l _SA. mwi thout any
changes- about 40也 市 s) 出 egl obal mi ni mumwas f ound 95%of theti me.
Run 1: Starti ngdesi gn= [-5, -5]; s =ー 2. 7567; Number of i terati ons = 1000
Useful number of i tcrati ons ( when dcsi gnchanges occurred) = 383
Fi nal desi gn= [3. 9992, 3. 9997]; Fi nal Obj ecti ve =ー 19. 6683
Fi gure 9.6i1l ustrates thetraceof thedesi gnchang田 . It escapes thestarti ngrel ati ve
mi ni mumfairlyeasi l ytoget 回 .ppedby a di fferent l ocal mi ni mum. It is abl etoescape
to the gl obal mi ni mumwhere it is strongl y attrac包 d to it. τbe 位置 .ces in the l ocal
mi ni mumappear toprefer tostay inthel ocal mi ni mumsuggesti ngthesol uti ons are
not bei ngencouraged 10escape.
Run 2: Starti ngdesi gn= [4,4]; ト ー 0. 0362; Number of i terati ons = 1000
Useful number of i terati ons ( when desi gnchang巴soccurred) = 991
Fi nal desi gn= [100. 72, 127.75]; Fi nal Obj ecti ve = 0.128
Fi gure 9.7iII u柑 ates thc traceof 山 esearchfor thegl obal mi ni mum. Most of thepoi nts
arenot inthepl ot as can be seenf romthesol uti on. Thi s sol uti onwas startedat the
gl obal mi ni mum. The searchprocess app回 rs towander all over thepl aceand finally
off 山 巴 map. There is no patteminth巴 l i mi tedpoi nts secnon the fi gure, except 血 e
suggesti on 曲 at the search is encouraged t oωcape. This can be ticd to the l ow
magni tude of s.
Run 3: Starti ngdesi gn= [4,4]; s = -1; Number of i terati ons = 10∞
Useful number of i terati ons ( when desi gnchanges occurred) = 553
Fi nal desi gn= [3. 8832, 4. 3709]; Fi nal Obj ecti ve =ー 19.173
Fi gure 9.8ilI ustra陥 thetraceof thedesi gnsearch. Inthis runtheval ueof s was sel
to-1. Once agai nthestarti ngval ue of thedesi gnwas thegl obal mi ni mumitself. Fr om
thefig町 eit is c1ear th鉱 山 esearchis not al l owed toescape出 egl obal mi ni mumthat
easily. The choi ce of thecontrol parameter was arbi 佐 町 y,and several ex創 npl es have
tobe studi ed10seeif 山 is val ueis probl emi ndependent . The val ueof I is typi cal l y a

- 倒

I 手


o 2
咽 lu鈎
- 2
- 4 - 6 - 8
10 B s 4 o 2
4 ー自 - 6
Si mul atedanneal i ng斗 owp.
Di scussi on on the Numeri cal Experi ments: τ'he fol l owi ng observati ons are
basedon theexperi enceof appl yi ngSA (A9.2) toExampl e 9. 1. They correspondto
出 ereportedexperi encewi thSA in山 egeneral literatu陀 .
Sl mul atedanneaJ ingExampl e9.1ー-convergedsolution.
good choi cefor opti mi zati ona1gori 出 ms,p紅 白 ularlyfor a weU-sca1edprobl em. What
is clear f romtheruns sofar is that a negati veva1ueof s of theorder of 1 i mpl ements
a loca1at凶 ctor.
Fl gure9.6
• Anneal i ngschedul e(changi ngs) is 叩 i mportant consi derati oninSA. Starti ng
wi tha l owva1ue出 econtrol p制 me胞 r s must bei ncreascd-not toofast or it
will get 回 pped泊 aloca1mi ni mum.
• Anneal i ng schedul 巴 is devel oped through tri a1 and error, insight 白 紙 is not
obvi ous, and basedon probabl ebehavi or of thes田 氏 h.
• Lengthof ti me(or number of iterations) that 出 eprobl emneeds tobesearchcd
is difficult to d巴termi ne. Whi l e 山 is can be based on moni tori ng, gi ven the
stochasti c naturc of thesearch, it cannot be assertedthat 出 egl obal mi ni mumhas
beenestab1i shed. The SA al gori thmitself canbe expectedtoconvergewhi l ethis
propertyis toosl owfor practical use[6]. A larger number of iterations is more
likelytoyi el dthegl oba1solution.
• SA searchdoes not subscri beto iterativei mprovement. It i5anexampl e of a
stochasti c evol uti onary process. The i mpl ementati on uscd here 陀 P問 seots a
memoryl ess se鉱 山 .
Run 4: Starti ngdesi gn=ト 5,-5]; s = -2. 7567; Number of iterations = 1000
Useful number of iteratioos (when desi gnchanges occurred) = 558
Fi nal desi gn= [-3. 3397, 6.2019]; Fi na1Obj ecti ve= -2. 5674
This is a repeat of Run 1: Fi 肝 民 9.9traces thesearchfor thegl oba1mi ni mumfor this
run. It appears 白 at 1000 iterations is not sufficieot todetermi oethegl obal mi ni mum.
Whi l e thesol uti onhas escapedthestartingloca1mi ni mum, it has not yet escaped出 e
next local mi ni mum. Si ncetheva11eys (loca1mi ni mums) inthisexampl e arecircular,
note出 at thesol uti onmoves aroundintheva11ey but has not managed toωcape.
Run 5: Starti ngdesi gn= [-5, -5]; s = -2. 7567; Number of iterations = 10,000
Useful number of iterations (when desi gnchanges occurred) = 3813
Fi nal desi gn= [4.0037, 3.9996]; Fi nal Obj ecti v巴 = ー 19. 6682
No figureis presented. The sol uti onconverges tothegl obal mi ni mum.




' 同
10 B e 4 - 2 0 2
η- val 凶 S
- 4 - 6 - 8 10 8 自 4 - 2 0 2
X, -val ues
- 4 - 8 - 8
Dl screte Probl ems: In di scrcte probl ems, the S A al gori thmwill use a di screte
desi gnvector det巴 四 uned stochasti cal l y 合 o mthe avai1abl e set Each el ement of 出 e
desi gnvector is randoml y chosen f rom血 epermi ssi bl eset of val ues. The remai ni ng
el ements of th巴 S Aal gori thmaredi rectl y appl i cabl e. If 山 eobj ec凶vedecrcωcs, then
the desi gn change is accepted. lf it i ncreascs, then a condi ti onal probabi l i ty of
acccptance is determi ned al l owi ngthesol uti ontoescape thc mi ni mum. Thi s is lcft to
出 巴 隠 ader. The si gni fi cant effort is thei denti fi cati onof 山 erandomdesi gnvector.
Exampl e 9. 2 Thi s is a functi onof t wo vari abl es that MATLAB uses for iIl ustrati ngthe
graphi cs capabi l i tyof 也巴 software. Typi ng peaks at theCommand prompt wi l l draw
the surface pl ot. Typi ng ηpe peaks. mat the Command prompt wi l l provi de 血 e
叫 uati ons that were uscd to obtai n the curve. Thi s functi onused as 山 eobj ecti vc is
avai l abl e in Exampl e9_2. mand Exampl e9_2pl ot. m. S A is used~   i denti fy the
gl obal mi ni mum. Fi gure 9. 10illustrates thecontour pl ot a10ng wi 自 由 巴 changes in出 e
desi gn. The functi on has t wo l ocal mi ni mums and thr田 local maxi mums. The
fol l owi ngreprcsents a converged run.
Sl mul al edanneal i ng: non屯 onvergence- l nsulficienlllerallons. Fl gure9. 9
Sl mul al edanneal l ng: Increasl ngβ- decreas自 doppロば unitytoescape. Fl gure9. 8
• Rest釘 ti ngf romdi ffercnt d巴 si gnmust stiUbe i mpl emented toovercome strong
l ocal i nfl uences.
• Restarti ng f romthe same initial de討 伊 is a1so i mportant for establ i shi ng
convcrgence prope凶 es as 出 is is a stochasti c searchprocess.
Const rsl ned Pr obl ems: This chaptcr does not deal wi th constrai nl s or di scrctc
probl cms. 百 l erc arc t wo ways of handl i ngconstrai nts di rcctl y
1. lf thenewdesi gnpoi nt is not feasi bl e, it is not acceptedand no dcsi gn change
tak回 pl ace. This is qui te1四 回 cti vc for r andoms回 民 hdi rccti ons.
2. 百 lefuncti on出 at dri vcs thea1gori thmduri ngeach i ternti onis theone wi ththe
l argcst vi ol ati on. If thecurrent poi nt is feasi bl e, thentheobj ecti vefuncti onwi l l
dri ve 白巴 search. This make sense ω 出 e process is l argel y stochasti c and is
. expected towander around曲 edesi gnspace.
Starti ng的 i gn= [- 2, 2]; ト ー 2.5; Numbcr of i terati ons = 1000
Useful number of i terati ons ( when desi go . changcs occurrcd) = 338
Cons紅泊 ncd probl cms c叩 be expected to take a si gni fi cantl y Iarger numbcr of
S伺 rches. The問 is currentl yno pr' 町 吋 uretoi denti fyconvergencc prope町 of t hcSA
wi th即 sp田 t toconstrai nedopti mi zati on.
- 4 - 3 - 2 -1 0 2 3 4 5
Fl gure9.10 Sfmul atedanneal i ng: Exampl e 9.2.
Fi nal desi gn= [0. 02279, -1. 6259]; Fi nal Obj ecti ve = - 6. 5511
Thi s is thegl obal mi ni mumof theprobl em.
Therewas a si gni fi cant change made to出 eS A al gori thmbased on initial 旬 stswi 出
血 eex田 uti on. At thestarti ngdesi gn, thedesi gnwas l argel yposi ti ve(8.95). Changi ng
it toa negati venumber di dnot permi t th巴 sol uti ontoescape thel ocol rni ni mumor to
swi ng oround the nearby l ocal maxi mum. Numeri cal experi menl s suggest 出 鉱 山 e
constant f 3be mai ntai ned between
一2.5sf 3S- 1 (9.6)
Otherwi se no change was made to山 eal gori thm.
9. 2. 2 Genet i c Al gor i t hm( GA)
百 leGeneti c Al gori thmis an i mportant part of 0 newareaof appl i edresearchtermed
Evol uti onary Computati on. Thcse areactual l y searchprocωses and natural l y useful
for di scoveri ngopt i mumsol uti ons. Hol l and [7] was thefirst tousc thetechni que, but
i ωuse as an opti mi zati ontool began ineamest in白 elatc 19808, devel oped moment um
inthemi d- 1990s, and conti nuωto attract seri ous i nterest to血 y. The termevol uti onary
is suggesti veof 出 enatural process associ atedwi 出 bi ol ogi cal evol uti on-pri mari l y
theOarwi ni an rul 巴 of thesel ecti onof thefittest. This takes pl acebecause出 econstant
mutati onand recombi nati on of thechromosomes in血 epopul ati onyi el da better gene
StructUI 芭. A11 of these terms appe釘 in the di scussi ons rel ated to evol uti onary
computati on. Thi s basi c pro田 ss is al sofol dedi nto白 evari ous numeri cal techni ques.
The di scussi ons can become cl ear if 恥 termchromosomes can be associ ated wi th
desi gn vari abl es. Over the y国 民 , Evol uti onary Computati on has i ncl uded Geneti c
Pr ogr ammi ng- whi ch devel ops prograrns 白 紙 rni rni ck 山 e evo1uti onary proc蹴 ;
Geneti c Al gori thmsー opti rni zati onof combi natori aVdi screteprob1ems; Evol uti o, 胤 ry
Programmi ng- opt i mi zi ng conti nuous functi ons wi thout recombi n甜 on;
Evol uti onary Strategi es-opti rni zi ngconti nuous functi ons wi th recombi nati on [8].
This c1assi fi cati onis qui tel oose. For exampl e, GAs have been used, and conti nue to
be used effecti vel y to sol ve conti nuous opti mi zati on prob1ems. Evoh瓜 onary
al gori thms have been successful l y appl i edtoopti mi zati onprob1cms inseveral areas:
engi necri ng desi gn, pararneter fitting, knapsack probl ems, transportati onprob1ems,
i mage processi ng, travel i ngsal esman, schedul i ng. and soon. As can be infe町 吋 f rom
山 elist. a si gni fi cant number are合 o mthc arcaof di scrctc programmi ng.
Thi s secti on and 出 is book i ncl ude onl y GAs, though al1of the evol uti onary
al gori thms share many common features. GAs generate and use stochasti c
i nformati on in thei r i mpl ementati on and therefore can be consi dered gl obal
opti mi zati ontechni ques. They areexcepti onal1y useful for handl i ng ill behaved,
di sconti nuous. and nondi fferenti abl eprobl ems. Unl i ke theS A, convergence tothe
gl oba1 opt i mumin GAs can onl y be weak1y establ i shed. GAs generate and use
popul ati on of sol uti ons- desi gn vectors ( X) - un1i ke 恥 S A where a search
di recti on (8) was generated. This di sti ncti on is essenti al to understand the
di ffi cul ty of handl i ng conti nuous probl ems in OAs. For exampl e. usi ngr andom
numbers tog巴 neratea val ue for Xl o theval ues of 3. 14,3. 141, or 3. 1415 wi l l al1be
di sti nct and have no rel ati on to each other.
Gener l c Genet l c Al gorl t hm( GGA) (A9. 3)
Step1: Set up an initial popul ati onP( O) - an initi a1set of sol uti onor chromosomes
Eval uate 血 ini 出 1sol uti on for fi tness-di 能 rentiate. col l ate, and rate
sol uti ons
Generati oni ndex I = 0
Step2: Use genel i c: operal ors togeneratetheset of chi l dren(crossover, mutati on)
Add a newset of randoml y generatedpopul ati on(i mmi gral l ts)
Reeval uate山 popul ati on-f i tness
Pcrf ormcompeti ti ve sel ecti on・ ・ ・ - whi ch mcmber s wil1 be part of next
generati on
Sel ectpop凶 ati onP(t + l ) - same number of member s
If not converged t←t + 1
GoToSt ep2
The vari ous terms areexpl ai nedbel ow.
Chr omosomes: Whi l e it is appropri ate to regard each chr omosome as a desi 伊
vector, thereis四 i ssueof representati onof thedesi gnvector for handl i ngby geneti c
op巴rators. It is possi bl e to wor k wi th白 巴 desi gn vector di rectl y (as in 出 e i ncl uded
exampl e) or use some ki nd of mappi ng, t l四 1 (real encodi ng) or bi nary (bi nary
encodi ng). Earl i er wor k inGA used bi nary encodi ng. In出 is book thedesi gnvector
is used di rect1y for a11 transacti ons. Real cncodi ng is 陀 commended. A pi ece of the
chr omosome is cal l edan allele.
Fi tness and Se/ ecti on: The fi tness functi on can be rel ated to thc obj ecti ve
釦 ncti on. τbe popul ati on can be rankedl sorted accordi ng to 血 e obj ecti ve functi on
val ue. A sel ecti onf romthis popul ati onis necess釘 y for (1) i denti fyi ngparents and (2)
i denti 匂 i ngi f 出 eywi11be pr omo飽 dt o曲 enext generati on. There arevari ous sel ecti on
schemes. A fracti on of the best i ndi vi dua1s can be chosen for reproducti on and
promoti on, theremai ni ng popul ati onbei ng made up of newi mmi grants. This isca11巴d
tournament sel ecti on. A modi fi cati on of 出 is procedure is to assi gn a probabi l i ty of
sel ecti on for 白色 orderedset of thepopul ati onfor bo由 reproducti onand promoti on.
This probabi l i ty can be based on roulettewheel , elitist selection, linear or geometri c
rankillg [10]. The i mpl ementati on in this book uses ranki ng wi thout assi gni ng a
probabi l i ty of sel 田 tion. 1t uses theobj ecti vefuncti onval uetorank sol uti ons.
Inlt, 加 I Popul ati on: The paramet巴 r here is the number of initi a1 desi gn vectors
(chromosomes) for 出 estarti nggenerati on, N
' 古 山 number is usual l y kept thesame
i nsuccessi ve generati ons. Usual l y a nor ma1r andomgenerati onof thedesi gnvectors
is recommended. In some i nstances, dopi ng, addi ng some good sol uti ons to 白 e
popul ati on, is performed. It is di ffi cul t to establ i sh how many member s must be
consi deredtorepresent thepopul ati oninevery generati on.
Eval uati on: The vari ous desi gnv配 tors have to be eva1uatedto s田 if 曲 ey can be
used for crea出 g chi l dren 白 at can become part of the next generati on. Fi tness
attri butes can serv巴 wel l for this pu叩 ose. For thecaseof unconstrai nedopt i mum, the
obj ecti ve functi onJ(X) is a good measure. Si nce由巳 gl obal mi ni mumis unknown, a
progressi vemeasure may be used.
副 司 星 芸 品 l ~ û 凶 m     ~ 品 幽 温
Geneti c Operators: 百 l ese provi 也 ways of defi ni ng new popul ati ons f rom
exi sti ngones. 百l I sis al sowhere theevol uti ontakes pl ace. Two types of operati onare
recogni zed. The first is crossover or recombi nati on, and thesecond mutati on.
Crossover Operators: There are several types of crossover strategies. A si mpl e
crossover is ilI山 trated in Fi gure 9.11. Here, a pi ece of 白 e chr omosome- an
a11el e- i s exchanged between t wo p紅 ents P" P
to produce chi l dren C" C
• The
l eng血 of thepi ece is usual l y determi ned randoml y. Mathemati cal l y, if theparents 訂 e
X = [x" X2, . • . ,Xn] and Y (defi nedsi mi l arl y) , and r is therandoml y sel ectedtruncati on
i ndex. 白 enthechi l drenU ( =C 1) and V ( =C
), by si mpl e crossover aredefi nedas [11]

t ﹄



i fi <r
o出 erwi s巴
if i < r
otherwi se
Ari thmeti c crossover is obtai nedby a Ii near combi nati on of t wo parent chromosomes
ωyi el d t wo chi l dren [9]. If Il.
and ^'z are randoml y generated numbers, then the
chi l drencan be defi nedぉ






い い 1; 入i ' ヤ 0ー it is a convex∞I巾 i nati on
P2 Ct C2
Chi l drcn
円 gure9
If thereareno restrictions on λ' s, it is anq伊 necombi nati on. For real λ' s it is a l i near
combi nati on. An i mportant crossover operator is a di recti on-based crossover
operator, if JtY) is less thanJtX) and r is a unit randomnumber:
c= r ( Y - X) + Y
Mutati on: Mutati onrefers to白 erepl acement of a singleel ement of a desi gnvector
by a randoml y genemted value. The el 巴ment is alsorandoml y chosen. For 曲 巴 desi gn
vector X, for a randoml y sel ectedel ement k
, i *k
, i =k
There are oth巴 r types of mutati on i ncl udi ng nonuni formmutati on and directional
mutati on. The reader is referredtoReference9 for a rel evant description.
I mm.匂 'fants: Thi s is 出 eaddi ti onof a set 01 ' l' andoml y generatedpopul ati onineach
generati onbeforetheselectionis made for 出 enext generati on. The searchprocess in
GA is broadl y classified 総 出 pl orati on and expl oi tati on. Expl orati on i mpl i es
searchi ngacross theentiredesi gnspace. Expl oi tati onis focusi ngmore in thearea
promi si nga solution. Typi cal l y, crossover and mutati onareways tonarrowthesearch
to anareaof desi gnspa田 basedon血, echaracteristics of thecurrent popul ati on-
suggesti nga local mi ni mum. To keep由 esearchopen for thegl obal mI nI mum, other
areas must bepresentedfor expl orati on.ηl i s Is achI cvedby bri ngi nginanunbI ased
popul ati onandhavi ngthemcompete for promoti onto血 enext generati onal ongwi 白
thec即 時 nt set . Inmany vari ati ons theseareal l owedtobreedwI th曲 ecurrent good
sol uti ons beforeeval uati on.
The ideas presentcdherearebasi c toall of evol uti onarycomputati on印 dspecific
toGA. They arealsosi mpl ebut compl ete. For eachcategorylistedabove thereare
more sophi sti catedmodel s 白 at arebei ngresearchedanddevel opedand出 ereader is
encouragedtoseek i nformati onfromthecurrent literature. Thc specificGA algorithrn
i mpl emented in 由巳 book i ncorporates all of 白 e el ements above in a si mpl e
i mpl emen組組 on.
Genet l c Al gorl t hm(A9. 4) The al gori thmactua11yi mpl emented isgi venbel ow. In
line wi th many other al gori thms avaIl abl e In thIs text, It Is provI ded as a way to
understandandappreci atetheworki ng of theGA. It isa1someant toprovi dea starting
poi nt for theuser toi mpl ement his own variationof theGA si ncethesetechni ques are
sensitiveto 血 especific probl embei ngs凶 died. Another i mportant featuremI ssi ng
合om出 Isi mpl ementati onisanencodi ngscheme. The desi gnvector ishandl eddirectly
toprovI dea direct fcedback.
  、 肺 , 三. . . 昭 三
よ , , ,  


S旬 P 1. Set up an initial popul ati on. Ten sets of desi gn vectors are randoml y
created. Fromthis set, anIniti a1generati onof two vectors is selected. Thi s is
出巴 pool avai l abl e for propagati on. Thi s is also the mI nI mumto perform
crossovers. The fitness functi onIs 出 eobj ectIvefunctIon.
Generati onI ndext = 1
Maxi mumgenemti onI ndex = tmax
Step2. Geneti c Operati ons
Two Si mpl e crossovers
Two Ari thmeti c crossovers (convex)
One Di recti onal crossover
Mutati onof all sevendesi gnvectors sofar
Fouri mr凶 g細 胞
Sel ecti onof thebest two desi gnvectors (from18vectors) for promoti on
to出 enext generati on
If 1 = 1m叫 , Stop
I ncrement generati onindex: I ←t + 1
Go toStep2
Exampl e 9.3 The exampl e for 也 issectionisa Bezi er curvefittingprobl em. Gi vena set
of 砂 data, findthefifth-order Bezi er curve白 at w迦 白 血 eda也. 百 leobj ecti vefunctionis
血 leぉ tsquar叫 errorbetw,田 n出 eoriginal dataandthedatageneratedby theBezi ercurve.
百世 sprobl emhas beenadclressedearlier 出 rougha conti nuous deterrninistictechni quein
anearlier chapter. 百 ereareeなhtdesi 伊 variablesass∞iatedwi ththeprobl em. 百 lereare
several reasons for thechoi ceof theprobl em. Fi 隠し theglobal solution, if it exists, must
be closeωzer o,胡 d鉛 cond,白 is provI des a way ωdescri be eight desi gnvariables in
two-di m巴nsi onal space(plot). 官 lebest val uefor thedesi gnvector 泊 everygener甜 oni s
plotted to see howthe curve fit i mproves. Si nce encodi ng Is not bei ng used in the
al gori thm, a reasonabl yl ongdesIgnvector is necessaryωobserve th巴 crossovers. 百 le
original dataareobtai nedfromthefolI owi ng function:
Jtx) = 1 + O. 25x + 2e-
cos 3x (9.9)
Fi gure9.12 illustrates 血 eseωP of theexampl e i ncl udi ngtheeight desi gnvariables,
whi ch are thex and y compo即 時 of thei ntemal vertices 伽 t determi ne曲 efitted
curvecompl etel y.τ 'be functi onishardcoded inCaI l GA. m. There areseveral reasons
why 出 is Is not 由 ebest exampl e for illustration of GA. FIrst, it Is 叩 exampl e in
conti nuous variables. Second, local mi ni mums arenot verydi sti ngui shabl eand there
are i nnumerabl e m朗 y of them. The gl obal mi ni mumis not substantially different
fromthelocal mi ni mumval uetow紅 rant expl omti onof thegl oba1solution. Thi rd, the
probl emIs not ill behaved. The most i mportant reasondepends not on theal gori thm
buton出 ecomputatI onof theobj ecti vefuncti on-ca1cul atIngtheleast squarederror.
2. 5
  1. 5


0. 4
円 gure9.12
0. 6
Bezier curves
ーー Orfgln剖 Dala
- - Rlleddala
o Vertlces
0 ・
(Xs' X
。... . …ト 一 一 / 0・ " い
旬 、 } ‘ 。 ソ / . 明 日 }
0.8 1
12 1.4 1. 6 1.8
GenetlcAlgorithm. descriptionof Exampl e9.3.
It uses a Newton- Raphson techni que出 at is prone tofai1urc if thc curvefolds over
severel y,or 出 cvertices areveryc10setoge出 er. One way toovercome thisis tocause
thex-Iocati onof thev巴 rtices tomonotoni cal l y i ncrease(whi chis i mpl emcntcdin出 e
popul ati ongcncrati onfunction). Thi s upsets many of thecrossover operati ons SIOCC
出 町 arenowco叫 伽 allyrandomin蜘 dof p叩 lyrandom.
百 lefol l owi ngfiles arenccessaryfor 山 ssp配 ificex創 npl erelatingtothehandl i ng
of Bezi cr curves: c僧 符 . m,Factorl al . m, Combl nat l on. m, Bez_SCLError. mand
areavai l abl einthecode di rcctoryfor thechapter.
The GA inAI gori thrn(A9.4) is i mpl ementedusi ngthefol l owi ngm- flI es.
CaI l GA. m
Genetl cAl gorl thm. m
popul ator. m
Sl mpl eCrossover. m
Ari 肋 metl cCros:関 I ver.m
Di 開 cti onal Crossover. m
Mutatl on. m
Dr awCur ve. m
Settingup parameters anddatafor thea1gori thm
Di 陀 cts theal gori thm
Gencrates randomdesi gnvcctor
Pcrforms si mpl ecrossover betweentwo parents
Performs ∞nvex crossovcr betw偲 ntwo parents
Performs one dircctional crossover
Performs mutati onof 出 edesi gnvectors
Draws thc curvefor eachdesi gnvcctor

Fi gures 9.13to9.17capturc thc app1i cati onof GA toExampl e 9.3through200
generati ons. Thc reader shoul dbe abl etorunthecxampl e fromthecode di rcctory
of Chapter 9. Because of the heuristic natureof the gcncrati onof sol uti ons, the
sol uti on trail will be di fferent f rom山 at shown bel ow. Runni ng the exampl c
several ti mes by i nvoki ngCal l GA f romtheCommand wi ndowshoul dillustrate
白 紙 the GA al ways manages to cl oscl y approxi mate thc curve after 200
generati ons. The maxi mumgcnerati on va1ue is not predi ctabl e and cannot bc
estab1i shed wi th certainty. It relies on numeri ca1 experi mentati on. This number
appears casi l y to be over 1000 in many i nvesti gati ons. The reader is urged to
expcri mcnt wi ththeexampl e tocxpl orethis issue. Inrunni ngCaJl GA. mthebcst
val ues of the desi gn vector and thc correspondi ngobj ccti ve functi on val uc are
capturcdin the vari abl eXbest - an array of 201 rows and 9 col umns (assumi ng
200 generati ons). The first 8 eol umns carrythebest desi gnfor thegenerati onand
thelast col umn is thecorrespondi ngobj ecti vefuncti onval ue.
Fi g町 e 9.13 is 白 e best desi gn fromthe starting generati on. It has 叩 obj ecti ve
functi onva1uc of 32.13. 百 lis is 山 ecπor squaredover 100datapoints of theorigina1
dataand山 ecurvegeneratedby 出 c desi gn variables. Thc desi gn vari abl es arc 出 e
verticω iII ustratcdinthefigurc. The basi c curveof Fi gu陀 9. t3is obtai nedusi ng由巳
2. 5
. .

0.5・・・ 6

-0. 5
: 。
Fl gure9.13

1.5 2 2.5
x data
Best fltatg自 neratlont", 1.
司g咽 0.5
= 均
" . . . ,. 司咽咽 ..咽 4ドー" " " ' ,・ . ,. . . . ,. . . . ,圃 . 唱 ・ ‘ . . . . ' 咽 ‘ ・ + 。、.
- 0.5

嗣 ZO L
自立 0.4 0.6 O. 目 官 1.2 1.4 1.6 1.8 2
x data
Fl gure9.14 Best f i t : generati ont=200.



に一。 唱。。
唱50 200 250
x data
円 gure9.15 Obj ecl i vefunctionfor everygenerati on.
>> Dr awCur ve( Xbest ( 1,1: 8) )
血 is pr' 田 吋 ure. 1t is possi bl e to scparatel y accumul ate onl y desi gns 白 紙 are feasi bl e
whi l e conti nui ngtoi mpl ement thestandardprocedure. Sccond, usea strategyto命 i ve
the search process through constrai nt sati sfacti on and decreasi ng 出 e obj ccti vc,
si mul taneousl y or separatel y. Thc al gori thms i ndi catcd in this chapter can easi l y
i ncorporatc出巴 n田 dedch叩 ges.
The a1gori thmi c procedure as i mpl ement ed do田 not real1y care if 也 eprobl em
is conti nuous or di screte. 1t j ust works on a current set of val ues for the desi gn
vari abl es. Di screte va1ues wil1 pri mari l y requi re mi nor changes in thc way the
popul ati on is generated. A1so thc i mpl cmcntati on of the crossover and mutati on
operators wil1need tobe changed. These changes inno way detract f romthem吋or
i dea bchi nd the GA.
D悶 wCur ve. mfuncti on. To drawtheiniti a1curve (出 efi gureis edi tedsubsequentl y
wi thpl otedi t) intheCommand wi ndow:
Fi gure 9. 14is thebest desi gnat 出 e fina1generati on(generati on number = 2∞) .
百 leobj ccti vefuncti onis 0. 0729. 1t is cl ear f romthefi gure白 紙 出 巴 G Ais attempti ng
to fi ndthesol uti on. This can be establ i shed by I∞ki ng at Fi gure 9.15 whi ch is 山 e
i terati onl generati on hi story of the G A for this exampl e. The obj ccti ve functi on is
genera11ydccreasi ngthrough出 egenerati ons. Fi gur芭 9. 16is a sca1ed representati onto
di spl ay thebehavi or of theG A at lat巴r iterations. Fr om白 edatait is al 80apparent thnt
出 is cxampl c needs torunfor mor c白 叩 200generati ons. The fina1fi gure, Fi gure 9. 17
represents 白巴 movcment and thecl usteri ngof theverti ces through thegenerati ons. It
is suggesti vc of l oca1refi ni ng白 紙 is associ atedwi ththeGA.
Suggest ed Extensi ons: Most of the engi n館 ri ng desi gn probl ems i nvol vc
cons町高 ints. Currcntl y themost popul ar way tohandl e constrai nts is toreformul ate thc
probl emusi ng pcna1ty functi ons (Chapter 7).τ 'here are scvcral oth巴r al temati ves to
Cl osl ng Comment s: I n summar y, for a heuri sti c techni quc, 出 e GA is very
i mp陀 ssi ve. 百 lC samc can be sai d for 出巴 SA. Incl udi ng the fact 白 at practi cal
engi neeri ngprobl ems must requi re partl y di sαete rep民 sentati on,血 e tcchni ques in
Chapte隠 8佃 d 9 arevery rel evant to出 epracti 田 of desi gnand devel opment . Wi t h
the enormous computati ona1 resou民 e in 白 e desktops and l aptops, an effecti vc
programmi ng l anguage Ii ke MAT L AB, and a wi de vari cty of al gori thms 山 at arc
avai l abl e, it is possi bl c nowto desi gn an i ndi vi dual approach to many opti mi zati on
0.4 scal edfor l owest val ues
0. 35





乞同 由由〉
由 コ個〉

- 0.5
-20」 .4
0.6 0.8

Locatl onof desl gnvarl abl es over 200generati ons
60 80 100 120 140 160 180 200
x data
2. 2 2.4
20 40 1.2 1.4 1.6 1.8
x val ueof vertex
Fl gure9.16 Ob同 tivefunctlonscal ed10showcontlnual decreaselor laler generatlon.
円 gure9.17 Trace01Ihedesl gnvarlables Ihroughgeneratfon.
probl ems. Thisisi mportant topracti ci ngengi neers who canbri nginluitionand i nsi ght
for theparti cul ar probl em白 eyhave tosol veina tight ti meframe. Intheengi neeri ng
curri cul ur丸 紅 adi ti onal desi gn opti mi zal i on courses l end to deal onl y wi 出
deteロ 凶 凶 stic conti nuous al gori thms. That initself was mo民 自 叩 theti me avai l abl e
for i nstructi on. Di sc問 te opti mi zati on or gl obal opti mi zati on are neeessari l y skills
l回 medby personal endeavor. 百 lCovercmphasi s on determi ni sti c programmi ng needs
tobe revi si tedinthelight of current devel op即 時 in白 earcaof opti mi zati on. T叫 ay,
computer programmi ng inengi n田 ri ngdepar加 ents and its rei nforccment is usual l y
adj uncl 10other devel opments in血 ecurri cul um. New programmi ng pl atforms Ii ke
MATLAB offer a way to resol vc 出 is di sadvanl age. New al gori thms al so reflecl a
si mpl i ci tythat canbe回 sily官 加 sl atedand appl i ed.
百 is book has been about programmi ng inMATLAB, l eami ng about traditional and
current techni ques in desi gn opti mi zati on, and transl oti ng these opti mi zati on
蹴 加 i ques intoappl i 回 .tions. lt is al soabout expl orati onand devel opi ngani nsi ght 凶 o
theprocess of desi gnopti mi zati on. The code白 紙 is provi dedis onl y meant tobe a
startingpoi nt .
1. Ar ora, J. S. ,l ntroducti on10 Opti mal Desi gn, McGr aw- Hi I I , NewYork, 1989.
2. K討'kpa凶 ck,S. , Ge)att, G. C. ,lI ndVecchi , M. P. , Optimizationby Si mul atedAnnea1ing,
Sci ence, Vol. 220, No. 4598, May 1983.
3. Me凶 polis,N. , Rosenbl uth, A. , Rosenbl uth, M. , Telle, A. , andTeller, E. , Equlltions of
Sate Ca1culations byFast Computi ngMachi nes, J oumal of Chemi cal Physi cs, Vol . 21,
pp. 1087-1092, 1953.
4. Bochavesky, 1. 0. , J ohnson, M. E" andMyron, L. S. , General i zedS回 ula除 dAnnea1ing
for Functi onOpti mi zati on, Tecl mometri cs, Vol . 28, No. 3, 1986.
5. J ones, A. E. W. , andForbes, G. W. , An Adapti veSi mul atedAnnea1i ngAl gori thmfor
Gl obal Opti mi zati onover Conti nuous Vari abl es, J our :聞 1of Gl obal Opl i mi l.Cl tion, Vol. 6,
pp. 1-37,1995.
6. Cohn, H. , andFielding, M. , Si mul atedAnneal i ng: Searchi ngfor anOpti ma1Tempe凶 ure
Schedul e, SI AMJ oumal on Opti mi zati on, VoL 9, No. 3, pp. 779-782, 1999.
7. Hol l and, J. H. , Outl i neof a Logi cal Theoryof Adapti veSystems, J oumal of A CM, Vol.
3,pp. 297-314,1962.
8. Dumi trescu, D. , Lazzerini , B. , Jai n, L. C. , and Dumi trcscu, A. , Evol uti onary
Computati on, Intemati ona1Seri es on Computati ona1Intel l i gence, L. C. Jain, (edふ CRC
Press, Boca Raton, FL, 2000.
9. Gen, M. , andCheng, R. , Geneti c Al gori thms and Engi neeri ngOpti mi zati on, Wi l ey Seri es
inEngi neeri ngDesi gnandAutomati on, H. R., Parsaci (scrics cd.) , Wi l cy- Intcrsci cnce,
NewYork, 2000.
10. Gol dbc屯, D. , Geneti c Al gori thms UI Search , Opti mi zari on , and Machi ne Lea mUl g,
Addi son- Wesl ey, R伺 di ngMA, 1989.
11. Houck, C. R. , J oi nes, J. A., and Kay, M. G. , A Geneti c Al gori thmfor Functi on
Opti mi zati on, p句 er accompanyi ng the GAOT (Geneti c Al gori thmOpti mi zati on
T∞Ibox) , No巾 Carol i naStateUni veぉ ity.
No fonnal probl ems are suggested. 百 le reader is encouraged to devel op hi s own
vari ants of 批 t wo al gori thms, i ncl udi ng f onnal ones avai l abl e in 出 e sugges削
literature. He is al so encouraged to consi der extensi ons to constrai ned and di screte
probl ems. He is r ecommended touse hi s versi ons toexpl o問 vari ous exampl es f ound
in the literature. I n compari son wi th 出 edetenni ni sti c techni ques (until Chapter 8) ,
thes巴 heuri sti c met hods are trul y fasci nati ng and il1ustrate consi derabl e robustness,
and what is cven mor e chal l engi ng10our pcrcepti on, they can sol veany probl emwi th
no reservati ons what soever. τhey are i ncredi bl y basi c in constructi on, si mpl e to
program, and easy todepl oy. They onl y requi rea lot of ti me tosearchthedesi gnspa田 .
百 l eMA11. ABf i創 凶 lyof producl s i ncl udes theOpti mi zati onTool box, whi ch is a l i brarv
of m・filω 蜘 伺 . n be used to叫 ve p帥 l ems in脚 蹴 aof opti mi z副 on. 1t con凶-
many of the met hods that have been expl ored in出 is book, and several 白 at arenot
avai l abl e in the Tool box. For 山 os巴 consi deri ngthepracti ce of desi gn opti mi zati on
seri ousl y, theOpti mi zati onTool box is a worthy i nvestment . Many of the techni ques
and di scussi ons inthebook were kept si mpl e toil1ustrate白巴 con田, pts, 回 d出 em-fi l es
were used toprovi de MA11. ABprogrammi ng experi ence. Most of them-fi l es avai l abl e
on 曲 e w中 山 wi l l need to be d附 l oped further for scri ous prac削 appl i cati on.
Another i mportant reason for havi ng access to 白 e Opti mi zati on Tool box is 白 紙 it
al l ows one 10takeadvantage of MA 11. AB' S open source pol i cy,也 at is, tohave低 cess
10the source files. You may be abl e to modi f y and extend those f i l esωmeet your
speci al opti mi zati on 問 qui r芭 ments. Whi l e some of the programs f romthe Tool box
have b田 n used or referencedel sewhere in曲 is book,出 is chapter col l ects t hemina
si ngl epl ace for 回 syreference. As can be expected, the materi al of 出 is chapter wi l l
be dated.τ 'he chapter is based on MA11. AB Versi on 5.3. As of 白 is wri ti ng, Versi on
6. 0f 6. 1was shi ppi ng. I n general ,the MATt AB products,especi al l y theOpti mi zati on
Tool box, have reached a l evel of maturi ty such 伽 t newer versi ons s 訂臼h削o
i nval i datethei nfonnati onin山刷i胞ss釦ec 叫“ωo 叩 n. What shoul d be exp 田 ted, however, is 伽 t
themateri al in白 is chapter wi11most l i kel ybe upgraded, enhanced, and extended wi th
each new rele凶 eof MA札 油 .
角 7 Q
Most of the basi c i nformati on in 出 is chapter is f rom山 user's gui de for the
Opti mi zati onToo1box [1]. If you arebuyi nga speci a1lybundl edCO that i ncl udes the
Opti mi zati onTool box, thenyou may not have Reference 1 inbook fonn. You may
have Reference1 as a POF fileon thcCO or you may have access toit. For exnmpl e,
on 山 eWi ndows NT pl atformfor dcfaul t di rcctory insta11ation the POF versi on of
Reference 1 is avai1abl c at
C: ¥MA TLABR 11¥hel p¥pdCdoc¥opti m¥opti m_tb. pdf
Refcrence 1 provi dcs exampl cs of usc, di scusses sp田 ia1cases, a1gori 山 ms 凶 ed,阻 d
many 0出巴 r si gni fi cant topics.τ 五is chapter does not attempt to document them.
Li mi 飽 d dupl i ca品on is 凶 ed to bri ng attenti on to the most useful f l回, tures. τbe
emphωi s is on i1l ustrati ngtheuseof 出 eTool box for vari ous exampl es in位。 du田 d泊
thebook. Infact , a l nrgenumber of thetechni ques inth巴 Tool boxarenot usedin血 e
book and are not covered in 白 is section. The experi ence provi ded in this chapter
shoul dbe sufficient tounderstandand usethenewones cぽectivcly.
To use 出 c programs or l cs in theTool box, thre巴 i mportant acti ons must bc
initiated. One, formul nte山 eprobl emintheformat expcctedby MATLAB wi th悶 pcct
to 山 c particular functi on thnt will bc calI ed. Most1y, 出 is will i nvol v巴 usi ng
approp由 teprocedures tosct theri ght.handsideof theconstrni nts tozeroor constant
va1ues. The Tool box a1so makes a poi nt of di sti ngui shi ng betwecn linear and
nonl i near constraints. S巴cond, set/ change 白 e paramete凶 for 白 e op白 ni znti on
a1gori thms. Thi吋, usetheappropri ateopti mi zer and ensure白 紙 ava1idsol uti onhas
bcenobtai ned. AII of 白巴 seacti ons natura1lydepend on theactua1mathcmati cal model
of 出 eprobl cm.
市0.1.1 Pr ogr ams
Thc different types of opti mi zati onprobl ems 出 at canbc sol vedwi 出 thc versi onof
theTool box installedon白巴 computer canbe di scoveredintheCommand wi ndow,
Hel p wi ndow, or inthebrowser usi ngHel p Oesk. In出 eMATLAB Command wi ndow
>>hel p opt i m
The output isreproduc巴dbel ow. They area11 of thevari ous l es intheTool box and
their opti mi zati onfuncti onis a1soidcntified.
Opt i mi zat i on Tool box.
Ver si on 2. 0 (Rl 1) 09・Oct ・ 1998
What ' s new.
Readme - New f eat ur es, bug f i xes, and changes i n t hi s
ver si on.
Non1i near mi ni mi zat i on of f unct i ons.
f a i a ba d- - Sc a l a r bounded nonl i near Eunct i on
mi ni mi zat i on.
mi : 22: :…Mul ωi men山 叫 const r ai ned nonl i near
定mi nsear ch- Mul t i di mensi onal unconst r ai ned nonl i near
mi ni mi zat i on, by Nel der - Mead di r ect sear ch met hod .
f mi aunc- Mul t i di mens i onal unconst r ai ned nonl i near
mi ni mi zat i on.
f gami nf - MI l l t i di mensi onal cons t r ai ned mi ni mi zat i on,
semi - i nf i ni t e const r ai nt s.
Nonl i near mi ni mi zat i on of mul t i - obj ecヒi ve f unct i ons.
f goa1at ? ai n- Mul t i di me悶 i ona1 goa1 at t ai nment
opt i mi zat i on.
まmi ni max ・ Mu1t i di mensi ona1 mi ni max opt i mi zat i on.
Li near 1east squar es (of mat r i x pr ob1ems) .
1aql i n- Li nea?l eas t s quar es wi t h l i near const r ai nt s.
∞::ご : t : : - L L n町 l east 叩 ar es wi t h non明 at i vi t y
Nonl i near l east squar es (of f unct i ons) .
?s qcuwef i t - Nonl i near cur vef i t t i ng vi a l east squar es
(wエt h bo山 l ds) .
l sqnonl i n - Nonl i near l east squar es wi t h upper and l ower
Non1i near zer o f i ndi ng ( equat i on sol vi ng) .
~ z   r o - Scal ar nonl i near zer o f i ndi ng.
f sol ve - Non1i near syst emof equat i ons s01ve f unct i on
Sol ve.
Mi ni mi zat i on of mat r i x pr obl ems.
1i npr og - Li near pr ogr ammi ng・
quadpr og - Quadr at i c pr ogr ammi ng.
Cont r ol l i ng def aul t s and opt i ons.
opt i mset Cr eat e or a1t er opt i mi zat i on OPT工ONS
st r uct ur e.
opt i mget - Get opt i mi zat i on par amet er s f r om OPTI ONS
st r uct ur e.
Demonst r at i ons of l ar ge- sca1e met hods.
ci r cust eat - Quadr at i c pr ogr ammi ng t of i nd Shape of a
ci r cus ヒent .
mol ec ul e- MOl ec ul e conf or mat i on SOI Ll t i on usi ng .
unconst r ai ned nonl i near mi ni mi zat i on.
opt debl ur 工mage debl ur r i ng usi ng bounded
l i near
l east - squar es.
Demonst r at i ons of medi um- scal e met hods.
opt demo - Demonst r at i on menu.
t ut demo - Tut or i al wal k- t hr ough.
bandemo - Mi ni mi zat i on of banana f unct i on.
goal demo - Goal at t ai nment .
df i l demo - Fi ni t e- pr eci si on f i l t er desi gn ( r equi r es
Si gnal Pr ocessi ng Tool box) .
dat demo - Fi t t i ng dat a t o a cur ve.
The t ut demo is recommendcd for a demonstrati onof 印 刷 uresof 山 Tool 胤
Of the above i tems, opt i mset , l i npr og, quadpr og, f mi nunc, and
f mi ncon arefurther expl oredin由 is chapter.
A cl earer pi ctureof thecapabi l i tyof 血 efuncti ons can be obtai nedf romthetabul ar
graphi 叫 rcpres側 a伽 1of thc P帥 l emscts inthefirst chapl er of Ref erence 1. For
exampl e,出 efour tcchni ques referencedin由 is chapter areiIl ustratedinTabl e 10. 1.
10. 1. 2 Usl ng Pr ogr ams
Eacb of thc functi ons intheTool box白 紙 i mpl ements an opti mi zati os tcchl l i quecan
be凶 edinsev釘 al ways. τ 'hesearedocumented inRef erence 1 and areal soavai l abl e
onl i ne through one of the mechani sms of ob也 i ni ng hel p inMATLAB・ For exampl e,
t ypMghel pmneoni n t hCommandwi ndowi dent i f i es thef ol l owi ngdi f f emtways
of usi ng由巳 functi onl program(thefol l owi ngis edi tedtoshowonl y types of usage).
FMXNCON Fi nds t he const r ai ned mi ni mumof a f unct i on of
sever al var i abl es.
FM工NCON sol ves pr obl ems of t he f or m:
mi n F( X) subj ect t o:
A* X <= B, Aeq* X Beq ( l i near const r ai nt s)
Tabl e 10.1 Graphi cal D笛 cri pti onor Some Tool box Functi ons
Uncons町 ai ncdmi ni mi zauon
Li near programmi ng
Quadrauc programmi ng
Cons釘泊 nedmi ni mi zauon
Notati on
mi n.f(x)
mi n. f
x suchthat
[A][μx] 壬 [b]; [A叫] [ μx] =[ b叫]; 当
m吋凶 T 刊l 剛 + ハ 叫 白
[ μAJ[x] 三 [ 股b]; [ μA.J[μ;x] = [ . 股b.ql よ 壬壬 x S u
m官 ni加n.f(x: 功); 凶   伐  
[AJ[.μ[X] 心] S [b); [A.叫qUμ凶x] =[ b叫]; 亘
f mi nunc,
f mi nsear ch
l i npr og
quadpr og
f mi ncon
C( X) <= 0, Ceq( X) 0 ( nonl i near const r ai nt s)
LB <= X <= UB
Pl ,P2,... )
FUN,XO,. . . ) .
There 蹴 12 ways of 山 崎 theρ ni ncon functi on. 百 1C di 仔erenωs depend on 血 e
i nformati on in the mathcmati cal model 白 at is bci ng usoo and the postopti mi zati on
i nformati onthat shoul dbe di spl ayed. Inusi ngtbedi ffercnt 釦 ncti oncalls, 叩 ymi ssi ngdata
白 at do not exist for theparticu1ar exampl es must be i ndi ωtOOusi nga null vector (0).
τbere are 由 民 eke戸 vord旨 加 the above list of functi on calls 白 at need speci al
al tenti onincons回 i nOOnonl i near opti mi zati on.
EXI TFLAG: For f mi ncon血 is i ndi ca飽 s if thc opti mi zati onwas successful .
I f EXI TFLAG i s:
>0 t hen FMI NCON conver ged t o a sol ut i on X.
o t hen t he maxi mumnumber of f unct i on eval uat i ons was
r eached.
< 0 t hen FMI NCON di d not conver ge t o a sol ut i on.
OUTPUT: 羽 山 is speci a1MATLAB structureel ement. Structureis a programmi ng
cons凶 ct usOOinmany hi gher-I evel l anguages. It is si mi1ar toarray inMATLAB. An
array wi l l contai nonl y si mi 加 el ementsー ukc all numbcrs or all strings. A s凶 cture,
on theother hand, can contai ndi sparateel ements, suchas severa1numbers and strings.
The el ements of 由 e structure can be accessed by usi ng the name of the 5回 C組 問
el ement fol l owed by a peri odand出 CPI 吋 efi nOOkeyword for theparti cul ar el cment .
For cxampl e, t wo of theel ements of theOUTPUT are:
OUTPUT. i t er at i ons: thenumber of functi oncval uati ons
OUTPUT. al gor i t hm: t henameof 出 eal gori thmusOOtosol vetheprobl cm
LAMBDA I This is al 50 another structure in MATLAB and contai ns 肱 Lagrange
m叫 ti puers. 百 1eval uesof 由 民 es mul ti pl i ers areuseful for veri 今 i ngthesati sfacti onof
血 e necessary condi ti ons of opti mi zati on. The val ues can be recover叫 by 山 esame
pr田 吋 山 ' C i ndi catedfor thc OUTPUT structurc.
- 圃
LAMBDA . l ower : val ueof themul ti pl i ers at thel ower bound
LAMBDA. i neqnonl i n : val ue of the mul ti pl i ers tied to the i nequal i ty
cons町剖 n悩( s' sin山 巴 加 ok)
Thesc i tems werc associ ated wi th the functi onρ ni ncon. Other functi ons may havc
di fferentl addi ti onal type喧 of i nformati onavai l abl 巴.
10. 1. 3 Setti ng Optl mi zati on Par amet er s
There ar芭 several parametcrs that areusedtocontrol the appl i cati onof a parti cul ar
opti mi 四 tion techni que. Thcse parameters are avai l abl e through thc OPTI ONS
strucl ure. AIl of t hemareprovi dedwi thdefaul t val ues. Thcre will be occasi ons when
it is n巴cessarytochange thedefaul t val ues. One exampl e may be出 at your constrai nt
tol erances arerequi redtobe l ower th姐 thedefaul t Or EXI TFLAG recommends a
l arger number of functi oneval uati ons tosol ve白 E 戸 obl em. Thesep紅 白 meters areset
usi ng a name/ val ue p瓜 r. Gencral l y 白 ename is usual l y a string whi l c the val ue is
typi cal l y a numb巴r. The list of parameters is avai l abl e in Reference 1 and can be
accessedmost easi l y intheCommand wi ndowby typi ng
>> hel p opt i mset
A partiallist of theparameters intheOPTI ONS structure, copi edf romtheCommand
wi ndow. fol l ows.
Di agnost i cs ・ pr i nt di agnost i c i nf or mat i on about t he
f unct i on t o be mi ni mi zed or sol ved [ on I {of f }l
Di f f MaxChange - Maxi mumchange i n var i abl es f or f i ni t e
di f f er ence gr adi ent s [ posi t i ve scal ar I . {l e- l } 1
Di f f Mi nChange - Mi ni mumchange i n var i abl es f or f i ni t e
di f f er ence gr adi ent s [ posi t i ve scal ar I {l e・ S} 1
Di spl ay - Level of di spl ay [ of f I i t er I {f i nal } ]
Gr adConst r - Gr adi ent s f or t he nonl i near const r ai nt s
def i ned by user [ on I {of f } 1
Gr adObj - Gr adi ent ( s) f or t he obj ect i ve f unct i on( s)
def i ned by user [ on I ( of f }l
Hessi an - Hessi an f or t he obj ect i ve f unct i on def i ned by
user [ on I {of f } ]
HessUpdat e - Quasi - Newt on updat i ng scheme [ {bf gs} I df p
I gi l l mur r ay I st eepdesc 1
Li neSear chType Li ne sear ch al gor i t hm choi ce
cubi cpol y I {quadcubi c} 1
MaxFunEval s - Maxi mumnumber of f unct i on eval uat i ons
al l owed [ posi t i ve i nt eger 1
Maxl t er Maxi mum number of i t er at i ons al l owed
posi t i ve i nt eger ]
10.2 EXAMPLES 385
Tol Con Ter mi nat i on t ol er ance on t he const r ai nt
vi ol at i on [ posi t i ve scal ar J
Tol Fun. ・ Ter mi nat i on t ol er ance on t he f unct i on val ue
posi t i ve scal ar J
Tol X - Ter mi nat i on t ol er ance on X [ posi t i ve scal ar 1
The parameter names arei denti fi edinbol dfon1. Theseparameters aretheones 白 紙
wi11con回, 1a standardopti rni zati onappl i cati on. The defaul t val ues for 肱 parame防 r
aregi venwi thi nthecurl ybraces. For exampl e:
>> opヒi ons opヒi mset ( ‘Di agnost i cs' ,' on' ,' Di spl ay' ,
' i t er ' ,' Maxl t er ' ,' 500' ,' HessUpdat e' ,' df p' ,' Tol Con' ,' l .
Oe- OS' )
U theabove statemcnt is usedpri or toactual l ycal l i nganopti mi zati ontechni que,血 e
defaul t val ues for thosep釘 倒 neters wi11be changed intheOPTI ONS structure. The
defaul t vaJ ues for a parti cul ar techni quecanbe found by typi ngtheCommand wi ndow
>> opt i ons opヒi mset ( ‘ f mi ncon' )
前 lere創 芭 several othcr ways to use theopti mset programitself. For details consul t
Reference 1.
百 1巴 exampl es i11us町 ated in 出 is secti on are conti nuous opti mi zati on probl ems
di scussedin山 ebook. 百 l ey are f romIi near programmi ng. quadrati c programmi ng.
unconstrai ned mi ni mi zati on. and constrai ned mi ni mi zati oo. Onl y the appropri ate
mathemati cal model isi otroducedpri or toi11us回 .tingthe凶 eof theTool box functi on.
10. 2. 1 Li near Pr ogr amml ng
The L P exampl e uSed here i ncl udes all three types of constrai nts. The tool box
functi onis l i nprog.
Exampl e 10. 1
Maxi mi zc z = 4xI + 6x
+ 7X3 + 8x
Subj ectto: XI + X2 + x) + X4 = 950
X   ~ 400
+ 3X2 + 4X3 + 7X4 S 4600
3xI +4x:け 5X3+ 6x4 S 5000
Xl o X2. X3. X4 ;?; 0
百 lel i nprogprogramsol ves thefol l owi ngprobl em:
Mi ni mi ze f Tx; subj ect to [A](x] S [b]
The obj ecti ve functi on for the exampl c needs to b巴 changed to a mi ni mi zati on
probl em. The greater 白 佃 or equal toconstrai nt needs tobe changed toa less 山 卸 or
cqual toonc. The l i nprogprograma1l ows equal i ty constrai nt 佃 dexpl i ci t bounds on
vari abl es. 百 leequal i tycons釘 ai nt can be set up wi thA句 andb句・Therearenot many
p紅 白 臨 む ers 白 at need tobe changed except swi tchi ngof the ' LargeScal c' probl emto
・off.' For onl y four desi gnvari abl cs I cavi ngit on shoul d be permi ssi bl e. The f ormof
l i nprogthat wi l l be used is
[X,F VAL,EXI TFLAG] LI NPROG( f ,A,b,Aeq,beq,L B,UB)
Thi s f ormis not expl i ci t1y document ed inRef erence 1 or inthehel pdocumentati on.
Ex10_1. m
is 出 em-ftl ethat is uscd toset up and run theprobl em. It is not di spl ayed
herebut runni ng thefilein自己 Commandwi ndowproduces th巳 fol l owi ngoutput
Opt i mi zat i on t er mi nat ed successf ul l y.
0. 0000
400. 0000
150. 0000
400. 0000
- 6. 6500e+003
The sol uti onto出巴 probl emis
x; = 0; X; =400; x; = 150; x; =4∞7. " =6650
10. 2. 2 Quadr at i c Pr ogr amml ng
Quadrati c programmi ng refers to a probl cmwi th a quadrati c obj ecti ve functi onand
Ii near constrai nts. Whi l e such a probl 巴mcanbe sol vedusi ngnonl i near opti mi zati on
t巴 chni ques of Chapter 7. speci al qua命 冨 tic programmi ng a1gori thms based on the
tech凶 ques of Ii near programmi ng arenormal l y出 edas 出 eyaresi mpl e and faster. If
theobj ecti vc functi onis a quadrati c functi oninthevari abl es, thenit can be cxpressed
i nqua的 自 form. For exampl c,
f ( x
) =x
  2 x
  寸   列 X 2   ~
can be cxpressed悩
I Fi lcs 10be downl ondedfromthc websilenrc indicalcdbyboldface岨 nsserif type.
10.2 EXAMPLES 387
川 ) = [1 2] 同+ド 1 x2] [ i !][::J
f(.日 ) = [c]T[x] 中 . X]T 間 [x]
Exampl e 10. 2 Sol ve thequa合 atic programmi ng probl em 「 3
M刷 in 凶n凶i m 山n凶由 ni ze 也邸 i包回 ze 仇 い ) 川 =斗 [ 一 1ト円一 - 2 一3]T 州T
Subj ect ω x 刻 1+2x勾2一x 勾3 壬 2.5 L 1
- X2 + 3X3 = 1
X1. X2' x) ~ 0
- 1 11
2 l l[x]
1 41
The programtobe used f romtheTool box is quadprog. The standardformat for this
Mi ni mi ze f(x) = c
X + x

Su同 ect to: A x S b
A句 x=b.句
The sol uti oncan be f ound in Ex10_2. m. The defaul t opti mi zati on p直 前 田 :ters are
used so 山 at the opti ons structure does not havc to be changed. The acωaI cal l i ng
statement used is
[x, f val ,EX工TFLAG] quadpr og( H,c ,A,b ,Aeq,beq,LB,UB)
The sol uti onis formattedand di spl ayedinthc Command wi ndow. 1t is copi edf rom
theCommand wi ndow:
Opt i mi zat i on t er mi nat ed successf ul l y.
Fi nal Val ues
opt i mumDesi gn Var i abl es
一一一一一ーーーー・ーーーーー - - - - - - ーーーーーー一一一一一一
0. 5818 1. 1182 0. 3182
Opt i mumf unct i on val ue
- 1. 9218

Lagr ange Mul t i pl i er s f or i nequal i t y const r ai nt
0. 0218
工nequal i t y const r ai nt
2. 5000
Lagr ange Mul t i pl i er s f or equal i t y const r ai nt
0. 0164
Equal i t y const r ai nt
1. 0000
10. 2. 3 Unconst r al ned Opti mi zati on
Exampl e 10.3 is used 10 ilI ustral e 山巴 usc of f mi nunc, 山 e programto be 山 ed for
unconstrai ned mi ni mi zati on. 11 is a si mpl e curv巴 自 民 i ng exampl c (a 101 of
unconstrai ned mi ni mi zati on probl ems in engi neeri ng are curve fitting used for
devel opi ngcorrel ati onf romempi ri cal data). The Tool box has programs speci fi cal l y
for curve fitting. It may be a useful i deafor thereader tocompare thesol uti ons usi ng
thos巴 programs 凶 白 is provi des an opportuni ty touseprograms f rom由 eTool box not
illus回 tedinthis chapter.
Extunpl e 10. 3 A set of dataover 100poi nts represen出 ga cosi nefuncti onisα 四 ted.
A qua世 atic pol ynomi al is used 10 fit the data. An unconstrai ned mi ni mi zati on
probl embased on the sumof squ紅 白 of the error bel ween ori gi nal data and the
pol ynomi al , over all 出 edatapoi nts is sol vedfor thecoeffi ci ents of thepol ynomi al .
百 由 is an ex創 npl eof thecl assi cal l east sq凶 rede町。 .rprobl em.
The exampl e is sol vedin曲巴 m- fiI eEx1 0_3. m. Runni ng thc exampl e al sodi spl ays
a compari son of the fit on a fi gure whi ch is not i ncl uded here. Two sol uti ons are
obtai nedintheexampl e startingf romthesame initial guess. The first one is sol ved
usi ng凶 'er-s叩 tpliedgradi e臨 . The second is thedefaul t finite di fference伊 dients.
The results ar芭 copi edf romtheCommand wi ndow:
Fi nal Val ues ・ Oser speci f i ed Gr adi ent s
Opt i mumDesi gn Var i abl es
- 0. 4309 - 0. 0365 1. 0033
Opt i mumf unct i on val ue
2. 4055e- 004
Gr adi ent s of t he f unct i on
1. 0e- 007*
ー0. 1324 - 0. 2198 - 0. 3275
10.2 EXAMPLES 389
Fi aal val ue- - Def aul t Fi ni t e Di f f er ence Gr adi ent s
Opt i mumDesi gn Var i abl es
ー ー , ー ー ー ー ・ ・ ー ー ー ー ー
一 一 一 ー ー ー - - ー ー ー 一 一 一 一 一 ー ー ー ー
ー 0. 4309 - 0. 0365 1. 0033
Opt i mumf unct i on val ue
ー ー ー ・ ー ー ・ 司 ・ ・ ー ー ー ー 一 一
一 一 ー ー ー ー ・ ー . ー
2. 4055e- 004
Gr adi ent s of t he f unct i on
ー ー ー ー ー ー ー ー ー 一 一 一 一 一 一 ー ー ー ー ー ・ ー ー ー ・ ー ー ー 一 一 ←
一 一 ー ー ー ー ー ー ー
1. 0e- 004 *
- 0. 0121 - 0. 0270 0. 5715
It is general l y bel i e叫 出 鉱 山 制 叫 叫 附 of gradi ents wouJ d Jead10a f iぉ ter and
better 叫 ution. For t i中 nunc to附 恥 gradi ents computed by山 町 , changcs havc
tobc made to山 eopti ons s凶 cl u隠 悩 wel l as thefilewh巴 問 theobj ecti vefuncti onis
c蜘 i削( 油 I J 10_3・m). In印 刷 ons struct脱 出 e' OradObj' parameter shoul dbc
sct to ' on・' i nI SI SEmponmt Otherwi seMATUB domnot cxpECt thEgradi ents tobE ・
provi d吋 bytheuser. The opti onfor this ex創 npl eis sel by
TZ; ; ? ? s、Tol Fun' ,マ;?と ; J : ??三;;zp;;;;ff; ,、山 pl ay' ,
τbi s causes 出 epro伊 mtol ook for 恥 伊 di en岱 泊 出 esecond el em則 oftheretum
val ue. The functI onStatement usdf or obj 10」3. mis
f unct i on [f , gr ad] obj l O_3( x)
Bo白 山 e o対ccti vefuncti on の叩 d 出 egradi e附 (grad) arecal cul ated in the same
functi onm- fiI e.
10. 2. 4 Const rai ned Opti mi zatl on
Exampl e 10. 4 The exampl e is the same as Exampl e 8.4 and 陀 P児 sents lhe
constrai nedex創 npl efor this sccti on. τne mathemati cal model is
Mi ni mi ze f (X) = pLAc
Subj ect to: g.(X): WL
- 12EI S 0. 25 (dcfl 田 tion)
g2(X): WLx. - 8/1σyS 0 (normal stress)
g3(X): WQc - 21xJ 'ty S 0 (shear sl ress)
Some gcomctri c constrai nts torelutc thevari ous desi gnvari ubl es are
g4(X): x, -3X2壬 0
gs(X): 2x
- x ( S 0
g6(X): X3- 1.5x4S 0
g7(X): 0.5x4- x3S 0
The si deconstrai nts on thedesi gnvari abl es are
3 Sx( S20, 2Sx
S 15, 0.125 S X
SO. 75, 0.25S X
S 1. 25
The foIl owi ng rel ati ons defi nethecross-secti onal area(A
)' moment of inertia(乃, and
first moment of areaabout thecentroi d(Qc):
Ac = 2x2x4 + x(xヨ ー Zちら
I ーら x t ( ろ - xJ ( x,-
12 12
Qc=0. 5.引 い l 一川 +0. 5x向 _ XJ 2
The parame旬 rs for 出巴 probl emar官 w=1. 4場 2000,L = 96, E = 29 E+06, P = 0. 284,
σ' y=36E+03,τ:y= 21E+03. There arethreenonl i near i nequal i tyconstrai nts and four
l i near i nequal i ty constrai nts. 官 le code for runni ng the prograrn is avai l abl e in
Ex10_4. m.
The programto be used is frni ncon. The fol l owi ng OPTI ON par副 neters are
opt i ons opt i mset ( ‘Lar geScal e' , ‘ of f ' , ‘ Di spl ay' ,
‘ i t er ' ,‘ Tol Fun' , 1. Oe・ 08) ;
百 le cal l i ng statement used for f mi ncon wi th the obj ecti ve functi on avai l abl e in
obj ecti ve. m, and thenonl i near constrai nts innonl i n_cons. m , is
[x, f val , exi t f l ag, out put , l ambda]
f mi ncon( ‘obj ect i ve' ,xO,A,B,日,口, LB,UB, ‘ nonl i n_cons' ,
opt i ons) ;
The Ii near i nequal i tyconstrai nts areset up inthematri x A and vector B. There are
no Ii near equal i ty constrai nts and they are i ndi cated by the nuIl matri ccs. The
l ower and upper bounds are appl i ed through the vectors L B and UB. The
customi zed part of 山 e output di spl ayed in the Command wi ndowis i ncl uded
bel ow. Accessi ng 出巴 vari ous Structure val ues wi l l provi de detai l s about the
perf ormance of 白巴 al gori 出 m.
Fi nal Val ues
Opt i mumDesi gn Var i abl es
- ー ー ー ー 一 一 ー 一 ー ー
一 ー 一 一 一 一 ー ー ー ・ ー - 司 - - - - ー ー ー ・ ・ ー
6. 8671 2. 2890 0. 1802 0. 2500
Opt i mumf unct i on val ue
ー . ー ー ー ー ー ー ー 一 一 一 一 一 一 一 一 一 ー ー ー ー ー ー 司 ・ ・ ー 噂 ,
62. 4905
Fi nal Nonl i near Const r ai nt s
- - - ー ー ー ー ー ー 一 一 一 一 一 一 一 一 ー ー ー ー ー ー - - ー
1. 0e+009 合
- 2. 7304 - 0. 0025 - 0. 0001
Lagr ange Mul t i pl i er s f or Nonl i near const r ai nヒs
ー ー ・ ・ ー ー ー ー ー ー
一 一 一 一 ー ー ー ・ ・ ー ー ー ー ー ー ー ー . ー ー ー ー ー ー ー ー 一 一 一 一 一 一 ー ー ー ー ー ー ー ー .
0 0 0
Lagr ange Mul t i pl i er s f or Li near const r ai nt s
ー ー ー ー ー ー ー ー ー ー
一 一 一 ー ー ー ・ ー ー ー ー - - -
2. 5879 0 0 0
The reader is encouraged to run the exampl e wi th ・Di agnosti cs' on, expl ore 恥
opttons structure, and山 eoutput s釘 ucture.
1. 白 l mmJ - R. BmEI 仇 M. A. , andGrnce, A. , Opti mi zati on Tool box- f or Use witlr
MATL AB穴 User's Gui de, Versi on2, MathWorks Inc・ , 1999.
理l i i I i 画園
Abstract mathemati ca1model , 5, 9- 10
Acceptabl esol uti ons, 17
Acti veconstraint set, 293
Acrospacedesi gn, 1
Al gori thms,21
AL M mcthod, seeAugmcnl cd Lagmnge
Mul ti pl i cr method
Ana1ytica1condi ti ons, 175-176
Anneal i ng8chcdul c, 359
Ani fici a1var泊 bles,98
Augmented Lagmnge Mul ti pl i cr ( ALM)
m官 thod, 276-281
algori 山 m,277- 278
appl i cati on, 278-281
unconstr凶 nedprobl em,
位 unsform瓜 i on10, 276- 277
Augmented matri x, 103- 104
Bez.i er parame凶 ccurvcs, fittingof,
258- 262
BFOS method, seeBroydon-Fl etcher-
Ool dfarb- Shanno me曲。 d
Bi sccti ontcchni que, 209- 210
Bmnch andBound
叫 gori 出 m,333
appl i ca山 n,333- 335
tcchni quc, 329田 336
Broydon- F1etchcr- Gol dfarb-Shanno
(BFOS) method
n1gori thm. 250
application. 250- 25 1
Cartωi anspaces. 106
C (I anguage) , 24
Cl assωof probl cms. 21
Computati ona1rcsources. 2
印 刷 gateOmdi cnt mcthod. 244- 246,
al gori thm, 244- 245
appl i cati on, 245- 246
Constmi nedopti mi zati onprobl ems.
265- 315,389-391
Augmentcd Lagmnge Mul ti pl i er
( ALM) mcthod for sol vi ng,
276- 281
beamdesi gn, 310- 312
control , opti ma 1 , 313- 315
dcfinitionof probl cm. 266
曲 ect methods for sol vi ng, 281- 302
• Extcrior Pcnnl tyFuncti on但 PF)
method for sol vi ng, 271- 276
fl agpol eprobl em, 307- 310
formul ati onof probl em, 266- 267
Gencrnl i zedReduced Gmdi ent ( GRG)
me伽 dfor sol vi ng, 297- 3但
generi c a1gori thmfor, 269- 270
indircct methods for sol vi ng, 270- 281
目 白 ω鈍 rycondi ti ons for, 267- 269
S叫 uentia1Gradi ent R凶 tomti on
a1gori thm( SGRA), 3但 ー 307
S叫 ucntiaJ Li near Progmmmi ng
(SLP) 叫 gori 出 m,284- 289
S叫 uential Quadmti c Programmi ng
(SQP) a1gori 山 m,289- 296
Cons位置 int functi ons, 6, 7
Cons位泊 nts,1,5
394 I NOEX
Cons位凶 n匂 (conri nued)
伺 ual i ty.7-8
inequality. 8
inLP prob1ems. 98
inNLPprob1ems. 157-159. 172-175.
si de.8
Conti nuous re1axalion. 321- 322
Conl i nuous vnriablcs. 20
Con回 1.0p山 nal, 313- 315
Convcrgi ng10thcsolution. 21
Convcx functi ons. 352- 353
Corners. 20
Davi don- Fl cl cher-PoweU(DFP) method.
246- 249
Dcgrees of freedom( 00町 . 104
Dcscent funclion. 292
D凶 i gn. 1. 2
Dcsi gnfuncti ons. 5- 6
Dcsi gno句eCl i vcs.5
Dcsi gnpammel crs. 5
Desi gnspace. 9
Designvariab1cs. 4- 5
Desl gnveclor. 5
Df P method. seeDavi don- Fl etcher-Powel l
me曲。 d
Di graphs. 336
Di screteopti mi z瓜 i onprob1ems (DP).
318- 348
Bmnch nndBound l echni quefor sol vi ng.
329- 336
conti nuous re1axati onωapproachfor
sol vi ng. 321-322
discretedynami c programmi ng ns
I民 hni quefor sol vi ng. 336-341
Exhnusti veEnumerati ontechni qu巴 for
sol vi ng. 326- 329
I・民監 ndcsi gn. 341- 343
sl nnd釘 dapproachtosol vi ng. 322- 324
Zero- Onc Integcr Programmi ng (ZIP).
343四 348
Di 鉱 偲 teprogrammi ng. 20
Di scretevarinblcs. 20
DOF (degrees of frecdom). 104
DP. seeDi scretcopti m包 ationprob1cms;
Dynami c progrnmmi ng
Dual probl cm. 138-148
Dynami c programmi ng( DP). 336- 341
algori 白m, 340
functi onal 句 U凶 onfor. 338- 339
EBV (Enl eri ngBasi c Vari abl c). 118
Edi tor (MATLAB). 33- 37
Engi nee巾 I g.2
Enteri ngBasi c Vari ab1e( EBV) , 118
EPF mcthod, seeExtcrior Penal tyFuncti on
Equal i ty∞nstrai nts.7-8
Eucl i dcnnspaccs. , ト di me目sionll1. 1O晶
Exhausti veEnumcraLi ontechni quc. 326- 329
al gori thm. 327
appl i cal i on. 327-329
Exteri or Penal tyFuncti on(EPF) method ,
271- 276
al gori thm. 272-274
applicll lI on. 274- 276
Fcasi b1edomai n. 23
Feasi b1esolutions. 17
First-order co目 ditions (FOC). 176-178
First-orderllincar variation. 169
Fl agpo1eprob1em. 307- 310
FOC. seeFirst-order condi ti ons
Formu1ati onof prob1em, seeProb1em
fonn叫8!ti on
FORTRAN. 24. 227
Functi on(s):
constrai nt , 6. 7
obj ecti vc,6
of one variuble. 162-164. 169-170
of t wov副 ab1cs.56- 63.164- 171
GA. seeGenctic A1gori thm
Guuss-J ordanc1i mi nati on. 107
General i zedReduecdGmdi cnt ( GRG)
me曲。d, 297- 302
a1gori thm. 299- 300
upplicalion. 3α ト 302
stepsize. ca1cu1alionof. 299
GenericGeneti c A1gori 出 m( GGA) .
367- 370
Gcncti c A1gori l hm(GA). 350. 366- 377
gcneri c fonn. 367- 370
suggcsl edextensi ons 10. 374. 375
G10bal opti mi zati on. 350- 377
definitionofprobl cm. 351- 357
Geneti c A1gori l hm(GA) for 出 ei n.
366- 377
mi ni mum, global. 351- 353
numcri cal techni qucs for. 356- 358
Si mul utedAnneal i ng(SA). 358- 366
solution. global. 354- 356
Gol densecti onmethod. 214- 217. 219-220.
223- 225
Gmphi ca1opti mi za由 民 16- 18. 45-92
definitionof probl em. 45- 48
nndformat for gmphi cal display. 47- 48
functi onof two variables. 56- 63
nndgraphi ca1user interface. 81-91
wi 白 Hnndl cGni phi cs, 79-81
恥 at transfer examp1e. 73- 79
solution. 伊 lphi 伺 1. 48-56
structural engi neeri ngdesi gnexamp1e.
Graphi cal solutions:
linear progrnmmi ng(LP) prob1ems.
¥07- 1¥5
nonl i near progrnmmi ng(NLP) prob1ems.
one-di mensi onal probl cms. 205- 206
Gmphi cal User Interface(GUI ). 24. 48. 49,
79. 81- 91
GRG method , seeGencral i zedReduced
Gmdi ent method
GUI . seeGmphi cl l 1User Interfacc
Handl e Gmphi cs. 79-81
HnndJ cs. 49
Heat 釘 ansfer. 73- 79
In伺 ualitycons回 ints.8
Infinitesol ul i ons, 17
lnitial d民 i gnsolutions. 21
Jnl eger progr佃 l mi ng. 20
IlI teger variab1cs. 20
llcrativetecbni qucs. 21
Kuhn-Tucker condi ti ons. 192-193.
196- 200,267- 269
Lagrnngemu1tipliers. 180-192
LAMBDA pr ogmm(Op白 川 Z8ti onTI∞1box).
383- 384
I NOEX 395
Leavi ngBasi c Vari ab1e(LBV). 118
Li neard叩宮 ndence,¥01
Li near progmmmi ng (LP) prob1cms. 23-24.
93- 153
cons住凶 nts in, 98
del ermi nants 泊 . ¥02- 104
dual prob1emassoci a包 dwi 白. ¥38- 148
cqual i tyconsb冨 ints,exnmpl ei nvol vl ng.
130- 134
cx師 .plcs of. 11ι 111, 124- 138.
385- 386
four-vari abl eprob1cms, 134- 138
graphi cal Sol ul i ons of. ¥07- 115
i nfmi tesol uti onof. 114
MATLAB, sol uti onusi ng, 120-124
modcl i ngissucs in,98- 107
negati ved邸 ignvari abl cs in. 98
NLP probl ems vs.. 20, 93- 94
nosol 凶 onfor. 114- 115
pri m叫 probl em. 138
sensitivitynnal ysi s wi 山 . 148-151
Si mpl cx methodforsol vi ng, 115-124
stllndardformat for. 94- 98
transponal i onprob1emex剖 np1e.
124- 130
unboundedsol uti onof, 114
uni quesol uti onof. 114
山首 tvωtors i民¥06- 107
Mni nf mme computi ngsysl ems. 24
Mul druw. 47
Mat hem8!ti cal mode. 5
Mat hem8!ti cnl mode1s/model i ng. 3. 10-16
MathWorks Inc. , 2s
MATLAB. 24- 44
codesnippet. creal i nga. 37- 40
edilor in. 33- 37
日間 l-limeuseof. 27. 29- 33
bi gh-Jeve1gr冨 phi cs functi ons of. 48- 50
insl a11ationof. 26- 28
operuωrs in. 30-31
Opti mi zati onT, ∞1boxof. 123-124.
379- 391
progrnm. c陀 ati nga , 40- 44
Si mp1ex Method using. 120-124
Symbol i c Math T, ∞1boxof. 159-161
usefu1ness of, 26
Matri cω:
augmenl ω. 103- 104
396 I NDEX
MBI I I αs (continue.の
rankof, 104- 106
Mu且 i mumprobl ems, 6
M- fi1ω,27
Mi ni mi zati onprobl ems, 6
Modi fi edNewton me曲。 d,252- 253
Mu1ti di sci pl i narydesi gns, 6
Mu1ti obj ccti vedωi gns, 6
n-di mensi onal Eucl i dennspaα : 5 , 106
Newton- Raphson techni que, 206- 209, 252
Non1i ncar programmi ng ( NLP) prob1cms,
21- 23, 154-2' 叩
W1a1yti ω1condi ti ons in, 175-176
cons町aints in, 157-159, 172-175,
examp1es of. 194-2∞
firsl -ordcr ∞ndi ti ons in, 176-178
formu1ati onof probl em, 155-157
graphi ca1sol uti ons of, 171-175
Kuhn- Tuckcr condi ti ons for, 192-193,
196- 2∞
Lagrangc mul ti pl i er mcthod, useof,
180- 192
LP prob1ems VS. , 20, 93- 94
one vari abl e, functi ons of, 162-164,
169- 170
mωnd・0吋 ercondi ti ons in, 178-179
symbol i c computoti on, 159-161
Tay10r scries for useinsol vi ng, 169-171
t woor mo陀 varillblcs,functi ons of,
164- 171
Obj 民 tivcfuncti ons, 6
0吋cct ori entcdprogram回 目 g,24
One-di mcnsi onal probl ems, 203- 225
bi sccti ontccbni quefor usewi 血, 209- 210
constr瓜 ned,204-205
dcfinitionof probl em, 204
gol densccti onmcthodfor usewi th,
214- 217,219- 220,223- 225
g匂 Mα1sol uti onof, 205- 206
i mportanceof, 217- 219
Newton- Rapbson tcc加岨 quefor usewi th,
206- 209
pol ynomi a1approxi mBti onwi th, 211- 214
two-poi nt boundary val ueprobl em,
220- 223
Operators, 30- 31
句 timal solu脚 色 17
Opti mi zati on,l - 4
Opti mi zati onTool box, 123-124, 379-391
constrai ncdopti mi l.ationusi ng, 389-391
exampl es usi ng, 385- 386
1inear progmmmi ng exampl c usi ng,
385- 386
opti mi zati onparameters, setting, 384- 385
qua也 温 icprogr四 uni ngcxamp1e usi ng,
386- 388
unconstr百 i nedopti mi zati onusi ng,
388- 389
usi ngprogmms in, 382- 384
OUTPUT program(Opti mi zati onToo1box) ,
Parameほお, d弱 ign , 5
Partial derivativ凶 , 1ω
Pascnl ,24
Pntl emScarchmetbod, 234- 238
Po1ynol1u叫阻 pproxi mati on,211- 214, 217
Powel l ' s mctbod, 238- 240
a1gori 山 m,238- 239
appl i cati on, 239- 240
andconj ugacy, 240
Pri mal probl em, 138
Probl cmdefi 副 onandfiロrmu1ati on,4- 10_
SeealsoMatbemati cal model sJ
model i ng
co目S甘凶 nedopti mi zati onprobl ems,
266- 267
∞nstr泊 nt functi ons in, 7
desi gnfuncti ons in, 5- 6
dωi 伊 par百 neters in, 5
anddesi gnspaα ,9
desi gnvnriab1es in, 4- 5
句 叫 ity∞nstr泊 目 白 i民 7- 8
gl obal opti mi i ロn,351- 357
gruphi cu1opti mi zati on, 45- 48
i nequal i tyconstraints in, 8
nonl i near probl ems, 21.,. 23
目白目I i n伺 r programmi ng(NLP) prob1ems,
obj ccuvefuncti onin, 6
one-di mcnsi onal prob1em, 204- 205
side∞ns甘 aints in, 8
standardformat for, 9- 10
unconstmi ncdopti mi zati onprobl ems,
227- 230
Prob1emrclaxation , 321- 322
Probl emsol uti ons, 16- 24
Programmi ng, discrcte, 20
針。 grammi ng,intcg白, 20
Programmi ng l anguagω,24
Programmi ng pamdi gms, 24
Pseudornndomnumbers, 230
Quadrati c convergence, 238
Qua命 aticpo1ynomi al s, 238
Qua世 aticprogr副 加 凶 ng,386- 388
RundomWal k method, 230- 233, 254- 255
Rosenbrock probl em , 253- 255
Rota也 Igdi sk, thrce-di mensi onal 110wnear,
255- 258
SA, seeSi mul atedAnnenl i ng
Script m- fiI es, 27
S回 開 ha1gori 出 m,218
Searchdircction, 22
Searchme曲 ods,21
Second-order condi ti ons (SOC) , 178- 179
ScωI I d-urderl l l near variation, 169
Sensitivityan叫 ysis,148-151
Sequenti al dcci si onprobl ems, 336
Sequenti nl Gradi ent Restorati ona1gori tbm
( SGRA) , 302- 307
nl gori tbm, 305
appli 回 出 n,305- 3σ7
gradi ent phnse, 303- 304
restorationphase, 304- 305
Sequenti al Li near Progr沼 町 凶 ng(SLP) ,
284- 289
叫 gori 山 m,285- 286
appl i cati on, 286- 289
Sequenti al Qua世 atlcProgmmmi ng (SQP) ,
289- 296
a1gori thm, 294
appl i cati on, 295- 296
and BFGS method , 293- 294
multipliers, cal cul ati onof, 293
stepsize, ca1cul ati onof, 292- 293
Sequenti al Unconstrni ncdMi ni mi zati on
Tcchni ques ( SUM' η,270- 271
SGRA, seeSequentl ul Omdi cnt Restorati on
aJ gori tbm
Si deconstrai nts, 8
Si mpl cx method, 115- 124
appl i cati onof, 117- 120
I NDEX 397
Enteri ngBasi c Vari abl e(EBV), 118
features of, 115- 117
Leavi ngBasi c Vari ub1c ( LBV), 118
MATI-AB, sol uti onusi ng, 120-124
t wo- phase, 125-130
Si mul l l tedAnneal i ng(SA), 350, 358- 366
a1temati vc a1gori 血 m,359- 365
basi cal gori thmof, 358- 359
∞nstraints, hand1i ng, 364
indlscrcteprobl cms, 365- 366
Sl ackvariab1es, 95
SLP, seeSequenti al Li ncar 町' Ogrammi ng
SOC, seeSccond・ordcr condi ti ons
Softw町 民 24
Sol uti ons, 16- 24
acceptobl c, 17
infinite, 17
initial desi gn, 21
opti m叫, 17
SQP, seeSequenti al Quadrnti c Programmi ng
Stnndardformal , 9- 10
Stecpest Dωcent me曲。 d,241- 244
SUMT, J-eeScquenti 凶 Unconstrai ned
Mi ni mi zati onTcchni ques
Symbo1i c Math T, ロ ol box(MATLAB) ,
Tayl or scric唱, 169-171
Tol 町 ance,209
TPBVP, secT wo- poi nt boundary yal ue
probl em
Trans戸 市 山 nprobl cmexampl e, 124- 130
Twcト poi nt boundary va1ueprobl cm
σ'PBVP) ,220- 223
Unconstrai nedoplimizlltionprobl ems,
227- 262 , 388- 389
Bczi cr paramctri ccurves, fittlng,
258- 262
Broydon- F1etcher-Go1dfarl:ト Shanno
methodfor sol vi ng, 249- 251
Co吋ugateGradi ent mctbodfor sol vi ng,
244- 246
Davi don- F1etchcr-Powcl l mel hodfor
sol vi ng,24ι 249
definitionof prob1em, 227- 230
generi c a1gori tbmfor, 229- 230
396 I NDEX
Mal ri 臨 (conti nued)
rankof,l04- 106
M脱 出 umprobl cms. 6
M-fi l ω,27
Mi ni mi zati onprobl cms, 6
Modi f i ωNewt on met hod, 252- 253
Mul l i di sci pl i nnrydesi gns, 6
Mul ti obj ecl i vedesi gns, 6
n-di mensi onal Eucl i deanspaces, 106
Newl On- Rapbso目 tech副 quc,206- 2ω,252
Nonl i near programmi ng ( NLP)   ,
21- 23, 154- 200
anal yti cal condI ti ons in, 175- 176
constrllInts in, 157- 159, 172- 175,
179- 191
cxampl 隠。 ι 194- 200
目指 1- 0凶 er condi l i ons in, 176- 178
伽 mul ati onof probl ern, 155- 157
graphi cal sol uti ons of, 171- 175
Kuhn・Tucker condi ti ons for, 192- 193,
19ふ 2∞
Lagmnge mul l I pl i er mel hod, useof,
180- 192
L P probl ems vs. , 20, 93- 94
one vari nbl e. funcl I ons of. 162- 1 64.
169- 170
second-order condI l i ons in, 178- 179
syrnbol i c computa凶on,159- 161
Tl1yl orseri 鎚 for useinsol vi ng. 169- 171
t wo or mor o varI abl es, funcl i ons of,
164- 171
Obj ecti vefuncti ons, 6
0尚氏 l ori cn凶 programmi ng,24
One-dI mensI onal probl cms, 203- 225
bi sccti ontechni quefor usewI th, 209- 210
COI 首 位 画 ned,204- 205
defi ni ti onof probl cm, 204
gol d叩 sccti onmet hod for usewi th.
214- 217,219- 220,223- 225
grnphi cal sol uti onof, 205 - 206
i mportanceof, 217- 219
Nc wl On-Raphson l echni quefor usewi th,
206- 209
pol ynomi al approxi mati onWi l h, 211- 214
two-poI nt boundnry val uc probl em,
220- 223
Opeml o路, 30- 31
Opti mal sol uti ons, 17
句 伽 凶 i on,l - 4
Opti mi zati onT, ωl box, 123- 124,379- 391
consl mI ncdoptI mi znl i onusi ng, 389- 391
exampl es usi ng, 385- 386
l i near prograrmni ngex田 npl eusI ng,
385- 386
opti mi zal i onp町耳 me股 rs,seui ng, 384・ ・ 385
qua企冨 .tic programmi ng cxampl e usi ng,
386- 388
un∞田 町 司 i nedopti mi zati onus泊 E・
388- 389
usi ngprograms In, 382- 384
0げ r pUT pr ogmm(Opti mi zati onT, ∞l box).
Parameters, desi gn, 5
Pani al deri vati ves, 164
Pascal ,24
Pat t cmS叫 rchmcthod, 234- 238
Pol ynomi al opproxi mati on, 211- 214. 217
Powell' s me山 od,238- 240
a1gori 白 m,238- 239
app1同 町 n. 239- 240
and conj ugacy, 240
Pri mal probl em, 138
Probl emdefi ni ti onand f ormu1ati on, 4- 10.
Seeal soMal hemati cal mode1s1
model i ng
cons町 話 回 dopl i mi zati oDprobl ems,
266- 267
∞nsuai nt functi ons in, 7
desi gnfuncti ons in, 5- 6
desi gnparamete四 in,5
and desi gnspace, 9
desi gnvnri abl cs in, 4- 5
equal i tyconsuai nts in, 7- 8
gl obal ロ , pt同 比 ati on,351- 357
graphi cal opl i mi zati on, 45- 48
i nequa1i ty∞nsuai nts in, 8
nonl i near probl ems, 21ヲ 23
nonl i near programmi ng ( NLP) probl ems,
155- 159
0同氏 tivefuncti oDin, 6
one-di mensi onal probl em, 204- 205
si deconstmi nts in, 8
standordformnt for, 9- 10
uncons回 i ncdopti mi zati onprobl ems,
227- 230
Probl emrel 以 ati on,321- 322
Probl emsol uti ons, 16ー24
Pr ogmmmi ng, di sc陀 te,20
Programmi ng, i ntegcr, 20
PrognunoUngl anguages,24
PrognunoUng paradi gms, 24
Pseudorandomnumbers, 230
Quadrati cconvergenω,238
Quadrati c pol ynomi al s, 238
Quadmti c programmi ng, 386- 388
RandomWa1k method, 230- 233, 254- 255
R吋 undancy,101
R05cnbrock probl em, 253- 255
Rotati ngdi sk, thr官 c-di mensi onal f l ownear,
お 5- 258
SA, .rtt Si mul atedAnneal i ng
Scri pt m-fi l cs, 27
Searcha1gori thm , 218
S側 同 hdi 悶 tion,22
S阻 rchmethods, 21
Second-ordcr condi ti ons ( SOC) , 178- 179
Second-or枇 r/l i near vari nti on, 169
Sensi ti vi l yannl ysi s, 148- 151
S句 uenti al d民 i SI ODprobl ems, 336
Sequcnti al Gradi ent Restorati ona1gori thm
( SGRA) , 302- 307
a1gori 白 m,305
appl i cati on, 305- 307
gradi ent phωe,303- 304
restorati ODphase. 304- 305
Sequenti al Li near Programmi ng (SLP) ,
al gori thm, 285- 286
app1i cati on, 286- 289
S巴quenti al Quadrati cProgrammi ng ( SQP) ,
289- 296
al gori 山 m. 294
appl i cnti on, 295- 296
andBFGSme山od, 293- 294
mu1ti pl i ers,ωI 印 刷 i onof,293
stepsi ze, cal cul ati onof, 292- 293
Sequenti al Unconsuai ncd恥鉛 I I mi 7.otion
Techni ques (SUMT>, 270- 271
SGRA, soe Scqucnti aJ Gradi enl Res[orati on
n1gロri thm
Si dc∞ns凶 nts,8
Si mpl ex met hod, 115- 124
叩 pl i cati onof, 117- 120
I NDEX 397
Enteri ngBasi c Vari abl e( EBV) , 118
features of, 115- 117
Leavi ng8asi c Vari abl e( LBV), 118
MATl AB, sol ul i onusi ng, 120- 124
two- phasc, 125- 130
Si mul 回 edAnneal i ng( SA), 350, 358- 366
a1temati veal gori thm, 359- 365
basi ca1gori thmof, 358- 359
∞n5U司in也, h邸 ldJ i ng,364
indi screteprobl ems, 365- 366
Sl ack vari abl es, 95
SLP, seeSequenl i al Li near Programmi ng
SOC, sl!e Sccond-order condi ti ons
Sol uti ons, 16- 24
a∞eptab1e,I 7
infinite, 17
inilial desi gn, 21
opti mal ,17
SQP, seeSequenti al Quudrnti c Prograrnmi ng
Standardfonnnt, 9- 10
St白 戸 st O飽 αnt met hod, 241- 244
SUMT,seeS句 uenti al Unconstmi ned
Ml nI mi zati onTechni ques
Sym加 l i cMat hT, ∞l box( MATl AB) ,
159- 161
Tayl or scri es, 169- 171
To1erunce, 209
TPBVP,s館 T wo・poi nt bound町 y va1ue
probl em
Transpon瓜 i onprob1emcx組司 p1e,124- 130
Twcト poi nt boundnry val ueprobl em
σ' PBVP) ,220- 223
Un∞n5U百 i nedopti mi zati onprobl ems,
227- 262,388- 389
8ezi cr pnrametri c cu円 ω,fitting,
258- 262
8roydon-Fl etcher-Gol dfartト Shanno
met hod for 50l vi ng, 249- 251
Conj ugateGradi ent mel hod for 801vi ng,
244- 246
Oavi don- F1cl cher-PowclI mel hod for
sol vi ng,24ι 249
defl DI tionof probl em, 227- 230
generi c a1gori thmfor, 229- 230
E留 置 箇 固
398 I NDEX
Uncorul l rai nedoplimiz.瓜 i onprobl ems
(conル拙 eの
g 百 di ent-based町 l ethods for sol vi ng,
241- 251
modi fi edNeWl On method for sol vi ng,
252- 253
neces姐 ry Il Dd sufficient condi ti ons for,
228- 229
nongradi ent mel hods for sol vi ng,
PattemS朗 氏 hmethod for sol vi ng,
234- 238
PowcU' s mel hod for sol vi ng, 238- 240
randomwa1ka1gori l hmfor, 230- 233
Rosenbrock probl em, 253- 255
rotatingdi sk, thrce-di mensi onal sow
目 国 r,255-お S
second-order mel hods for sol vi ng,
251ーお 3
Sl eepcst Dcscent mcl hod for sol vi ng,
241- 244
Uni que opti rna1solutions. 17
Uni tv出 ors,106- 107
VarillbleMetri c Methods ( VMM) , 246
Vari nbl es:
nrtifici H l , 98
conti nuous, 20
desi 伊,4- 5
discrel e , 20
i nteg民 20
sl ack,95
desi gn, 5
uni t , l O岳ー 107
V MM (Vari abl eMetri c Mel hods) , 246
Zero- Onel n崎 町 P r ogramrni ng(ZI P) ,
343- 348
Zero- order methods, 230
ZI P, seeZero- One l nl eger Programmi ng