Applied Optimization With MATLAB Programming

MAn. AB is a regisleredl rademarkof leMal hWorks Inc.
Thi s book is p ledonncid paper . @

Copyri ghl @22 by J ohnWi l ey & Sons NewYork. AII righls ed.
Publ i shedsi mul taneousl yinCnnadn.
No pnrt of Ihis publiclllionmay be reprodu dstoredina triev systemor 1 nsmi ttedinnny formor
byanym nsel ecl roni c mechani caJ phol ocopyi ng cordi ngscanni ngor otherwi se except as
permi ttcdunder Secti ons 107or 108ofthe 1976Uni ted Stalcs Copyri ghl Acr wi rhoul either theprior
wri ttcnp rmi ssi onof IhePubl i sher or outhori 7.3lionthroughpaymenl of theappropri ul eper-copyfcc 10
IheCopyri ghl Cl earanceCenl er 222Ro WdDri ve Dnnver MA01923 (978) 7508400fnx(978)
7504744. Requ IS10IhePublisher for permi ssi onshoul dbe nddressed10thePeml i ssi ons Departmenl
J ohnWi l cy & Sons Inc. 605 irdAvenue NewYork NY 10158-12 (212) 850- 11 (212)
850- 6008 E-Mai l : PERMREQ@WI LEY. COM.
spubl i cal i onis desi gned10provi deaccu lenndauthori l al i vei nfomtal i onin E 10 esubj ecl
mattercove d. It is 80l dwi ththeundersl undi ngIhal thepubl i sher is nol engagedinrend ng
professi onal services. If professi onnl ndvi ceor othcr expert assi sl anceis qui red eservi ces of a
compel ent professi onal pe nashoul dbesoughl.
Ubrary ofCongress Catal og; lI g; n P"bl;catiotl Data:
Venkol urom nP.
Appl i edopl i mi Zl l lionwi th Programmi ngI P. Venkal ar mlln.

p. com.
A Wi l ey-I ntersci encc publ i cnl i on."
I SBN0-471 958.5(c1oth: alk. pnper)
1. Mnl hemal i caJ opti mi zati on--Dol l l proc ing2. MATLAB. J. Title.
QA402. 5. V42 2001
519. 3- dc21
Pri nl edinIheUni l edSlales of Ameri cn.
10987654321
2001026938

T 0 coexi stence of peace and har mony bet ween the di fferent
mel odi es of peopl e ever ywher e
6
CONTENTS
PREFACE
1 I ntroductl on
1.1 Opti mi zati onFundamental s I 2
1.1.1 El ements of Probl emFormul ati on I 4
1.1. 2 Mathemati cal Model i ng I 10
1.1. 3 Nature of Sol uti on I 16
1.1.4 Characteri sti cs of theSearchProcedure I 20
1.2 I ntroducti ontoMA11..As I 25
1. 2.1 Why MATLAB? I 25
1. 2.2 MATLAB Instal l ati onI ssues I 26
1. 2.3 Usi ng MATLAB theFirst Ti me I 27
1.2.4 Usi ng theEdi tor / 33
1.2.5 Creati nga Code Sni ppet / 37
1. 2.6 Creati nga ProgramI 40
Probl ems I 44
xiii
1
2 Graphl cal Opt l mi zat l on 45
2.1 bl emDefi ni ti on I 45
2. 1.1 Exampl e 2.1 I 46
2. 1. 2 Format for l heGraphi cal Di spl ay I 47
2.2 Graphi cal Sol uti on I 48
2.2.1 MATLAB Hi gh- Level Graphi cs Functi ons I 48
2.2.2 Exampl e 2. I - Graphi cal Sol uti on I 50
2.2.3 Di spl ayi ngtheGraphi cs I 53
2.2.4 Customi zi ng theFi gure I 54
2.3 Addi ti onal Exampl es I 56
vll
vlll CONTENTS
2.3.1 Exampl e 2.2 I 56
2.3.2 Exampl e 2.3 I 64
2.3.3 Exampl e 2.4 I 73
2.4 Addi ti onal MA11..AB Graphi cs I 79
2.4.1 Handl e Graphi cs I 80
2.4.2 Graphi cal User I nterface I 81
2.4.3 GUI Code I 84
References I 91
Probl ems I 92
3 Ll near Pr ogr amml ng
3.1 Probl emDefi ni ti on I 94
3.1.1 StandardFormat / 94
3.1.2 Model i ng I ssues / 98
3.2 Graphi cal Sol uti on / 107
3.2.1 Exampl e 3.1 / 110
3.2.2 Characteri sti cs of theSol uti on / I I 1
3.2.3 Di fferent Sol uti onTypes I 114
3.3 Numeri cal Sol uti on- the Si mpl ex Met hod I 115
3.3.1 Features of theSi mpl ex Met hod / 115
3.3.2 Appl i cati onof Si mpl ex Met hod / 117
3.3.3 Sol uti onUsi ng MA11..AB / 120
3.3.4 Sol uti onUsi ng MATLAB' S Opti mi zati onTool box I 123
3.4 Addi ti onal Exampl es / 124
3.4.1 Exampl e 3. 2-Transportati on Probl em/ 124
3.4.2 Exampl e 3. 3- EqualityConstrai nts and Unres cted
Vari abl es / 130
3.4.3 Exampl e 3.4- A Four-Vari abl e Probl em/ 134
3.5 Addi ti onal Topi cs inLi near Programmi ng / 138
3.5.1 Pri mal and Dual Probl em/ 138
3.5.2 Sensi ti vi tyAnal ysi s / 148
Refer nces / 151
Probl ems / 152
4 Nonl i near Pr ogr ammi ng
93
154
CONTENTS Ix
4.1.1 Probl emFormul at i on- Exampl e 4.1 / 155
4.1.2 Di scussi onof Cons aints / 157
4.2 Mathemati cal Concepts / 159
4.2.1 Symbol i c Computati on Usi ng MATLAB / 159
4.2.2 a ~ i c Mathemati cal Concepts I 162
4.2.3 Tayl or' s Theorem/Seri es / 169
4.3 Graphi ca1Sol uti ons / 171
4.3.1 Unconstrai nedProbl em/ 171
4.3.2 Equal i ty Constrai nedProbl em/ 172
4.3.3 I nequal i tyConstrai nedProbl em/ 173
4.3.4 Equal i ty and I nequal i tyConstrai nts / 174
4.4 Anal yti ca1Condi ti ons / 175
4.4.1 Unconstrai ned Probl em/ 176
4.4.2 EqualityCons i nedProbl emI 179
4.4.3 I nequal i tyConstrai nedOpti mi zati on I 186
4.4.4 General Opti mi zati onProbl em/ 191
4.5 Ex npl es / 194
4.5.1 Exampl e4. 2/ 194
4.5.2 Exampl e 4.3 / 196
References / 200
Probl ems / 201
5 Numeri cal Technl ques-The One- Dl mensl onal Pr obl em
5. 1.1 Constrai nedOne- Di mensi onal Probl em/ 204
5.2 Sol uti ontotheProbl em/ 205
5.2.1 Graphi cal Sol uti on / 205
5.2.2 Newton- Raphson Techni que / 206
5.2.3 Bi secti onTechni que I 209
5.2.4 Pol ynomi al Approxi mati on / 211
5.2.5 Gol denSecti onMet hod I 214
5.3 I mportance of theOne- Di mensi onal Probl em/ 217
203
5.4.1 ampl e5. 2-Il1us tionof Geneml Gol denSecti onMethod / 219
5.4.2 Exampl e 5.3 T wo- Poi nt Boundary Val ue Probl emI 220
5.4.3 Exampl e 5. 4- Root Fi ndi ngwi thGol den Secti on I 223
X CONTENTS
References I 225
Probl ems I 225
6 Numerl cal Technl ques f or Unconst r al ned Optl ml zatl on
6. 1.1 Exampl e 6.1 I 228
6.1.2 Necessary and Suffi ci ent Condi ti ons I 228
6.1.3 El ements of a Numeri ca1Techni que I 229
6.2 Numeri cal Techni ques- Nongradi ent Methods I 230
6.2.1 RandomWa1k I 230
6.2.2 PattemSearch / 234
6.2.3 PoweJ1' s Met hod I 238
6.3 Numeri cal T hni ques- Gradi ent- BasedMethods I 241
'6.3.1 Steepest Descent Met hod I 241
6.3.2 Conj ugate Gradi ent (F1etcher-Reeves) Met hod I 244
6.3.3 Davi don-FJ etcher-PowelI Met hod I 246
6.3.4 Broydon- Fl etcher-Gol dfart ShannoMet hod I 249
6.4 Numeri cal Techni ques- Second Order I 251
6.5.1 Exampl e 6. 2- Rosenbrock Probl em/ 253
227
6.5.2 Exampl e 6. 3-Three-Di mensi onal Aow n r aRo ngDi sk I 255
6.5.3 ExampJ e 6.4- Fi tti ng Bezi er Parametri c Curvcs I 258
References I 262
Probl ems I 263
7 Numerl cal Techni ques f or Const ral ned Optl ml zatl on 265
7. 1.1 Probl emFormul ati on- Exampl e 7.1 I 266
7. 1. 2 Necessary Condi ti ons / 267
7.1.3 El ements of a Numeri cal Techni que I 269
7.2 Indi rect Methods for Constrai nedOpti mi zati on I 270
7.2.1 Exteri or Penal ty Functi on( EPF) Me od I 271
7.2.2 Augment ed Lagrange Mul ti pl i er ( ALM) Met hod I 276
7.3 Di rect Methods for Cons ai nedOpti mi zati on I 281
7.3.1 Sequenti a1Li near Programmi ng (SLP) I 284
7.3.2 Sequenti al Quadrati c Programmi ng (SQP) I 289
CONTENTS xl
7.3.3 Genera1i zedReduced Gradi ent ( GRG) Met hod I 297
7.3.4 S uenti al Gradi ent Restorati onAJ gori thm( SGRA) I 302
7.4 Addi ti ona1ExampJ es I 307
7.4.1 ExampJ e 7. 2- FJ agpoJ e Probl emI 307
7.4.2 ExampJ e 7. 3- BeamDesi gn I 310
7.4.3 ExampJ e 7.4 Opt i ma1Control I 313
References I 316
Probl ems I 316
8 Dl screte Optl ml zati on
8.1 Concepts inDi screteProgrammi ng / 320
8. 1.1 Probl emRel axati on I 321
8. 1. 2 Di screteOpti mal Sol uti on I 322
8.2 Di screteOpti mi zati onTechni ques I 324
8.2.1 Exhausti ve Enumerati on I 326
8.2.2 Branch and Bound I 329
8.2.3 Dynami c Programmi ng / 336
8.3.1 Exnmpl e 8.4--1BeumDesi gn I 341
8.3.2 Zero- One I nteger Programmi ng I 343
References I 348
Probl ems I 348
9 Gl obal Optl ml zati on
9. 1.1 Gl obal Mi ni mum/ 351
9.1.2 Nature of theSol uti on I 354
9. 1. 3 El ements of a Numeri ca1Techni que I 356
9.2 Numeri cal T hni qusand Addi ti onal Exampl es I 357
9.2.1 Si mul atedAnneal i ng (SA) I 358
9.2.2 Geneti c Al gori thm( GA) I 366
References I 377
Probl ems I 378
10 Optl mi zati on Tool box f r omMATl AB
10.1 The Opti mi zati onTool box I 380
318
350
379
xll CONTENTS
10. 1. 1 Programs I 380
10. 1. 2 Usi ng Programs I 382
10.1.3 Setti ngOpti mi zati onParameters I 384
10.2 Exampl es I 385
10.2.1 Li near Programmi ng I 385
10.2.2 Quadrati c Programmi ng I 386
10.2.3 Unconstrai ned Opti mi zati on I 388
10.2.4 Constrai nedOpti mi zati on I 389
Reference I 391
I ndex
393
PREFACE
les u ~ j e t of opti mi zati onis recei vi ngseri ous attenti on o mengi neers sci enti sts
managers and most everybody else. Thi s is dri venby competi on qual i tyassurance
cost of producti on and fi nal l y the success of the busi ness ente e. l gnori ng e
practi ceof opti mi zati onis not an opti onduri ngcurrent ti mes.
Opti mi zati on is practi ced through softwar programs and requi res si gni fi cant
computer resources. The techni ques of opti mi zati onhave not changed si gni fi cantl yin
cent years but tlle areas of appl i cati ons have mushr oomed at a si gni fi cant rate.
Successful l y embeddi ng theuse of opti mi zati on in professi onal practi cerequi res at
least threeP qui si tes. beyi ncl udemathemati cal model i ng of thedesi gnprobl em
knowl edge of computer programmi ng and knowl edge of opti mi zati on techni ques.
Many speci al -purpose opti mi zati on software packages rel ax requi red
knowl edge of programmi ng are avai l abl e today. To use t hemeffi ci entl y the
remai ni ng t wo areas still have tobe addressed.
l ere are several excel l ent books on the subj ect of opti mi zati on a f ewof them
rel easedrecentl y. Most of t hemcov rthesubj ect indepth. whi ch is necessary because
th mathemati cal model s arenonl i near and requi respeci al techni qusthat areusual l y
notpa of any corecurri cul um. Al l of ebooks assume ereader isfami l i ar wi l h
a programmi ng l anguage adi ti onal l yFORTRAN and recentl y C. Another assumpl i on
h quentl y made is the opti mi zati on chni ques wi1l be i mpl emented in a
mai nf rame computi ng envi ronmenl. Such a combi nati on is ex emel ydi ffi cu1t for the
sel f-I earner even wi thenormous moti vati on.
An expl osi onof i nexpensi ve desktopcomputi ng resources ai ds engi neeri ngdesi gn
practi today. Paral l el i ngthis devel opment is eavai l abi l i ty of extensi ve sou
on theI nternet bothfor l earni ngand depl oyment. This isj oi nedby theavai l abi l i tyof
sof systems that provi de opportuni tyfor handl i ngmathemati cs graphi cs. and
programmi ng ina consi stent manner. These softwaresystems aresi gni fi cant1y easy to
master compar to the hi gher-l evel programmi ng l anguages of thep vi ous ye S.
lis book pri mari l y seeks to harness this i angul ati on of s rvi ces to provi de a
practi cal approach to the study of desi gn opti mi zati on. le book uses MATI . AB@to
i11ustrateand i mpl ement thevari ous techni ques of opti mi zati on. MATLAB is a product
f romMat hWorks. Inc.
Thi s book attempts 10accompl i sh t wo i mportant obj tives. The fir
xl v PREFACE
of symbol i c. numeri cal . and graphi cal f l tures of MATLABlt i ntegrates is powerful
combi nati on duri ng nsl ati on of many al gori thms i nto app1i ed numeri cal
techni ques for desi gn opti mi zati on. There is a constant enhancement of the
programmi ng skills throughout th book. The second 0ecti veis th pri mary taskof
book- t ocommuni cate and demonstrate vari ous lI umeri cal techni ques that are
currenl usedintltearea ol opti mal des .
AH of thc numeri cal t echni quesMEsupport ed by MMLAB code avai l abl e
computer files. These files are avai l abl e on a compani on web site
www. wi l ev. coml venkat. l ti s necessary to visit theweb sitetodownl oad thc files to
fol l o w a l l of t h e E X tl I n p i es -Th e b oo k w i l l b e e ntial l yi n c ompl et.ewi tho 0u tthcsc files.
At the web site you will al so fi nd uscful l i nks 10other resources a web course on
MATLAB programmi ngand updated i nformati on about Appl i ed Opti nti muon wi th
MA Tl.AB @ Programmi ng.
The deci si onto l ocate thefiles on a web sitepmvi dEs an avenue toshadowthe
dynami c changes inMATLAs sof1wareitsclf whi ch is bci ng transformedin a maj or
way al most every ycar.l t provi des a mechani sm10correct bugs i ecode ina ti mel y
manner. It estabi shes an opportuni ty toi mprove codethroughreader suggcsti ons
and provi des a way for eauthor 10keep thc rcader engaged wi thnewdcvcl opments.
It docut down thesi zc of thebook enormousl y by l ocati ngall of thepag of code
outsi dethebook.
An unusual featureof thebook is ci ncl usi onof di screteopti mi zati onand gl obal
opti mi zati on. Tradi l i onal conti nuous desi gn opti mi zati on techni ques do take up a
si gni fi cant porti onof thebook. Conti nuous probl ems general l y al l owtheal gori thms
tobe mathemati cal l y devel oped ina convi nci ngmanncr and ei deas coul d be
core of ori gi nal a1gori thms dcvc10pcd by the readers to address their own speci al
necds. Every chaptcr i ncl udes addi ti onal nontri vi al cxamp1es at oftcn prescnt a
novel use or extcnsi onof thc opti mi zati ontechni ques inthechapter. The mathcmati cal
al gori thms and cxampl es ar accompani ed by MA' AB code avai l abl e at the
compani on web sue- Thenumeri cal techni ques usual l y i ncl ude strong graphi cal
support toil1ustrate copcrati onof thc techni ques.
Di screteandgl obal opti mi zati ontechni ques arebasedon verydi fferental gori thms
and .ch of t hcmcan easi l y j usti fy ful l -fl edged i ndependent courscs de tcd
excl usi vel y 10thcir S1udy. In a comprchensi vc book such
PREFACE X V
cl assroomsetti ngwi thou1computati onal cxperi ence is al so feasib1e. In is casethe
a1gori thms can be presentedand thenurneri cal resul ts illustrated. The seni or/gradua1e
s1udents in vari ous di sci p i nes. especi al l y engi nceri ng are the targc1 audi ence.
Opti mi zati on techni qucs are an i mportanl 1001 to sol ve dcsi gn probl ems in all
professi onal areas of study. Many i l l ustrati ons are o mthe area of mechani cal
engi neeri ngrefl ecti ng experi enceof eau or.
Independent l earnel' s parti cuI arl yprofessi onal s who need tounderstandthesubj ect
of opti mi zati on shoul dal so fi ndthebook very useful . A reader who has access to
MATLAB software c us the book to its best advantage. The book is l argel y
sel f-contai ned and devel ops all ncccssary mathemati cal concepts when ncedcd.
Abstract mathemati cal i dcas of opti mi ti on i ntroduccdgraphi cal l y and through
iIl ustrati veexampl es inthc book. Inmany i nstances theoperati onof thc numcri cal
techni que is atcdto showhow thedesi gnis changi ng wi thiterations. Thi s has
been possi bl ebecause MATLAB graphi cs areri chinfcatures and si mpl e toi nco oratc.
maki ng Ul cmcffccti ve for convcyi ng e i dcas. It is for this reason e reader is
cxposed to graphi cal programmi ng earl y on incI udi ng a chapter on graphi cal
opti mi zati on. To bal ance thc devel opment of programmi ng skills and th presentati on
of opti mi zati on concepts programmi ng i ssues are i ncremental l y expl ored and
i mpl emented. I nvestment of ti me in d vel opi ng programmi ng skills is e most
effecti ve way to i mbi be them. Thc author is nOl aware of shortcuts. Studcnts arc
expected to programi ndependentl y. make mi stakes debug errors. and i ncorporate
i mprovements as p t of their l earni ngexperi ence del i veredthrough is book.
The book proceeds at a bri sk pace toprovi de experi encc inMATLA8 programmi ng
and communi cate i deas inopti mi zati onina reasonabl e number of pages. This coul d
not be possi bl ewi thout taki ngadvantage of Iheaccompanyi ng web site10l ocatemuch
of the code for al gori thms and exampl es. Merel y runni ng the code will n01
si gni fi cantl y e cethereader' s programmi ng skills. It is i mportant for thestudent
tounderstandthecode also. To sist in is process thecode carri es Ii beral comment s
oftencorrespondi ngtotheal gori thmthat is bei ngi mpl emcnted. The au or expects
ereader tohave read ecomments. The accompanyi ng code al l ows thei nstructor
toassi gnboth programmi ng assi gnments as wel l as so
xvl PREFACE
approaches the i nstructor can choose to i ncl ude onl y a subset of conti nuous
opti mi zati on al gori thms to al l owthe i ncl usi on of ese popul ar techni ques. I f e
student has access totheOpti mi zati onTool box f romMATl AB enit canbe i ntegrated
i nto e course for handl i ng the final desi gn proj ect . The last chapter provi des an
overvi ewon theuse of thetool box.
Thi s book startedout usi ngMATI . AB Versi on5. About hal f way throughthebook
e transi ti on to MATl AB Versi on 5.3 was compl ete. At the compl eti on of the
m uscri pt MATLAB Versi on6 wshi ppi ng for some ti me. As of this tirne Versi on
6.1is avai l abl e. Most i nsti tuti onal MATl AB l i censes areusual l ybasedon subscri pti on
so they will have latest versi onof esoftware. Ina sense and is is trueof
all books that depend on parti cul ar softwaresystems thebook rnayappear out of date
wi threspect to esoftware. Thi s is not real l yso. Whi l e theusage of MATLAB functi ons
rnay change between versi ons thereshoul dbe sui tabl ewami ngs and i nstructi onabout
e di fferent usageAn i mportant i nsurance is built i nto the book parti cul arl y to
handl e such changes if they arise. Spri nkl edthroughout thebook is thehabi t of e
usi ng eonl i nehel p. Inmost i nstances is was pri mari l ydone toexpose thereader
toal ternati veuseof thesame command. Another si gni fi cant change themanuscri pt
had toendure was eloss of access toa UNI X versi onbecause of th uni versi ty' s
change to di stri butedcomputi ng on PC clusters. The author' s experi ence wi e
earl y chapters suggs18 that this shoul d not mat1er. Except for external files and
resources useof MATl AB as illustratedinthebook is i ndi fferent tothe vari ous
operati ngsy ( Wi ndowsMac UNI X or Li nux).
The topi cs chosen for i ncl usi onand thesi mpl i ci tyof presentati onof etopi cs are
di rectl y rel ated to the experi ence of teachi ng the course on opti mi zati on at the
seni or/graduatel evel in e Dpartment of Mechani cal Engi neeri ngfor over tenyears.
Experi ence proved at comprehensi on i rnprovedwi thsi mpl e iII ustrati ons however
compl i catedthemathemati cal i deas. On theother hand strai ghorwardmathernati cal
expressi ons elicitedno i nterest or understandi ng. The emphasi s on appl i cati on was
i mportant tokeep thestudents' attenti on. Inthis connecti ontheauthor woul d liketo
1kall hi s studen18for pushi ng hi rnfor si mpl e expl anati ons for provi di ngposi ti ve
i nteracti on and for thei r contri buti onof ori gi nal and useful i deas inthedi scuss
PREFACE xvii
d
l
l
p
d
L
Y
The book was made possi bl ethroughsupport f romJ ohn Wi l ey dSonsInc. d

Mat hWor ks Inc. Si ncere thanks are owed to Bob Argenti eri seni or edi tor at J obn
Wi l ey for accepti ng theproposal and who all al ongdi spl ayeda lot of pati encei n
getti ngthe book movi ng f orward. Same is due toBob Hi l bed-associ ate managi ng
edi tor at J ohn Wi l ey for his i mp ssi ve work at c1eani ngup the manuscri pt . Bri an
Snapp New Medi a edi tor at J ohn Wi l ey created compani on web site
(www. wi l ey. coml venkat) and w al so be rnai ntai ni ng it. Naomi Femandes o m
Mat hWor ks Inc. sawtoit at 1 had thelatest versi onof MATLAB as soon as it was
avai l abl e. My regardfor Dr. Angel o Mi el e Professor Emeri tus at Ri ceUni versi ty is
mor e thancan be expressedintheseIi nes. It was he who i ntroducedme to1h subj ecl
of opti mi zati on and demonstrated the effecti veness of si mpl e presenuti on-I will
al ways regardhi mas a great teacher. Of course my fami l y deserves speci al rnenti on
for putti ng up wi th all the Not now" Later" How about tornorrow?" d i n2
debuggi ng ecode. Sp ial ks are toA ana d Vi nayak rny offspri ng t i
their pati ence understandi ng and encouragement. The au or apol ogi zes for any
shortcorni ngs on the presentati on and wel comes comment s cri ti ci sms and
suggesti ons for i mprovement at all ti mes.
P. VENKATARAMhN
Rocheste NewYork
1- a . .
t
J
F
J
J
-
p
d
r
h
w
A
b
r
r
=
3
5
3
t
h
t
Y
4
y
I NTRODUCTI ON
Opti mi zati on has become a necessary part of desi gn acti vi ty in all maj or
di sci pl i nes. These di sci pl i nes arenot restri ctedtoengi neeri ng. The moti vati onto
produce economi cal l y rel evant products or servi ces wi thembedded qual i ty is the
pri nci pal reason for is i ncl usi on. I mproved producti onand desi gntool s wi a
synergi sti c thrust through i nexpensi ve computati onal resources have ai ded the
consi derati on of opti mi zati on methods in new devel opments parti cul arl y
engi neeri ng products. Even in the absencc of a tangi bl e product opti mi zati on
i deas provi de the ability to defi ne and expl ore probl ems whi l e focusi ng on
sol uti ons at subscri be tosome measure of useful ness. General l y theuse of the
wor d opti mi zati on i mpl i s thc best resul t under th ci rcumstances. Thi s i ncl udes
theparti cul ar set of constrai nts on thedevel opment rcsources current knowl edge
market condi ti ons and so on. Every one of us has probabl y used thc termat some
ti me to descri be e pri mary qual i ty of our wor k or endeavor. It is probabl y e
mosl used or abused l ermin adverti si ng and presentati ons. Neverthcl ess c
abi l i ty tomake thebest choi ce is a perpel ual desi reamong us al l.
Opti mi zati on is frequentl y associ ated wi th desi gn be it a product servi ce or
strategy. Aerospace desi gnwas among earliest di sci pl i nes toembrace opti mi zati on
ina si gni fi cant way dri venby a natura1need tol ower thetremendous cost associ ated
wi thcarryi ngunnecess ywei ght inaerospace vehi cl es. Mi ni mummass structures arl
norm. Opti mi zati onf orms part of thepsyche of every aerospacedesi gner. Savi ng
on fuel throughtrj ectorydesi gnwas another probl emthat suggesteditself. Very soon
enti engi neeri ngcommuni t y coul d recogni ze cneed todefi nesol uti ons based
on meri t. Recogni zi ng thedesi refor opti mi zati onand actua1lyi mpl ementi ng we t wo
di fferent issues.
2 I NTRODUCTI ON
Unti l recentl y. for much of the ti me. opti mi zati onwas usual l y attemptedonl y in
those si tuati ons where there were si gni fi cant penal ti es for generi c desi gns. The
appl i cati onof opti mi ti ondemanded l argecomputati onal resources. Inthe nascent
years of di gi tal computati on these were avai l abl eonl y to l arge nati onal l aboratori es
and research programs. These resources were necessary to handl e th nonl i near
probl ems at associ ated wi th engi neeri ng opti mi .tion. As a resul t of these
constrai nts most of the everyday products were desi gned wi thout regard to
opti mi zati on. This i ncl udes everythi ngyou s oundyou or use inyour dai l ylife. It
is i nconcei vabl e at the newgenerati on of repl acement products. likc the car. the
house. the desk. or epenci l . redesi gnedand manuf actured today arc not desi gned
opti mal l y inone sense or anothcr.
Today. you woul d def mi tIy expl ore procedures toopti mi ze your i nves l ents by
tai l ori ng your portfol i o. You woul d opti mi ze your busi ness travel ti me by
appropri a Iychoosi ng your dcsti nati ons. You can opti mi ze your commut i ng ti m by
choosi ng your ti me and route. You can opti mi ze your necessary expendi ture for Ii vi ng
by choosi ng your day and storefor shoppi ng. You can opti mi zc theuseful ti me you
connectto I ntemet by detcrmi ni ng your ti me of connecti on. You can buy software
wi l l opti mi ze your connccti on tothc l ntemct. You can buy books or readarticles
tcll you how to perf ormthese vari ous opti mi zati ons. The above acti vi ti cs
pri mari l y rel atc to :;crvIces or strategy. It is now apparcnt at every acti vi ty except
a S eticprovi des the scope for opti mi zati on. lis j usti fi es l ooki ng at estudy of
ooti mi z onas a tool canbe appl i ed10a vari ety of di sci pl i nes. If so themyri a
d
of o p ti m iz at io n prob le m ns s o 11 I
Thi sis eempha sisof thebook.
The partnershi p between d si gn and opti mi zati on acti vi ty is often f ound in
engi neeri ng. Thi s book recogni zes that connecti on and many of thc probl ems used for
i l l ustrati ons and practi ce are f romengi n ri ngpri mariI y mechani cal civil and
aerospace desi gn. Ncverthel ess the study of opti mi zati on. parti cul arl y appl i ed
opti mi zati on. i s not m excl usi ve pmpert y of my sptci nc di sci purl e.I t i nvol ves the
dECOVEr yanddesi gnof uti ons through appropri atetechni ques oci atedwi th the
formul ati on of the probl emin a sp ific manner. Thi s can be done for exampl e in
economi cs chemi stry and busi ness man
1.1 OPTI MI ZATI ON F UNDAMENTAL S
Opti mi zati on can be appl i ed to all di sci pl i ncs. Qual i tati vel y this asserti on i mpl i es
mul ti pl e deci si on choi ces; i mpl i ci tl y recogni zi ng the ncessi ty of choosi ng among
a1temati ves. Thisbooks deal s wi th opti rni zati onina quanti tati veway. Thi s means that
an out come of appl yi ng opti mi zati on to e probl em desi gn. or servi ce must yi el d
number at wi l l defi nethesol uti on. or inother words numbers or val ues wi l l
characteri ze theparti cul ar desi gnor servi ce. Quanti tati vedcscri pti onof l uti on
requi res a quanti tati ve descri pti on of theprobl emitself. This descri pti onis cal l eda
; : ~
".
"
1.1 OPTI MI ZATI ONFUNDAMENTALS 3
mathemati cal model . bedi . i charactcri zati on. d its ci rcumstances must be
expressedmathemati cal l y. Consi der thedesi gnacti vi ty inthefol l owi ngcases:
New consumcr researchi ndi catthat peopl e like10dri nk about 0.5litcr of soda
pop al a ti me duri ngthesummer months. The fabri cati onCOSI of theredesi gned
soda can is proporti onal to surfacearea. and can be esti matcdat $1. 00 pcr
square meter of themateri aI used. A ci rcul ar cross secti onis themost pl ausi bl e
gi ven current tool i ng avai l abl e for manufacture. For aestheti c reasons e
hei ght must be at l east twi ce thedi ametcr. Studi es i ndi cate at hol di ngcomf ort
requi res a di ameter between 6 and 9 cm.
A canti l ever beamnceds to be dcsi gnd tocarry a poi nt l oadF at thecnd of a
beamof l engthL. The cross secti onof thebeamwiII be intheshupe of th l tter
I ( edas I-b m). The beamshoul d meet prescri bedfai l urecriteria.
ereis al soa l i mi t on i defI tion. A beamof mi ni mummass is requi r to
be desi gned.
MyPC Company has deci dcd to i nvcst $12 mi l l i on in acqui ri ng several nW
Componenl Pl acem nl Machi nes to manuf acture di ffercnt ki nds of
mothcrboards for a ncwgcncrati on of personal compul ers. Three model s of
thcsc machi nes areunder consi derati on. Total numbcr of operators avai l abl eis
100 becausc of the l ocal l abor mar ket . A fI oor spacc constrai nt needs 10 bc
sati sfi ed because of the di ffcrent di mcnsi ons of thcse machi nes. Addi ti onal
i nformati onrcl ati ngtoeach of themachi nes is gi ven inTabl e 1.1. The company
wi shes to determi ne how many of each ki nd is appropri ate 10 maxi mi ze the
number of boards manuf actured per day.
The above list rcpr ents threeprobl ems at wiII be used 10defi nc formal el ements
of an opti mi zati onprobl em. Each probl em qui res i nformati onf romthespeci fi c arca
or di sci pl i ne to whi ch it refers. To rccogni ze or desi gntheseprobl ems assumes at
desi gncr is conversant wi eparti cul ar subj t matter. leprobl ems arekept
si mpl e tofocus on opti mi zati oni ssucs. Probl ems of such vari el yarei mportant today.
Recent adverti sements in general consumer magazi nes i1I ustrate at Al coa (an
al umi numm ufacl urer) is qui t happy to have rcducedLhc wei ght of the standard
soda pop can by over 30%inrecenl years. A si mi l ar c1ai mby thepl asti cs i ndustry wi th
respect tothestandardmi l k j ug (gaIl on) is al 50evi dent inthesemagazi nes al though
in is case thenumber is 409. Roof coUapses in cNortheast due 10excessi ve snow
Tabl e 1.1 Component Pl acement Machi r
Machi ne Operabl c
Model BoardTypes Boardsl Hour Opemt ors/Shift HoursIDay CostI Machi ne
A 10 55 18 400000
B 20 50 2 18 600000
C 18 50 2 21 700000
4 I NTRODUCTI ON
duri ng the 1998- 1999 wi nter will have structura1 desi gners and homebui l ders
expl ori ng second probl em. Th vi brant stock market in these ti mes has made
ba1anci ng the i nvestment portfol i o mor e chal l engi ng. Thc thirdcase may suggest a
mathemati ca1model appropri atefor such deci si onmaki ng.
1. 1. 1 El ement s of Pr obl emFor mul at l on
Inthis secti on we will in duce ef orma1el ements of thc opti mi zati onprobl em. In
this book the tcrmproduct a1so fers to a servi c or a strategy. It shoul d be
understood at op zati onpresupposes theknowl cdge of edesi gnrul es for the
sp ific probl em pri mari l y theability todescri bethedesi gninmathemati ca1terms.
These terms cJ ude desi gn vari abl es desi gn parameters and desi gnfuncti ons.
Tradi ti ona1desi gnpracti at is desi gnwi thout regardtoopti mi zati on incJ udes a11
of these el ements ugh ey were not formal l y recogni zed as such. Thi s al so
j usti fi es eprerequi si that you must be capabl e of dcsi gni ngthe0t if you are
pl anni ngtoappl y etechni qucs of opti mi tion. It is al 80a good i dea 10recogni zc
that opti mi zati onis a procedurc for scarchi ng1hebest desi gnamong candi datscach
of whi ch can produce an acceptabl eproduct . The oced for thc obj ec1or product is not
questi onedhere. but this may be due toa deci si onbased 00 opti mi zati onappl i edin
another di sci pl i nc.
Desl gn Varl abl es: Desi gn vari abl es areentities that i denti fya parti cul ar desi gn. In
esearchfor opti mal desi gn eseentities will change over a prcscri bedrange.
The val ues of a compl ete set of these vari abl es charactcri ze a speci fi c desi gn. Th
number and typeof cntities bcl ongi ng tothis set arevcry i mportant ini denti fyi ngand
setti ngup equ titativedesi gnprobl em. It is essenti al is choi ce captu e
essence of obj ect bei ng desi ed d at e samc ti me provi de a quanti tati ve
characteri zati onof thedesi gnprobl em. Inappl i cdmathemati cal termi nol ogy. desi gn
vari abl es serveas theunknowns of theprobl embei ngsol vcd. Borrowi ng an ana10gy
f romtheareaof systemdynami andcontrol theory ey areequi va1ent todefi ni ng
thestateof thesystcm in is case thc statcof desi gn. Typi cal l y desi gnvari abl es Can
be associ atcdwi thdescri bi ng obj ect' ssi zesuchas its l engthand hei ght. Inother
c esthey may represen1thenumber of i tems. The choi ce of desi gnvari abl es is the
responsi bi l i ty of edesi gner gui dcd by i ntui ti on expe seand knowl edge. le is
a f undamenta1requi rement tobe mct by is set of desi gnvari abl es namel y eymust
be I i nearl y i ndependent . This means a1you cannot cstabl i shth val ueof onc of e
desi gnvari abl es f rom eval ues of eremai ni ng vari abl es throughbasi c ari thmeti c
(scal i ngor addi ti on) operati ons. For exampl e ina desi gnhavi ng a rcctangul ar cross
secti on you cannot have evari abl es reprenti ng el cng wi dth and area. If the
first t wo ar prescri bed e thi rdis automati ca1l y establ i shed. Incompl ex desi gns
theserel ati onshi ps may not be very apparent . Neverthel ess thechoi of theset of
desi gn vari abl es must meet cri teri on of l i near i ndependence for appl yi ng the
techni ques of opti mi zati on. Fr oma practi cnl pcrspcc1i vc thc property of I i near
i ndependence i denti fi es a mi ni mumset of vari abl es that cun compl ctel y descri bethe
r::.-:
l
'"

desi gn. This is si gni fi cant because the effort in obtni ni ng the sol uti on vari es as an
i nteger power of thenumber of vari abl es and this power is 1ypi ca11y greater an2.
M tingtherequi rement ensures reduced di ffi cul ty inmathemati cal l y expl ori ngthc
sol uti on.
leset of desi gn vari abl es is i denti fi edas thedesi gn vector. 1I s v tor will be
consi der acol umn vector inthis book. Infac1 a11 vectors arecol umn vectors E
text. The l engthof is vector whi ch is n is thenumber of desi gnvari les io E
probl em. The desi gn vari abl es can cxpress di fferent di mensi onal qu tities in
probl em but inthemal hemati cal model they aredi sti ngui shedby charac1er x. AlI
of thetechni ques of opti mi zati oninthis book nrebascd on eabstract mathemati cal
model. lesubscri pt on x for cxampl c X3rcpresents thethirddesi gnvari abl e whi ch
may be thehei ght of an obj ec1in e ch ucteri zati onof theproduct. Thi s abstract
model is ne ssary for mathemati cal conveni ence. Thi s book will refer to edesi gn
vari abl es inone of fol l owi ngways:
(1) [ - r ef c i ngtothevcc10r 01 ' d si gnvari abl cs
(2) X or x- ref erri ng to thc vcctor agai n omi l l i ng 1hc square brackets for
convcni cncc if appropri ate
(3) [xj x2' ... XIl ] T_i ndi cati ng thc vccl or l hrough its el ements. Nol e e
superscri pt transposi ti onsymbol toi denti fy it as a col unmvector.
(4) X;. i = 1 2. . . . n- ref erri ng toall of theel ements of thedesi gnvctor.
Thc above notati ona1conveni ence is extcndcd toall vcctors inthebook.
Desl gn Paramet ers: In is book esei denti fyconstan 1wi l l not change
di erent desi gns are compared. Many 1exts use termdesi gn parameters 10
prescnt dcsi gnvari abl es we defi nedearl i er and do not formal l y recogni ze desi gn
parameters as defi ncdhc . The pri nci pnl reasonis that parameters have no rol e10pl ay
deten ni ng opti ma1desi gn. They aresi gni fi cant inthedi scussi onof model i ng
issues. Exampl es of parameters i ncl udematcri a1propc yappl i edl oads. and choi cc
of shape. The paramctcrs in the abstract mathemati cal model are rep sentcd in a
si mi l ar f ormas thedesi gnvec1or excep1 t we use thecharactcr p. Therefore. [P]
P [P"P2' .. .Pq1rep sent eparameters of theprobl em. Note el eng ofthe
par ne r vector is q. Except in c di scussi onof model i ng epar neters will not
be expl i ci tl y rredto as they arepri mari l y predetrmi ncdconstants in desi gn.
Des 'nFunct l ons: Desi gn functi ons defi ne meani ngf ul i nformati on about the
desi gn. They are eval uated usi ng the desi gn vari abl es and desi gn parameters
di scussedearlier. They establ i shthemathemati cal model of thedesi gnprobl em. These
functi ons can represent desi gnobj ecti ve(s) andl or constraillts. As i name i mpl i es
desi gn 0ecti ve dri ves the search for the opti mal desi gn. The sati sfacti on of the
traillts establ i shes theval i di ty of thedesi gn. If not expl i ci t1y stated edesi gner
is responsi bl c for id nti fyi ngthc obj ecti vc and constrai nts. Mi ni mi ze themass of the
6 I NTRODUCTI ON
structurewi l l transl ateto obj ecti veU1ction. The sinthemateri al must be less
thantheyi eI d strengthwi1l transl atetoa constrai ntfuncti on. Inmany probl ems it is
possi bl efor thc same functi ontoswi tchrol es toprovi de di ffcrcnt desi gnscenari os.
ObJ ect l ve Funct i on The tradi ti onal desi gn opti mi zati on probl emis dcfi nd
usi nga si ngl eobj ecti vefuncti on. leformat of this statemcnt is usual l ytomi ni mi zc
or maxi mi ze some quanti ty is cal cul ated usi ng somc dcsi gn functi on. Thi s
functi on must depend expl i ci tl y or i mpl i ci tl y on the desi gn vari abl es. In the
literaturc is probl cmis cxp ssed exc1usi vel y wi thout loss of generaI ity as a
mi ni mumprobl em. A maxi mumprobl emcan b rccast as a mi ni mi zati on probl cm
usi ngthencgati vc or reci procal of thc functi onused for theobj ecti vc functi on. ln
thefirst exampl e in ducedearl i er c oecti veis tomi ni mi ze cost. l erefore
desi functi onrepres g cost wilI be theobj ecti vefuncti on. In esecond se
e obj ecti ve is to mi ni mi ze mass. I n the thi rd case the obj ecti ve is to maxi mi zc
machi ne utilization. lear of si ngl eobj ecti vedesi is consi deredmature today.
No newsol uti ontechni ques for cl assi cal probl cms hav becnadvanced for somc ti me
now. Today. much of the wor k in appl i ed opti mi ti on is di rccted at expandi ng
appl i cati ons topracti cal probl ems. lnmany cases. this has i nvol vedcrcati veuse of the
sol uti on techni ques. In abstract mathemati cal model . the obj ecti ve functi on is
reprosentedby thesymbol f To i ndi cateits dependence on thedesi gnvari abl es. it is
f quentl y cxpressed f ( x" x2.. . . x
n
). A more conci sc prcsentati on is j{X).
Si ngl e obj ecti veprobl cms have onl y one functi ondcnoted by f It is a scaI ar (not a
V tor). Not c at al thoughtheobj ccti vefuncti ondepends on P (parameter vector). it
is not expl i ci tl y inc1uded inthef ormat .
Mul ti obj ecti ve and mul ti di sci pl i nary dsi gns arc i mportant devel opments today.
Mul ti obj tivedesi gn or mul ti pl eobj ecti vedesi gn. refersusi ngsevcraI di erent desi gn
functi ons to e s h for op nal di gn. GeneraIl y ey are expec d b e
confl i cti ng0cctives. They coul dal sobeopcrati ngobj ecti ves. lecurrent approach
to esol uti onof theseprobl ems i nvol ves standardopti mi zati onproced appl i cdtoa
si ngl e ( onstructed obj ecti ve opti mi zati on probl cmbased on the di fferent mul ti pl e
obj ecti ves. A pop ar approachis tousea sui tabl y wei gh dl i n r combi nati onof
mul ti pl c obj ecti ves. A practi cal l i mi tati onwi isapproachis .echoi of wei ghts us
inthemodel . Thi s approachhas not been cmbraced wi del y. An al temati veapproachof
recogni zi nga prcn ecti veand sol vi nga si ngl c obj ecti vc probl emwi thaddi ti onal
constrai nts basedon ni ngo ve ncti onscanusual l ygene accep le
sol uti on. Inmul ti obj ecti veprobl en
l
l
Y
2
4
4
y
f
J
Const rai nt Funct l ons: As desi gn functi ons these wi l l be i nfl uenced by the
desi gnvari abl es. The format of thesefuncti ons requi res t hemtobe compared tosome
numeri cal l y l i mi ti ngval ue at is establ i shedby desi gnrequi rement or thedesi gner.
This val ueremai ns constant duri ngthc opti mi zati onof theprobl cm. A wcl1-descri bcd
dcsi gn probl emis expected to inc1ude scveral such functi ons. whi ch can be
represented as a vcctor. The compari son is usual1y set up usi ng the threc standard
onal operators: = S and. Consi der our first exampl e. Letf i mJ (X) represent
efuncti on cal cul ates the vol ume of the newsoda can we aredesi gni ng. le
constr nt on thed si gncan bc expresscdas
I I J (X) = 500 c m
J
IntlI e second exampl e. l et m2( X) b thefuncti onthat cal cul ates thedefl ecti onof the
beamunder theappl i edl oad. The constrai nt can be statcdas
ft11l 2(X) :S; I mm
Thccons int functi ons can be c1assi fi edas eqllalityconstrai nts [l i kefunl (X) aboveJ
or i nequal i tycons nts [l i kefun2(X)].
Probl ems wi thout constrai nts aretcrmcd unconstrai nedprobl ems. If constrai nts are
present then meeti ng themis mor e p mount an opti mi zal i on. Constrai nt
sati sfacti on is necessary beforc the dcsi gn establ i shed by currcnt val ue of the
desi vari abl es is consi dered val i dand acceptabl e. If constrai nts arenot s sfied
thenthereis no sol uti on. A feasi bl edesi gnis one inwhi ch all of theconstrai nts are
satisfied. An opti mal sol uti on is one at has met the desi gn obj ecti ve. An opti mal
dcsi gn must be fcasiblc. The desi gn spacc cncl oscd by thc constrai nts is cal l ed the
feasi bl edomai n. Dcsi gn space is descri beda f ewparagraphs bcl ow.
Equal l ty Constral nts: Equal i ty constrai nts are mathemati cal l y neat and easy to
handl c. Numeri cal l y eyJ1 ui remore effort tosatisfy. They areal somore restrictive
on tlI e desi gnt hey l i mi t theregi onf romwhi ch thesol uti oncan be obtai ned. The
symbol representi ngequal i ty constrai nts intheabstract model is h. Theremay be
mor e thanone equal i ty constrai nt inthedesi gn probl cm. A vector representati on
for ual i ty constrai nts is i ntroducedthrough thefol l owi ngrepresentati on. [8] [h
l

h
2
. . . . h]. and h
k
: k = 12 . . . .1areways of i denti fyi ng eequal i tyconstrai nts. le
dependence on thedesi gnvari abl es X is omi ttedfor convcni ence. Notc at thel ength
of thevector is 1. An i mportant onfor di sti ngui shi ngtheequal i ty and i nequal i ty
constrai nts is at they arc mnni pul ated di fferentl y in the senrch for the opti mal
sol uti on. The number n of desi gnvari abl es in c probl cmmust be greatcr the
number of equal i ty constrai nts 1 for opti mi zati ontotakepl ace. lf n is cqual to1 . then
probl emwi l l be sol vedwi thout referencetotheobj ecti ve. lnmathemati cal terms
thenumber of equati ons matches enumber of unknowns. If n is less than1 thenyou
have an overdetermi ned scl of rel ati ons whi ch coul dresult inan i nconsi stent probl cm
defi ni ti on. The sct of cqual i tyconstrai nts must be l i nearl yi ndcpendent . Broadl y this
8 I NTAODUCTI ON
i mpl i es you cannot obtai n one of the consaints f romel ementary ari thmeti c
operati ons on thercmai ni ngconstraints. This serves 10ensurethat themathemati cal
searchfor sol uti onwill not fai l. These tcchni ques arebasedon methods f romIi ncar
al gebra. Inthestandardformat for opti mi zati onprobl ems the ua1ityconsaintsare
wri ttenwi tha 0 on theri ght-handside. Thi s means eequal i tyconstrai nt inthc
first exampl e will be exp ssedas
hl (X): f unl (X) - 500 = 0
Inpractica1probl ems equal i tyconstrai nts arerar Iyencountcred.
I nequal l ty Constrai nts: In ua1itycons aints app r more natural l y inprobl cm
formul ati on. Inequal i tyconstrai nts alsoprovi demore flexibility indcsi gnselection.
The symbol represcnti ngi nequal i tycons'aints intheabstract model is 8. There may
be more onc in ua1ity constrai nt in the desi gn probl em. The vector
reprsentati onfor i nequal i tyconstrai nts is si mi l ar towhat we hav seenbefore. Thus
[G) [8182 . . . 811/] and: j = 1 2 . . . m arc ways of i denti fyi ngIhei nequal i ty
cons aints. m reprents enumber of i nequa1ity constraints. AIl desi gn functi ons
explicitlyor implicitlydepend on thedesi gn(or i ndependent) vari abl eX. 8 is usedto
descri ho bo loss thun or cqual 10 ) dgreal cr 01" equlll 10() constralnts.
Strictly greater ( >) and strictly Icss th( <) arenot usedmuch inopti mi zati on
because sol uti ons are ua11y expectedto lie at econstrai nt boundary. In the
standard format all probl ems arc expressed wi eS relationship. Moreover the
ri ght-handsi deof eSsi gni sO. lei nequal i tyconstrai nt f romthesecondexampl e
f t m2( X) is set up as
81(X): n2( X) - 1 O
In case of i nequal i ty constrai nts a di sti ncti onis madc as to whether the dcsi gn
vari abl es lieon theconstrai nt boundary or in interior of theregi onbounded by E
constrai nt. If the set of d si gn vari abl es lie on the boundary of e constrai nt
mathemati cal l y this expresses efact that constrai nt is satisfiedwi thstrict equal i ty
isg = O. The constrai nt acts like an equal i ty constrai nt. In opti mi zati on
termi nol ogy is parti cul ar cons aint is referredtoas anacti veconstrai nt. If theset
of desi gnvari abl es do not lieon theboundary that is theyli i nsi de eregi onof thc
constrai nts they arc consi de d i nacti veconstraints. Mathemati ca1ly econstrai nt
satisfies erclationg < O. An i nequal i tyconsU' ai nt canthercforebe either activeor
lllactJ ve.
Sl de Constral nts: Si decons aints area necessarypart of thesol uti ontechni ques
es ci a1ly numeri cal oncs. They express the range for the desi gn variablcs. Each
desi gnvari abl emust be bound by numcri cal va1ues for its l ower and upper Ii mi t. The
desi gner makes this choi cebasedon his anti ci pati onof anacceptabl edesi gn.
4
F
Desi gn Space: l e desi gn space e space at wil be searched for opti mal
desi gn is the Eucl i dean or Cartesi an n-di mensi onal space generated by the n
i ndependent desi gn vari abl es X. Thi s is a genera1ization of the three-di mensi onal
physi cal space wi th whi ch we are familiar. For ten dsi gn vari abl es it is a
ten-di mensi onal space. Thi s is not easy toi magi ne. Il is a1sonot casy toexpress this
i nformati on rough a fi gure or graph because of the Ii mi tati on of the
three-di mensi onal worl d. However if ed:signvari abl es arei ndependent enthe
l1-di mensi onal consi derati ons aremere extrapol ati ons of the e-di mensi ona1reality.
Of course we cannot geometri ca1lydefi n themthoughwe will be worki ng wi ththc
numbers. The si deconstrai nts Ii mi t thesearch gi oni mpl yi ngthat onl y sol uti ons at
lie wi thi n a c rtain regi on will be acceptabl e. They defi ne an n-di mensi onal
rectangul ar regi on(hypercube) f romwhi ch efeasi bl eand opti mal sol uti ons must be
chosen. Later we wil s at themathemati ca1model s inopti mi zati onareusual l y
descri bed by nonl i near rel ati onshi ps.11e sol uti ons to such probl ems cannot be
ana1ytically predi cted they are typi ca1ly govemed by e underl yi ng numeri cal
tec i queusedtosol vethem. It is neccssary to( ict thesol uti ons toanacceptabl e
regi on. The si deconstrai nts provi des ready mechani smfor i mpl ementi ngthis limi t.
Care must be taken that thcse limits arenot i mposed overzeal ousl y. Thcre must be
reasonabl espacefor numeri cal techni ques 10operate.
The St andar d Format : The above defi ni ti ons a1l owus to assembl e Ihe general
abs .ct mathernati cal model as
Mi ni mi ze f(x. x2 . . . x
n
)
Subj ect to: hl (x" X2 XI I )=O
h2(x. x2 . . . x
n
) = 0 (1. 2)
. 1

l

h/(XI' X2.. XI I ) =0
8. (X" X2.. Xn) O
82(XI' X2 Xn) S 0 (1.3)
gm(XI X2.. Xn) SO
xl SXj Sxi i = 12... n (1.4)
The same probl emcanbe expressedconci sel yusi ngthefol l owi ngnotation:
Mi ni mi ze f(x. X2 . . . X
n
) (1. 5)
Subj ect to: hk(xt X2 X
n
) = 0 k = 1 2 . . . I (1.6)
(X" X2 . . X
I I
) S 0 j = 12. . . . m (1. 7)

10 I NTROOUCT10N
xl sxj Sx i =12... n
Expl oi ti ngvector notati onthemathemati cal model is
Mi ni mi ze j(X) [X].
Subj ect to: [h( X)]1= 0
( X)]tnS 0
X10wSX S Xup
(1. 8)
(1. 9)
(1.10)
( 1. 11)
(1.12)
The above mathemati cal model expresses fol l owi ng standard format of the
opti mi z onprobl emexpressedinnatural l anguage:
Mi ni mi ze theobj ecti vefuncti onj subj ect to1 equaityconstrai nts m i ncqual i ty
constrai nts wi th e n desi gn vari abl es I yi ng between pr cri bed l ower and
upper limits.
letechni qucs in is book will appl y tothc probl emdescri bcdin c abovc f ormat.
To sol veany speci fi c desi gnprobl emit is requi rcd10refunnul atc theprobl cminthe
above manner so methods can be appl i ed di rectl y. Al so in this book the
techni ques are devel oped progressi vel y by consi deri ng the standard model wi
ducedel ements. For exampl e th unconstrai nedprobl emwil be expl o dfirst. The
equal i ty cons ints a consi derednext fol l owedby thei nequal i ty constrai nts. d
fi nal l y thecompl ete model . Thi s represents a natural progressi onas pri or knowl edge
is used to devel op addi ti onal condi ti ons need to be sati sfi edby the sol uti on in
ei nstances.
1. 1. 2 Mat hemat l cal Model l ng
Inthis secti on ethreedesi gnprobl ems i ' Oducedearl i er wi l l be ansl atedto e
standardf ormat. The mathemati cal mode1wi l l first be i denti fi ed. The second probl em
requi res i nformati onf roma course inmechani cs and materi al s. Thi s shoul dbe wi thi n
.epurvi ewof most engi neeri ngstudents.
Exampl e 1.1 New consumer research i ndi cates at peopl e like to dri nk about 0.5
liter of soda pop at a ti me duri ng the summer months. The fabri cati on cost of the
redesi gnedsoda can is proporti onal 10thesurfaceareaand can be esti mal edal $1. 00
per square meter of l hemat rial used. A ci rcul ar cross secti on is the most pl ausi bl e
gi vencurrent l ool i ngavai l abl efor manufacl ure. For aestheti c reasons tl1e hei ght must
be at least twi ce tl1e di ameter. Studi es i ndi cate at hol di ng comf ort requi res a
di ameter between 6 and 9 cm.
Fi gurc 1.1 shows a sketch of tl1e can. In most product desi gns parti cul arl y in
engi neri ngthe model is easi er to devel op usi ng a figure. The di ameter d and the
hei ght h are suffi ci ent to dcscri be tl1e soda can. What about the tl1i ckness t of e

p
J
:<:
d
1.1 OPTI MI ZATI ONFUNOAMENTAl S 11
d
h
Fl gure1.1 Exampl e 1.1- Desi gn01a newbeveragecan.
matcri al of the can? What arc l he assumpti ons for thc dcsi gn probl cm? One of the
assumpl i ons coul d be lhat t is smal l cnough to be i gnored in thc cal cul ati on of thc
vol ume of soda in ecan. Another assumpti onis that emateri al qui redfor l hecanis
onl y thecyl i ndri cal surfac . 'hetopand botl omof thccan will be fittedwi thcnd caps
at w provi de emechani smby whi ch thesoda canbc pow . is not part of is
dcsi gnprobl em. Inthe tattempt at dcvel opi ngthemathemati cal model we coul dstart
out by consi deri ng equ tities i denti fi ed oveas des 1variables:
Desi gn vari abl es: d h t
Revi ewi ng thestal ement of thedesi gnprobl em one of theparameters is thecost of
materi al per uni t area is gi venas l (t per sqU8I centi meter. bi s is i denl i fi ed C.
Duri ng thesearchfor theopti mal sol uti on is quanti ty will be hcl dconstant at e
gi ven val ue. Note that if this val uc changes l hen e cost of thc can will
correspondi ngl y change. This is what we mean by a desi gn p ameter. Typi cal l y
change inparameters wil cause thesol uti onto computed.
Desi gn parameter: C
The desi gnfuncti ons wi l l i ncl ude ecomputati on of thevol ume encl osedby thecan
and thesurfaceareaof thecyl i ndri cal secti on. The vol ume inthecan is cf2h14. l e
surfaceareais 1tdh. The aesthcti c constrai nt requi res at h 2d. The si decons aints
on di ameter areprescri b in eprobl em. For compl el ness thesi deconstrai nts
on other vari abl es have tobe prescri bedby thedesi gner. We can formal l y set up
theopti mi zati onprobl em
Mi ni mi ze f ( d h t): C7rd1z (1.13)
Subj ecl l O: h Mht): 7rd
2
/1/4- 500 = 0 (1.14)

12 I NTRODUCTI ON
/
Intui ti vel y therc is some concemwi th the probl emas expressed by Equati ons
(1.13( 1. 15) even though descri pti onis valid. How can theval ue of thedesi gn
vari abl et be estab1i shed? The vari ati onint does not a 'ect thedesi gn. Changi ng the
val ueof t does not changef h
l
or gl Henceit cannot be a desi gnvari abl e. (Note: the
vari ati onint may affect theval ue of c.) lf is werc a practi cal desi gnprobl cm then
cans have to be desi gned for i mpact and stacki ng strength. In C et wi l l
probabl y be a critical desi gnvari abl e. Thi s wi l l requi resever addi ti onal s cl ural
constrai nts intheprobl em. It coul d serveas an i nteresti ngextensi ontothis probl em
for homewor k or proj ect leopti mi zati onprobl emafter droppi ng t d expressi ng
[d. h] [XI X2] becomes:

4
I
V
-
1
-
-
gM. h. t): 2d- h S O
6Sds 9; 5 s /t s20; O. OOI St sO. Ol (1.15)
Fl gure 1.2 Exampl e 1.2- Canlllever beam.
Mi ni mi ze f(XI X2): c : ( 1.16)
(1.17)
gl (lXU: 2x1- X2S0 (1.18)
l ntui ti vel y el arger theval ue of F. thegreater is cro s8-Secl lio na al are = anecs v
t .0ha nd leit. 11c a n b e i l yconcl uded at i f F w ad si
b sel at its l ower limit a8l hemass of thebeamwill be di recl l y proporti onal 10 e
areaof croS88cc(I un. No techni ques arer ui redf. or lhisconcl usi . on. By . our defi ni ti on
F is a go. od candi date as a par neter rather as a desi gnvari abl e. Si mi l arl y the
l arger theval ue of L eg ater is themoment F generates about thecanti l evered
end. lis wi l l requi re an i ncreasein earea. of cross-secti on. Therefore L is not a
good choi ce f. or edesi gn vari abl eei ther as thebest choi ce f. or it will bc i l. ower
I i mi t. To represcnt materi al as a desi gnvari abl e. we necd touseitsSl ruCl ural propcrti es
in eformul al i onof the probl em. Typi cal l y. a mal eri al is characl eri zedby its spec c
Subj ecl l o: hl (xl Xv: ru / 4- 500=0
6 SX) S 9; 5 SX2 S 20
The probl em presenl edby Equati . ons (1.16)(1. 18) is mathemati cal modef. or
th desi gnprobl emexpressedinthesl andardf ormat. For this probl emno extranc. ous
i nformal i . on was n dtoset up eprobl emexcept for some geometri cal rel ati ons.
Exampl e 1. 2 A canti l ever beamneeds 10be desi gnedtoc ry a poi nt l oadF al the
end . of thebeam. of l engl hL. The cross secti onof thebeamwiII be intheshape of the
letter 1 ( ferredtoas 1b m). lebeamshoul dmeet prescri bedfai l urecriteria.
Thcre is al so a I i mi t on its defl ecti . on. A beamof mi ni mummass is qui red 10be
di gncd.
Fi gure 1.2shows a si devi ew. of thebeamcarryi ngthel oadand Fi gure 1.3a typi cal
cr. oss s nof t h be n. The I -shapedcross secti onis symmetri c. The symbol s d tw.
b
r
and t r c. orrespondtothedep. of thebcam thi ckness of theweb wi dth of th fl ange
and thi ckness . of thefl ange respecti vel y. Thes quanti ti es aresuffi ci ent todefi ne E
cr. oss secti on. A handbo. ok or textb. ook on strcngth of materi al s can ai d e
dvel opment . of th desi gnfuncti ons. An i mportant assumpti on for this probl emis
we wi l l be w. orki ng wi thi n the elastic I i mi t of thc materi al where there is a I i near
relati . onshi pbetween estress and thestrain. AlI . of thevari abl es i denti fi edinFi gure
1.3wi l l s ngl y affect thesoluti . on. They aredesi gnvari abl es. The appl i edf. orceF
wi l l al sodi rectl y affect theprobl em. So will its locati . on L. H. ow about materi al ?
Stecl is dcfi ni tel y supcri . or 1.0 c. opper for the beam. Shoul d F L and the materi al
pr. operti es be i ncl udedas desi gnvari abl es?
bt
~
' l
4t W -
Fl guro1.3 Exampl e 1.2- Cross-sectlonal shape.

14 I NTRODUCTI ON
wei ght y its val ue of the modul us of elasticity. E. its modul us of rigidity. G. its yi el d
l i mi t intensi onand compr si on.yi e' d. i yi el d insh yield. its ul ti matestreng
intensi on.ul t. and its ul ti matestrengthinshear.ul l Opti mi l i ontechni ques i denti fy
di ffe nt va1ues of the desi gn vari abl es. If E is the vari abl e used to present E
materi al desi gn vari abl e in a probl em. then it is possi bl e the sol uti oncan qui rea
materi a1 at does not exi st at is. a va1ue of E still undi scover d. For a11 i ntents and
p poses. is is beyond the scope of thedesi gner. Agai n. materi a1 as a parameter
makes a lot of. sense and eopti mi zati onprobl emshoul dbe rei nvesti gatedif F
or themateri al changes. Concl udi ng f rom is di scussi onon model i ng. thefol l owi ng
I i st can be establ i shedfor this exampl e:
Desi gn p 1eters F. L . {y. G. Y'eld'1: yi eld' u11' 1: E
Desi gn vari abl es: d t
w
' b r ' I
f
In the dev l opment of the mathemati cal model standard techni cal defi ni ti ons
used. Much of the i nformati on can be obtai ned f roma mechani cal engi neeri ng
handbook. Th first desi gnfuncti onis wei ght of thebeam. Thi s is theproduct of
y L and Ac where Ac is eareaof cross secti on. The maxi mumslress due tobendi ng
""ndcan be cal cul atedas theproduct of FLdI21cwher e i s thc momenl of inertia
about thc ccntroi d of the cross secti on al ong the axi s paral l el to the f1ange. The
m i mumshear stress in cross secti onis expressedas FQcl l c1w wherc Qc is first
moment of areaabout thecen idparal l cl tothc fl ange. The maxi mumdefl ecti on
of thebeamwi11be at theend of thebeamcal cul atedby theexpressi onF L
3
/ 3El
c
' Ac
of thecross secti onof thebeamis 2brtr + Iw(d- 2tf). The first momcnt of areaQc can
bc cal cul atedas 0. 5b
f
tf (d- tr) + 0.51
w
(0. 5d- tf)2. Fi nal l y. c momcnt of i nerti ai s
obtai nedf rom(b 112) ((b
r
- tw)(d- 2tf)3/12). In fol l owi ngfor convcni ence
we wi l l conti nue to use Ae Qc' and i nst cad of detai1i ng their dependence on e
desi gnva1ues. Associ ati ngx wi thd X2wi tht
w
' X3wi thb
r
and X4wi thlr so at X=
[xX2X3' x41 eprobl eminstandardformat is
Mi ni mi ze .ftX): YLAC ( 1.19)
Subj ect to: g(X): FL x / 21c- yie'd 0
g2(x ): FQJ 1.x: x2- 1: yie e
g3(X):

0. 01 x 0.25; 0. 001 X2 0.05;
0. 01 X3 0.25; 0. 001 X4 0. 05
(1. 20)
To sol ve is probl em F must bc prescri bed (10000 N) L must be gi vcn (3 m).
Matcri a1must bc sel ected(stee1: =7860 kgl m
3
; O"yield= 250 E+06 N/ m
2
; yicld= 145
E+06 N/ m
2
). A1so themaxi mumdct1ecti onis prcscri bcd = 0. 005m).
h
F
d
f
J
d
J
i
J
d
h
h
b
M
U
F
F
1.1 OPTI MI ZA TI ONFUNDAMENTALS 15

Thi s is an exampl e wi th four desi gnvari abl es threei nequal i ty constrai nts. and
no equal i ty constrai nts. Othcr vcrsi ons of this probl emcan easi l y be formul ated.
It can be reduced toa two-vari abl c probl cm. Standard fai l urecri teri awi th respect
to combi ned stresscs or pri nci pa1 strcsses can bc formul ated. If the canti l evered
end is bol ted then addi ti ona1 desi gn functi ons regardi ng bol t failure need to be
exar ned.
Exampl e1. 3 MyPC Company has deci dedtoi nvest $12 mi l l i oninacqui ri ngsevera1
ncwComponent P1acement Machi nes tomanuf acture di erent ki nds of mo e oards
for a newgenerati onof personal computers. Three model s of thesemachi nes areunder
consi derati on. Tota1number of operators avai l abl e is 100 becausc of el ocal l abor
market. A fl oor space constrai nt needs to be sati sfi ed because of e di fferent
di mensi ons of se machi nes. Addi ti ona1 i nformati on rel ati ng to each of e
machi nes is gi veninTabl e 1.1. ecompany wi shes todetermi ne how m y of each
ki ndis appropri atetomaxi mi zc thenumber of boards manuf actu dperday.
lt is difficu1t touse a fi gureinthis probl emtosct up our mathemati cal model . The
number of machi nes of cach model necds to be determi ned. T his wi l l serveas our
desi gnvari abl es. Let XI represent th number of Component Pl acement Machi nes of
Model A. Si mi l arl y X2wi1l be associ atcdwi thModel B and X3wi thModel C.
i
!
i

D igll vari ubl es: xX2X3
T he i nfonnati on in Tabl e 1.1 is usedto set up the desi gn functi ons interms of e
desi gnvari abl es. An assumpl i on is made that all machi nes arerunfor threesh s. T he
cost of acqui si ti onof themachi nes is thesumof ecost per machi ne mu1ti pl i edby
number of machi nes (g1)' Th machi nes must satisfy efl oor spacc constrai nt (82)
whi ch is defi ned wi thout details in Eq. (1. 23). The constrai nt on the number of
opemtors is three ti mcs sumof eproduct of the number of machi nes of ch
model and theoperators per shift (g3)' T he uti l i zati onof each machi ne is thenumber
of boards per hour ti mes the number of hours the machi ne operates per day. T hc
opti mi zati onprobl emcan be assembl ed in fol l owi ngform:
f
s
Ma i mi ze f (X): 18 (1. 21)
or
T Mi ni mi ze f ( X) : - 18x 55 x x - 18x 50 X x
2
- 21X 50 x X
3
Su ct to: g(X): 400OOOx + 600. 000X2 + 700000X3 120 000 (1. 22)
g2(X): 3x -X2+ X3 30 (1. 23)
83(X): 3xI + 6X2+ 6x
3
100 (1. 24)

XI 0; X2 0; X) 0
E
r
i
-
5
2
2
J
J
y
d
-

17
9
OPTrMI ZATI ONFUNDAMENTALS
6
1.1
5
I NTRODUCTI ON
Equati ons (1. 21)(1. 24) express mathemati cal model of theprobl em. Note tin
this probl emthereis no product bci ngdcsi gned. Here a strategyfor pl aci ngorder for
number of machi nes isbei ngdctermi ned. Equati on(1.21) illus thetranslation
of the m i mi zati on obj ecti ve into an ui val ent mi ni mi zi ng one. The i nequal i ty
constrai nts inEquati ons (1.22)(1. 24) aredifferent f romtheprevi ous two ex nples.
He theri ght-handsideof theS operator is nonzero. Si mi l arl y thesi deconstrai nts
are bound on the l ower sideonl y. Thi s is done deliberately. There is a significanl
di fferencebetween isprobl emand theprevi ous two. Al l of thedesi gnfuncti ons here
arelinear. Thi s probl emis c1assifiedas a linear prograrnmi ngprobl em. The sol uti ons
10 these typcs of probl ems are very different than thc sol uti on 10 thc first two
cxampl swhi ch arerecogni zedas nonl i nar programmi ng probl ms.
1.1.3 Nature of Sol uti on
le ecxarnpI sincprevi ous sectionarc usedtodi scuss enatureof sol uti ons
the opti mi zati onprobl em. Exarnpl es 1.1 and 1.2dcscri be enonl i n r programmi ng
probl cm whi chforms themaj or part of is book. Exarnple1.3is a lincar programmi ng
probl em whi chis very significant indcci si onsci cnccs but ra inproduct design. Its
inc1usi on here whi l e necessary for compl ctcness is also i mportant for understandi ng
contemporary opti mi zati ontechni ques for nonl i near programmi ng probl cms. Exarnpl es
1.1and1. 2a dealt wi thftrSt. Between thel wo we noticethat Exarnpl e1.1isquitesi mpl e
relativetoExarnpl e1.2. Second thefour variables inExampl e 1.2make it difficult touse
i1l ustrationtoestablishsome of thcdiscussion.
16
representation01Exampl e1.1.
bo of thesc functions. le uality constraint gl is lincar. lnEquation(1.18) is is
evi dentbausethedesi variablesappcar by emsel veswi thout bei ngraised10a powcr
o 1. As we devel op etechni ques for appl yi ngopti mi zati on is distinctionis
i mportant to keep in mi nd. It is also significant graphi cal representati on is
typicallyrestrictedtotwo variables. For reevari abl es we needa fourthdi mensi on
10 resol ve the i nformalI on and three-di mensi onal conl our plots are not easy to
iII ustrate. The power of i magi nati onisnecessarytoovercome sehurdles.
1 probl emreprcsentedinFi gure1.5provi dcs opportuni tytoidentifythegraphical
solution. First thefeasibleregi onisidentified. InFi gure1.5 is is E ual i tycons int
above theincquality cons int . Bothfeibleandopl i mal must occur fromthis :gion.
Any solutionfrom is regi onis anaccepl abl e( sible) solution. There areusual l ya
largenumber of suchsol uti ons infinilesolutions. If opti mi zati onis i nvol ved thcn
thesc choi ces must be rcduced tothebest one wi th spcct to some criteria the
objeclive. 1nthis probl eml fi gure thesmal l cst val ueoffi s desired. The l owest val uc
of f is just less than the contour val ueof 308. lt is at eintersection of the two
constraints. Whi l e the val ueof f needs to be cal cul ated theopti mal val ues of the
desi gnvari abl es readf romthefigure areabout6. 75and 13. 5 respectively. Another
significant i temof i nformati on obtai ned f romFi gure 1.5 is gl is an active
constrai nt . Whi l e there are infinite feasible sol uti ons 10 thc probl em the opti mal
sol uti onis Ul tique.
Fl gure1.4
E
Z

5
SOLUT10N T O EXAMPLE 1.1 Thc si mpl cst determi nati on of nonl i neari ty is
througha graphi cal P sentati onof c dcsi gnfuncti ons i nvol vedinthc probl em.
lis canbe donc sily for ODe or two variables. If thefUDCti ODdoes not plot as a
straight lineor a pl ane enit is nonl i ncar. Fi gu 1.4shows a three-di mcnsi onal plot
of Exampl c 1.1. The figureisobtai nedusi ngMATLAB. Chapter 2 will provi d dctailcd
instructions for drawi ngsuchplots for graphi cal opti mi zati on. The threc-di mcnsi onal
pr entati onof Exarnpl e1.1inFi gure1.4does not real1y enhance our understandi ng
of theprobl emor thesolution. Fi gure1.5isanal tematerep scntati onof theprobl em.
Thi s is th oneencouragedinthis book. Thc hori zontal is rep scnts thedi ameter
(XI) and vertical axis e hei ght (xu. 1n Fi gure 1. 5 the equal i constrai nt is
markcd appropri atel yas h 1. Any p rof val ucs on is linewill gi vea vol ume of 500
cm
3
The i nequa/i tyconstrai nt is shown as g 1. The pair of val ues on this lineexact 1y
satisfies thc aes etic requi ment. Hash lines on thesideof thei nequal i tyconstraint
cstablish thedi sal l owed gi on for thedesi gn variables. leconstrai nts aredrawn
thi cker for emph is. The scal ed 0'eclivenCl i on is repr en d through sevcral
l abel cd contours. Each contour is associ ated wi th a xed val ue of the obj ccti ve
functi on and these val ues are shown on the figure. le range of the two axes
establ i shes thesi deCOl l Straints. The obj ecti vefuncti onf andtheequal i l yconstrai nt h 1
'cnonl i near si ncc theydo not plot IISstraight lines. Rcferri ngtoEquati ons (1.16) and
(1.17) this is substanti atedby thepl ' OduCISof thetwo unknowns (desi gnvarillbles) in
19
An i nteresti ngobservati on in the above set of equati ons is at there areonl y
t wo desi gn vari abl es but three constrai nts. Thi s is a val i d probl em. If the
constrai nts were equal i ty constrai nts en this woul d not be an acceptabl e
probl emdefi ni ti on as it woul d vi ol ate e rel ati onshi p between the number of
vari abl es and the number of equal i ty constrai nts. Theref ore the number of
i nequal i constrai nts arenot rel atedto thenumb r of desi gnvari abl es used. Thi s
fact al so appl i es to nonl i near constrni nts. Fi gure 1. 6 is a scal ed pl ot of c
functi ons (1.25) (1.28). Choosi ng thefirst quadrant sati sfi es thesi deconstrai nts.
'he l i neari ty of all of the functi ons is i ndi catedby the strai ght-l i ne pl ots on the
fi gure. Agai n several contours of f are shown toai dtherecogni ti onof the sol uti on.
Al l of thei nequal i ti es aredrawn wi thhash marks i ndi cati ngthenonfeasi b1e si de
of the constrai nt . The feasi bl e regi on is encl osed by the t wo axes al ong wi th
constrai nts g2 and g3' The obj ecti veis toi ncreas theval ue of f (whi ch is i ndi cated
by the contour l evel s) wi thout l eavi ng the feasi bl e regi on. The sol uti on can be
(1. 28)
1.1 OPTI MI ZATI ONFUNOAMENTALS
83(X): 3xI + 6x2 70
X 2 ~ X
1
O ;
J
t
u
b
/
/
d
r
v
d
r
f
L
b
f
f
/
y
J
X
I NTROOUCTI ON 18
20
{
E
H
U
}
-
Z
3
4
25 20
Conl our plol 01modl l l edExampl e 1.3.
1S
number 01Model A
10
Fl gure1.6
5
9
SOLUTI ON T O EXAMPL E 1.3 Incontrast toExampl e 1.1 all of therel ati onshi ps
in mathemati cal model expressed by Equati ons (1.21)(1. 24)are linear. The
word equati onis general l y used todescri betheequi val ence of t wo quanti ti es on ei ther
si de of the = sign. In opti rni zati on we come across mostl y inequalities. e word
equati ol l will be edtoi ncl udeboth of thesesi tuati ons. InExampl e 1. 1 ena
of sol uti on was expl ai ned usi ng Fi gu1. 5. In Exampl e 1. 3 the p sence of thr
desi gnvari abl es deni es is approach. I nsteadof desi gni nga newprobl em suppose
thecompany p si dent has d i dedto buy fi vemachi nes of thethi rdtype. The
mathemati cal model wi tht wo desi gnvari abl es Xl and X2' is reconstructedusi ngthe
i nformati onthat X3 is now a parameter wi ththeval ueof 5. Substi tuti ngfor x3 inthe
model for Exampl e 1. 3 and cl eani ngup thefirst cons int :
M i mi ze
Subj ect to:
(1. 25)
(1. 26)
( 1. 27)
H137
8.5
j(X): 990 XXI + 9x X2 +5250
Conl our plol 01Exampl e 1.1.

gl ) : O.4xl + 0. 6x2 8.5
7.5
dlameler (cm)
g2( X) : 3xI - X2 25
7
Agure 1.5
6.5
5
6
20 l NTRODUCT10N
spottcdat thei ntersecti onof constrai nts g2 dg3. The val ues of thedesi gnvari Ics
at thesol uti oncan be readoff f romthc pl ot. It is not over yet. Bef ore proceedi ng
further il is neccssary toacknowl edgc that a sol uti onof 6.5machi nes for Model B
is unacceptabl e. The sol uti on must be adj usted to the ncarest i nteger val ues. In
aCl ual practi ce. this adj ustmcnt has to bc made wi thout vi ol ati ng any of the
constrai nts. The need for i nteger vari abl es is an i mportant consi derati onindcsi gn.
V abl es at ar qui redtohave onl y i nteger val ues bel ong to etypeof vari abl es
cal l eddi scretevari abl es. 'his book most1y consi ders conti nuous vari abl es. Al most a11
of the mathemati cs an engi n r encounters especi al l y academi cal l y bel ong to E
domai n of conti nuous vari abl es. Thc assumpti on of conti nui typrovi dcs very fast and
cffi ci cnt techni ques for the search of constrai ned opt i mumsol uti ons as we will
devel op in this text . Many real- I i fc desi gns especi al l y those that use off-the-shel f
materi al s and components determi ne di screte model s. Di screte programmi ng and
i nteger programmi ng are di sci pl i nes that address these model s. The nature of
searchmethods at is methods used for searchi ng eopti mal sol uti on indi scrctc
prograrnmi ng is very di fferent f rom at inconti nuous programmi ng. Many of th
methods are based on the scanni ng and repl acement of the sol uti ons through
exhausl i ve search of the desi gn space in the desi gn regi on. In this book di sc tc
probl ems arc typi cal l y handl edas if they were conti nuous probl ems. The conversi on
to edi scrctesol uti onis thefinal part of thedesi gneffort and is usual l y perf ormed
by the desi gner outsi d of any opti mi zati on techni que we devel op in this book.
Chapter 8 di scusses di screteopti mi zati onwi th exampl es.
Getti ngback to el i near programmi ng probl emj ust di scussed it shoul d al sobe
notedthat onl y theboundary of the feasi bl eregi onwi l l affect thesol uti on. He is a
way that will make is apparcnt . Not e that theobj cti vefuncti oncontours ar strai ght
lines. A di fferent obj ecti vefuncti onwill be di spl ayedby paral l el l i nes wi tha di fferent
sl ope. I magi ne theseIi nes on thefi gurei nsteadof theactllal obj ecti ve. Not e at the
sol uti onal ways appears tolieat th i ntcrsecti onof theconstrai nts that is th corners
of thed si gnspacc. Sol uti oncan nevcr com f romi nsi deof thefcasi bl e gi onunl i ke
in the nonl i near programmi ng probl em whi ch accommodat es sol uti on on the
boundary and f romwi thi nthefeasi bl eregi on. Recogni zi ng thc natu of sol uti ons is
i mportant in devel opi ng a search procedure. For exampl e in l i near prograrnmi ng
probl ems a techni que empl oycd tosearchfor theopti mal dsi gnneed onl y searchthc
boundari es parti cul arl y the i ntcrsecti on of the boundari es whereas nonl i near
probl ems thesearchprocedure cannot i gnorei nteri or val ues.
1.1.4 Characterl stl cs of t he Sear ch Pr ocedur e
lesearchfor theopti mal sol uti onwill depend on enatureof theprobl embei ng
sol ved. For nonl i near probl ems it must consi der thefact that sol uti oncoul d liei nsi de
the feasi bl e regi on. Exampl e 1.1 throl l gh Fi gurc 1.5 wi l l provi de e context for
di scussi ngthescarchprocedure for nonl i ncar probl ems. Si mi 1arI y thecharacteri sti cs
1.1 OPT1M1ZAT10NFUNDAMENTALS 21
of the search process for l i near probl cms wi l l be expl oredusi ngEx lple 1. 3 and
Fi gure 1. 6.
p
d
y
d
d
y
d
p
d
d
h
u
u
k
x
y
d
y
M
d
r
i
Nonl l near Prob/ ems: Except for a f ewc1asses of probl cms sol uti ons tonon1i near
probl ems are obtai ned through numeri cal anal ysi s. Through computr code
numeri ca1 anal ysi s becomes numcri cal techni ques. The methods or techni ques for
fi ndi ngthesol uti onto opti mi zati onprob1ems arecal l edsearchmethods. Inappl i ed
mathemati cs and numeri cal t hni ques these referredto as i terati ve techni ques.
I mplici t1y this means seveml tries will be necessary beforc the sol uti on can be
obtai ned. A1so i mpl i edis that cach try or searchis executedina consi stent manner.
I nformati onf rom previ ous iterationis uti l i zedinthecomputati on of val ues inthe
present sequence. Thi s consi stcnt manner or press by whi ch thesearchiscarri edout
and sol uti on determi ned is ca1ed al gori thm. Thi s terma1so refers to the
tr sl ati onof theparti cul ar searchprocedure i ntoan ordereds uenceof step-bystep
acti ons. The advantage is deri ved mostl y in theconversi on of these steps through
some computer 1anguage i ntocode at wi1l execute on ecomputcr. Devel opi ng E
a1gori thmand trans1ati ngit i ntocode that wilexecute inMATLAB wilbe a standard
approach in is book.
The searchprocess is startedtypi ca1y by tryi ngtoguess th initial desi gnsol uti on.
The desi gner can base this sel ecti on on his experi ence. On sevcm1 occasi ons the
success of opti mi zati onmay hi nge on his abilitytochoose a good initial sol uti on. The
search met hod even when used consi stent1y responds di fferent1y to the parti cul ar
probl em at is bei ng sol ved. The degree and type of nonl i neari ty may frequent1y
cause e met hod to fail assurni ng there are no i nconsi stenci es in probl em
formu1ati on. Often number of desi gnvari abl es enumber of cons intsas we1
as a bad initial guess p1ay a part in unexpected behavi or under standard
i mDl ementati on of the method. Thi s has l ed to the creati on of seveml di ffercnt
techni ques for opti mi zati on. Practi ca1experi ence bas a1so suggested at a cl ass of
probl ems respondwel l tocertai na1gori thms.
The mcthods for e search or it mti ve methods. are current1y the onl y way to
i mpl ement numeri cal sol uti ons tononl i near probl ems i rrespecti veof thedi sci pl i nein
whi ch these probl ems app r. 1 e probl ems are p1enti fu1 indesi gnopti mi zati on.
opti mal contro1. structural and t1ui d mechani cs. quant umphysi cs. and astronomy.
Whi1eopti mi zati ontechni ques areappl i ed all of thenoteddi sci pl i nes. in di ti onal
desi gn
23
at a distan sS.. Thi s poi nt satisfies theconstrai nt h.. A irds Ywoul d beto
choosc X. This lies on theconstrai nt boundary of g.. lt is tobe notedthat all E
choi ces arenot feasible. Whi ch poi nt is chosenis dcci dedby therules of a particular
al gori thm. Assume poi nt X) is thenext poi nt in is di scussi on.
Point X) nowbecomes newpoint xoAnew hdircction is de nni n .
Poi nt X
2
is10 d. Anoth S uenceof calculations
next iterationyields
esolution. The essenceof updati ng edesi duri ngsuc ssiveiterationlies intwo
maj or computati ons thc sea h directioll 8/ andstepsi ze. lCchange in edesi gn
vector defin I as .6X whi chrepresents veclor diffcrence for ex npl eX
2
- X. is
obtai nedasS/. For eachiterationthefol l owi ngs uenceof activitycan be identified:
The above is thebasi c structureof a typical opti mi zati onal gori thm. A compl ete
one will indicate m ner inwhi ch S andar e computed. Il wil1incJ ude tests
whi ch will determi nc if sol uti on has bcen reached (convergence) or if the
procedureneeds tobesuspendedor restarted. The diffcrent techni ques tob expl orcd
inthis book mostl y differ in eway S is established. Take a l ook at Fi gure1. 7 again.
At X
o
there are infinite choi ces for the search direction. To reachthe sol uti on in
onabl eti meanacceptabl eal gori thmwill trynot tosearchal ongdi recti ons do
not i mprove thecurrent desi gn. lerearemany ways toaccompl i sh is strategy.
1.1 OPTI MI ZATI ONFUNDAMENTALS
ChooseX
o
IdentifyS
Determi ne
AX=S
XM W= Xo+. 6X
Set Xo Xnen
GoToSt epO
8tep0:
Step1:
:r;-:.
I NTRODUCTI ON
FJgure1.7
g 8.5
A typical ssarchmethod.
7.5
dlamoler (cm)
22
20
5
6
{
E
S
E
m
g
z

Li near Pr ogr ammi ng Pr obl em: It was previ ousl ymenti oned thesol uti onto
se probl ems woul d be on the boundary of feasible gi oo (also called the
j ibledomai n). InFi gure 1.6 e es andsome of thecons ints detenni nethis
regi on. The lalter is di sti ngui shedalsoby epoi nts of interscctionof theconstrai nLs
among themsel ves eorigin and eintersectionof thebi ndi ng(active) coostrai nts
wi ththeaxes. These areverti ces or corners of thequadrilateral. Essenti al l ythedesi gn
i mproves by movi ng f romone of thesevertices to thenext one throughi mprovi ng
feasibledesi gns. To do that you have tostart at a feasibl comer. Thi s will beexpl ored
more fullyinthechapter on linear programmi ng. It is tobe emphasi zed atonl ywi th
two vari abl es canwe actual l yseethegeometri c iII ustrationof thetechni que is
theselectionof vertices as thesol uti onconverges.
Thi s part of Chapter 1 has dealt wi ththei ntroducti onof theop tionprobl em.
Whi leit was approachedf romanengi neeri ngdesi gnperspccti ve themathemati cal
model inabstract terms is not sensitiveto any p ticular discipline. A f ewdesi gn
rules wi th resp t to specific exampl es were exami ned. TIlestandardmathemati cal
model for opti mi zati on probl ems was established. Two broad cJ asses of probl ems
initial starting ess is e poi nt marked xo Remember when sol vi ng
opti mi zati onprobl em. sucha figuredoes not ist. If it di d enwe caneasilyobtai n
thesol uti onby i nspecti on. lis statement is espcci al l ytrueif erearemore thantwo
variables. In the event th are two vari abl es is no reason not to obtai n a
graphi cal sol uti on (Chapter 2). Thc mcthods in opti mi zati on rely on thc dircct
ex pol ati onof theconcepts andi deas devel o dfor two vari abl es toany number of
desi gnvariables. That is thereasonwe will usetwo vari abl es todi scuss anddesi gn
most of theal gori thms andmethods. Moreover usi ngtwo vari abl es conceptscan
alsobereinfor dusi nggeometry. The desi gnat thepoi nt X
o
is not feasible. lt is not
useful to di scuss whether it is an opti mal solution. At least one more iteration is
necessary toob nthesolution. Consi der enext sol uti onat X
I
. Most methods in
opti mi zati onwill attempt tomove toX I by first i denti fyi nga searchdi recti on S 1Thi s
is a vector (same di mensi onas desi gnvari abl evcctor X) poi mXol oX
I e
actual di stancef romxotoX
I
call it XoX.. is some mul ti pl eof SI' s SI X

1
0nce
agai n is not feasible al tbough it is an i mprovement on xosi nce it satis E
i ncqual i l yconstrai nl gl (l yi nginlheinlerior of econstraint). Al tematel y usi nglhe
same scal'chdi rccti onS 1 another techni queJ me odmay choose for itsnext poi nt X 1'
.1""':-......1:

n ' '
24 I NTRODUCTI ON
linear andnonl i near. were i ntroduc . Thegeome ical characteristics of thesol uti on
for thesc probl ems were shown rough graphi cal descri pti onof eprobl ems. An
overvi ewof esearchtechni ques for obtai ni ngthesol uti onto theseprobl ems w
also iII ustrated. It was a1so menti oned at e sol uti ons were to be obtai ned
numeri cal . pri mari l ythroughdigital computati on.
The numeri cal methods usedin isareaareiterativemethods. Numeri cal sol uti ons
i nvol ve two components: al gori thmthat will stablish iterative set of
calculations. andsecond. thetranslationof theal gori thmintocomputer codes. usi ng
aprogrammi ng l ang 'ge. Thi s fi ni shedcode is referredto software. Until recentl y
inengi neeri ngeducati on as wel l as practice. thecompu r l anguageof choi cewas
predomi nantl y FORTRANwi tha I i ttlePascal and C. The computati onal envi ronment in
whi ch it was del i veredandpracti cedw emai ' amecomputi ng systems. Today
esceneis verydi fferent. Indi vi dual personal computers Cs)runni ngWi ndows
Maci ntosh LI NUX or other flavors of UNI X operati ngsystems arenowthemeans
for techni cal computi ng. Theypossess more power thantheol dmai n amecomputers
andtheyarei nexpensi ve. The power of thegraphi cal user interface(GUI ) eway
euser interacts wi thecomputer and esoftware h revol uti oni zedthecreati on
of software today. New programmi ng paradi gms a 11y 0'ecl ori ented
programmi ng have led to newand i mproved ways to devel op software. Today
cngi nccri ngsoftwarc is crcatcdby Buchgcncl' al programmi ng l anguagcs Il S C C++
J ava Pascal andVi sual Basi c. Many softwarevendors devel optheir own extensi ol1s
tothestandardl anguage provi di nga compl e1e nvi ronment for softwaredevl opment
by e user. Thi s makes it very conveni ent for users who are not necessari l y
programmers toget eir j obdone ina reasonab1y short time.
One suchvendor is Ma Wo swi their fl agshi pproduct MA TLABa10ng wi tha
compl ementary col l ecti on of tool boxes at del i ver specific di sci pl i ne-ori ented
i nformati on. In i sbkMATLAB is usedexcl usi vel yfor numeri cal support. No pri or
fami l i ari tywi thMATLAB is expected thoughfamiliaritywi ththePC is requi red. Li ke
all hi gher-1cvcl l anguages effecti veness wi ththis newprogrammi ng resourcecomes
wi theffort frequency of use. original work anda consi stent manner of application.
By thecndof this book it is hoped the ader will bevery fami l i ar wi ththc u
1.2 I NTRODUCTI ONTOMA11.AII 25

1.2 I NTRODUCTl ON T O MATLAB
J
A
2
2
/
/
J
1
MATLAB is i ntroduccdby MathWorks Inc. as thel anguagefor techni cal computi ng.
Borrowi ng fromthe descri pti on in y one of its manual s MATLAB integrates
mputati on visualizatio dprogrammi ng inaneasy-touseenvi ronment where
probl ems andsol ul i ons areexpressedinfarniliar mathemati cal notation. The typical
uses for MATI.. ADi ncl ud
Mat h andcomputati on
A1gori tl l mdevel opment
Model i ng si mul ati on and prototypi ng
Data anal ysi s expl orati on. andvi sual i zati on
Scientificand engi neeri nggraphi cs
Appl i cati ondevel opment i ncl udi ngGUI bui l di ng
The useof MATI..AB has expl odedrecentl yandsohave i features. Tbisconti nuous
enhancement makes MATI..AB anideal vehi cl efor expl ori ngprobl ems inall disciplines
that mani pu1ate mathemati cal content . Thi s abi1i ty is mul ti pl i ed by appl i cati on-
specific sol uti ons through tool boxes. To learn more about the latest MATI.. AB. its
features the tool boxes col l ecti on of user-supported archi ves i nformati on about
Usenet activities other forums i nformati onabout books and80on visit MATLAIl a1
hup: /l www. ma1hworks. com.
1.2.1 Why MA11.AB?
MATLAB is a standard tool for in ' oductory and advanced courses inmathemati cs
engi neeri ng andsci enceinmany universities around worl d. Ini ndustry it isa too1
of choi cefor res hdevel opment andanalysis. In is book MATLAB' S basi c array
el ement is expl oi tedtomani pul atevectors andmatri ces that arenatural tonumeri cal
techni ques. In the next chapter its powerful vi sual i zati on features are used for
graphi cal opti mi zati on. In erest of this book theinteractivefeaturl dbuilt-in
support of MATLAB are utilized to translate a1gori thms into functi oni ngcode in a
fractionof theti meneeded inother l anguages likeC or FORTRAN. Toward eendwe
will inoducetheuseof its Opti mi tionTool box.
Most books on opti mi zati on provi de excell nt coverage on the ~ n i q u s and
a1gori thms andsome a1soprovi dea pri ntedversi onof thecode. The code is qui d
tobe usedinmai nf mmEor l egacyenvi ronments that must support a compi l er inthe
l anguage ecode is wri tten whi ch is predomi nantl y FORTRAN. Each student en
ansl ates theori ntedcode intoa forrnsuitablefor computati on. lenext stepis to
compi l e the code debug compi l ati on errors run programand troubl hoot
executi oncrrors beforeuseful sults canbeobtai ned. An i mportant consi derati onis
thestudent is expectedtobeconversant wi theprogrammi ng l anguage. Whi l e
FORTRANwas formal l y taught toengi n rs several y rs ago is is no l onger true.
7 -
26 I NTRODUCTI ON
Many electrical engi neeri ngdepartments have startedtoembrace C and C++. PCs are
great for theuseof ready-made appl i cati ons especi al l ythosewi thGUI but its usefor
programmi ng inFORTRANhas yet totakeoff.
MATI..ABsuffers veryf ewof thesedi sadvantages. Itsde iscompact di ti si ni ti ve
to l eam. To make probl ems work requires f ewlines of code compared to traditional
l anguages. Insteadof programmi ng f rom egroundup stan rdMA TI..ABpi eces of code
canbe adedtoge eri mpl ement thenewal gori thm. It avoi ds thes ndard :pru:
mpi l ati onandlinks uenceby usi ng inte. reter. Another i mportant asoni scode
po bility. ledewrittenfor ePC versi onof MATI . AB d snot have tobechanged
for theMaci ntoshor theUNI Xsystcms. liswill not be trueif system-dependent resources
medi n ec e. As anilI ustrationof MATI . AB'Sl eami ngagility bo MATI . AB d
appl i edopti mi zati onareveredinthisbook somethi ngverydiffic t aompl i shwi
say FORTRANandopti mi zati on or C++ dopti mi zati on. Fl ermoreIA'..AB likeC or
FORTRAN isa gl obal l yrel evant product .
1.2.2 MATLABI nstal l al i onI ssues
Thi s book does not Care if you are runni ngMATLAB on ani ndi vi dual PC or Mac a
networked PC or Mac or i ndi vi dual or networked Uni x workstati oJ l s. MATLAB takcs
careof this issu. The book assumes you have a worki ng and appropri atel y Ii censed
copy of MATLAB in the machi ne in ont of you. On your personal machi ne for
installingMATI . AB fol l owthe i nstructi ons that accompani dthebox. Typi cal l y e
systemadmi ni strator handl es network installations.
Thi s book uses MATt. AB Versi on5.2(until Chapter 4) and Versi on5.3( erest).
Versi on 6 has been shi ppi ng si ncethefall of 2000. MA ABversi on5.0and above
shoul dwork. If MATI . ABhas changed any built-incommands innewer versi ons
theone edin thebook it shoul dprovi dei nformati onon theavailabilityof newer
versi on of commands (or l hrowan error a rare case). The MATLAB code and
exampl es were created on an Inlel Penti umPro 200 runni ng the Wi ndows NT 4
operati ngsysl em. The earlier codes were al sotestedon a DEC Al pha server runni ng
OSFl V4. The MATI..ABversi onon theserver was 5. 1.
In organi zati ons and instilutions MATt. AB software is usual l y l i censed on a
subscri pti on basis. Users therefore hav the 1atest versi on of the software in these
situations. Insucha casethenewversi on(at ti methebook is first publ i shed) will
beVersi on6 (or Rel easeR 12). Thi s book is basedon Versi on5 (Rel easeRI 1). le
code d programmi ngi nstructi ons inthebook shou1dexecutewi thoUl probl em
in the newversi on. In the newRel ease R12 on the PC p1atform the defaul t
appearanceof MATI..ABis a composi te of several wi ndows presenti ngdi fferent pi eces
of i nformati on. Thi s is shown inFi gure1.8. Thi s c becustomi zedthroughthemenu
b by Vi ewDesktopLayout Command Wi ndowOnl y whi ch al socoponds
to e previ ous 1ease. It is shown in Fi gure 1. 9. lis vi ewis si mpl e and less
di stracti ngfor getti ngfami l i ar wi MATl ..AB. Note eMATt. AB command prompt
is >>. Thi s wi ndowdel i vers standard operati ng systemcommands for savi ng
pri nti ng openi ng files and so on l hrough menus and a command bar. In order to
T
h'
m.
f.
1.2 I NTRODucnONTOMATl . AB 27
F1gure1.8 MATLAB openi ngview.
createfiles (or code) another graphi ca1wi ndowis neded. Thi s is eMATLABMfile
edi tor/debugger wi ndowand is shown in Fi gu1. 10. The edi tor uses co10r for
enhanci ngcode readability. Each of t hesewi ndows spawn addi ti ona1di a10gboxes (if
n ec) f io r . s ctttin 1gfie atu re e sd u I ri ir
workspace browser.
Whi lethisisnot somuch asaninstallationissue it wou1d be appropri al elodi scuss e
way k pl aI 1s towri andexec MATI . ABC Gen y erearetwo ways to
work MATI . AB: i nteracti vel yandthro 'pts. Ininteractivemode. r nn1ands
areente d d utedone at a ti meintheMATt. AB cOrnn1and wi ndow. Once sess10n
is over andMATt. AB exited thereis no i nformati on1 tai nedon set of commands
were used and what worked (unless a choi ce was made to save the session). Large
sea uen s of cOrnn1aI 1ds areusual l y di cu 1u.ok P ack of inE eracUv sess10n.
4o o p t i
fileinMATI . ABThi siscalleda script m-ftle. Most MATLAB fil lledm-ftles. The
m is a meextBI ESi onusual l ymserved for MA-I -LAB. Any code changEsmemade thmush
eeditor. thechanges to efilesaved and e fi l cexecu dinMATt. AB again. Excutin
M ATt.A Btl ghsc dp t mle e si spredo m 1 lin
tore vert toi nteracti vemode any ti me.
1.2.3 Usi ng MATLABthe Fi rst Ti me
E g
MA T wil be used inte i ve1y in is section. The au.thor very ny
recornn1ends a hands-on approachtounderstandi ngMATt. ABThis means typmg tt1e
codes yourself. It a1soinc1udes undersl andi ngthesyntax rrorsanddebuggi ng and
28 I NTRODUCTI ON
i s i s a Cl asstoomLi cense fOt i nsttucti onal use onl y.
Reseatch and com etci al use i s ptohi b1ted.
To qet statted sel ect " HATLAB Hel p" f tom.the Hel p menu.
Fl gure1.9 MATLAB Command wl ndow.
Opti m1zati on 1 1
'iI J ohn
Dt. P. VenkataJ ; amenl
F1gure 1.10 MaUab Edi tor/Debugger wi ndow.
1.2 I NTRODUCTI ONTOMAl UB 29
recti ring ecode. The author is not aware of a si ngl eexampl e where programmi ng
was l earnedwi thout thereader typi ng de. l n is bk MATLAB code segments are
couri er font wi th e bol dstyl e lI sedfor emphasi zi ng commands or other pi eces of
i nformati on. Anythi ng el se is recommendati ons suggesti ons or exerci ses. The
pl acehol der for command is initalic.
l
Bef or e We Start: The fol owi ng punctuati on d speci al characters are worth
noti ng:
Al text after the % si gn is consi dered a
comment. MATLAB i gnores anythi ng to
the ri ght of the % si gn. Comment s are
used l i beral l y i n this book to i ndi cate
why' s and what' s of command or code
usage. Do not overl ook them. You do not
have tocode t hemin however probabl y
t wo weeks later you wil not r emember
why you devel oped that parti cul ar pi ece
of code.
Thi s is thedefaul t MATLAB pr ompt.
A semi col on at theend of a line prevents
MA ABf romechoi ng thei nformati onyou
enter on thescreen.
A comma will cause the i nformati on to
A
.
J
h.
U
>>
F
t
J
-
echo. Defaul t is a comma.

A successi onof threeperi ods at theend of
the line i nf orms MATLAB code will
conti nue t he next line. You cannot split
a vari abl e name across t wo l i nes. You
cannot conti nue a comment on another
Ii n.
You can s to 'p MATLAB executi on and get
back t h c omm!md pr omp t b y typi ng ^c
(Cl- C)by h 01di n gdown
tog e er.
WilI di spl ay i nformati on on the vari ous
ways thecommand canbe used. Thi s is the
qui ckest way touse help.
Opens a hel p text wi ndowthat provi des
mor e i nformati on on all of the MATLAB
resources i nstal l edon your system.
Provi des hel pusi nga browser wi ndow.
v
l
r
E
Y
H
H
E
J
r
p
L
i
A
c
>> hel p command_name
>> hel pwi n
>> hel pdesk
30 I NTAODUCTI ON
For thoseof you programmi ng for thefirst ti me theequal tosi gn(=) has a very speci al
meani ng inall programmi ng l anguages incJ udi ng MATLAB. It is cal l edtheassi gnment
operal or. le vari abl eon thc I cft-handsi deof thesi gnis assi gnedtheval ue of e
ri ght-handside. Thc actual eqllal l Ooperati oninMATLAB is usual l y accompl i shed by
a doubl e cqual tosi gn(==).
Some Addi ti onsl Feat ures
MA AB is case sensi ti ve. An a is di fferent than A. Thi s is di ffi cul t for
persons who arc uscd to FORTRAN. Al l bui l t-i n MATLAB commands arc in
l owercase.
MATLAB does not necd a typedefi ni ti onor a di mensi onstatement toi ntroduce
vari abl es. It automati cal l y cr tes one on first encounter incode. The typeis
assi gnedincontext
Vari abl names start wi tha letter and contai nup to31cbaractcrs (onl y Icttrs
digits and underscorc).
MATLAB uscs some built-invari abl enames. Avoi d usi ngbuilt-invari abl enam.
Al l numbers arc storcd i nternal l y usi ng the l ong format spcci fi cd by I EEE
f1oati ng-poi nt standard. Thesenumbers have roughl y a preci si onof 16deci mal
digits. They range roughl y between 10E- 308 and 10E+308. Howeve yare
di spl ayeddi ffercntl ydependi ng on thecontext.
MATLAB uses conventi onal deci mal notati on for numbers usi ng deci mal
poi nts and l eadi ng si gns opti onal l y for exampl e 1 - 9 +9. 0 0. 001
99. 9999.
Sci enti fi c notati onis cxprcssedwi ththeletter e for cxampl e 2. 0e-03 1. 07e23
1. 732e+03.
l magi nary numbers useei ther i or j a suffi x for exampl e li -3.l 4j 3e5i .
Operators: The fol l owi ngarc theari thmeti c operators inMATLAB.
+ Addi ti on(wben addi ng matri ccs/arrays sizemust match)
Sub acti on(same as above)
* Mul ti pl i cati on (the subscri pts of arrays must be consi stent when
mul ti pl yi ngthem)
/ Di vi si on
Power
Compl cx co ugat transpose( 0 aytranspose)
Inthecaseof arrays each of theseoperators can be used wi aperi odprefi xedto e
operator for cxampl e ( . * ) or ( ^ ) or ( 1) . has a speci al meani ng inMATLAB. It
i mp1 i es l ement-by-el ement operati on. It is uscful for qui ck computati on. It has no
"
1.2 I NTAODUCTl ON TOMATl AD 31
rel evanceinmathemati cs or anywhere else. We wiJ1usc it a lot inthenext chapter for
gClI erati ngdatafor graphi cal op zati on.
An I nteractl veSessl on: S MA AB. On PCprcss theStart button. Press the
Pr ogr ambutton. Prcss thc MATLAB ' gramGroup. Press MA TLAB. The MATLAB
Command wi ndowopens up and after somc i ntroductorymessages E will be a MATLAB
prompti ndi dby doubl ef orward arrow. lis is i ndi cati on MATLABis ready to
aeptyour mmands. On thenetworked s tionstypi ngmatJab at thewi ndowprompt
shoul ddo thesame. Note that theMATLAB programmust be inthepa as def med in e
.Iogin or. h file. Ask thelabassistant for help. The standardUNI X proccd
t rnat theend of elincor befo ecomment . The commcnts rclatc towhat is
bei ngtyped dwhat isdi spl ayedintheMATLAB command wi ndow. FI 1frcc your
own variatiosS. Infact tounde dandrei nf o commandsit is commended
you make up your own exampl es often.
>>a 1. 0; b 2. 0; c 3. 0 d 4. 0; e 5. 0
> > why di d onl y c and e echo on t he scr een?
>> who l i st s al l t he var i abl es i n t he wor kspace
>> a gi ves t he val ue st or ed i n a
>>A 1. 5; var i abl e A
>> a A % case mat t er s
>>one a; t wo b; t hr ee C;
> > assi gni ng val ues t o new var i abl es
>>f our d; f i ve e; si x pi ; val ue of pi avai l abl e
>>A1 [a b c ;' d e f] A1 i s a 2 by 3 mat r i x
>> % space or comma separ at es col umns
>> % semi - col on separ at es r ows
>>A1( 22) accesses t he Mat r i x el ement on t he
>> % second r ow and second col umn
>> si ze (A1l gi ves you t he si ze of t he mat r i x
> > ( r ow col umI l sl
>>AA1 si ze( A1l What shoul d happen her e?
> > f r ompr evi ous sat ement t he si ze of A1
> > cont ai ns t wo number s or gani zed as a r ow
> > mat r i x. Thi s i s assi gned t o AA1
>> si ze( AA1l AA1 i s a one by t wo mat r i x
>> A1' t hi s t r ansposes t he mat r i x A1
>>B1 A1' t he t r anspose of mat r i x A1
> > i s assi gned t o B1. B1 i s a t hr ee by t wo mat r i x
>> C1 ' " A1 * B1 Si nce A1 and B1 ar e mat r i ces t hi s
> > i 8 a mat r i x mul t i pl i cat i on
> > Shoul d t hi s mul t i pl i cat i on be al l owed?
> > consi der mat r i x mul t i pl i cat i on i n C or
32 I NTRODUCTI ON
>> % For t r an
> > t he power of MATLAB der i ves f r omi t s
> > abi l i t y t o handl e mat r i ces l i ke number s
>>C2 81 * A1 How about t hi s?
>> C1 * C2 What about t hi s?
>> % r ead t he er r or message
> > i t i s qui t e i nf or mat i ve
>>D1 [1 2]' D1 i s a col umn vect or
>>C3 [C1 D1] C1 i s augment ed by an ext r a col umn
>>C3 [C3 C2( 3 : ) ] Not e i s an assi gnment
> > means do t he r i ght hand si de and over wr i t e t he
> > ol d i nf or mat i on i n C3 wi t h t he r esul t
> > of t he r i ght
> > hand si de cal cul at i on
> > On t he r i ght you ar e addi ng a r ow t o cur r ent
> > mat r i x C3. Thi s r ow has t he val ue of t he t hi r d
> > r ow of C2 - Not i ce t he pr ocedur e of
> > i dent i f yi ng t he t hi r d r ow. The col on
>> % r epr esent s al l t he col umns
>>C4 C2 * C3 per mi ssi bl e mul t i pl i cat i on
> > Not e t he pr esence of a scal i ng f act or
> > i n t he di spl ayed out put
>>CS C2 * C3 seems t o mul t i pl yl
> > I s t her e a di f f er ence bet ween C4 and CS?
>> % The * r epr esent s t he pr oduct of each el ement
> > of C2 mul t i pl i ed wi t h t he cor r espondi ng
> > el ement of C3
>>C6 i nver se( C2) % f i nd t he i nver se of C2
> > appar ent l y i nver se i s not a command i n MATLAB
> > i f command name i s known i t i s easy t o obai n
> > hel p
>> l ookf or i nver se t hi s command wi l l f i nd al l f i l es
> > wher e i t comes acr oss t he wor d "i nver se" i n
> > t he i ni t i al comment l i nes
> > The command we need appear s t o be I NV whi ch
> > says I nver se of a Mat r i x
>> % The act ual command i s i n l ower case. To f i nd
> > out how t o use i t - Now
>> hel p i nv % shows how t o use t he command
>> i nv( C2) i nver se of C2
>> f or i ' " 1: 20
f ( i ) i "' 2;
end
> > Thi s i s an exampl e of a f or l oop
1. 2 I NTRODUCnONTOMAn. AD 33
> > t he i ndex r anges f r om1 t o 20 i n st eps of
> > 1( def aul t )
> > t he l oop i s t er mi nat ed wi t h "end"
> > t he pr ompt does not appear unt i l "end" i s
> > ent er ed
>>pl ot ( si n( 0. 01*f ) cos( 0. 03 f) )
>>xl abel ( ' si n( O. Ol *f ) ' ) st r i ngs appear i n si ngl e
> > quot es
>>yl abel ( ' cos( 0. 03*f ) ' )
>> l egend ( ' Exampl e' )
>> t i t l e ( ' A Pl ot Exampl e' )
>> gr i d
> > The pr evi ous set of commands wi l l cr eat e pl ot
> > l abel axes wr i t e a l egend t i t l e and gr i d t he
> > pl ot
>> exi t f i ni shed wi h MATLAB
Thi s compl etes thefirst sessi onwi thMATLAB. Addi ti ona1commands and features will
be encounteredthroughout thebook. In is sessi on it is evi dent MA . ABalows
easy mani pul ati onof matri ces defi ni tel yinrelationtoother programmi ng l anguages.
Pl otti ngis not difficult either. Theseadvantages ar qui tesubstanti al inthesubj ect of
opti mi zati on. 1n next sessi on we wil1 use the edi tor 10 I1ccompl i sh e same
throughscripts. Thi s sessi oni ntroduced
MATLAB Command wi ndowand Workspace
Vari abl eassi gnment
Basi c matri x operati ons
Accessi ngrows and col umns
Suppressi ngechoes
who i nversecommands
. * mul ti pl i cati on
Basi c pl otti ngcommands
1.2.4 Usi ng t he Edi tor
Inthis secti on we will usetheeditor to te druna MATLAB a script file. Nor ma1ly
eeditor isusedtogeneratetwo ki nds of MA . ABfiles. 1 efilesaretermedscriptftles
andfuncti onftl es. AI oughbo ofthsefiI es con nMATLAB commands likeeones
we have used thesecondtype of fiI es needs tobe organi zedina speci fi edformat . Bo
file ty s shoul dhave theextensi on .m. Al though thesefiles areASCI I text fi I e
generi c . mshoul dbe edbeca eMATLABs hes for i sex si on.hi sextensi onis
quet oMA. AB. 1c script filecontai ns a Ii st of MA TLABcommands areexe d
insequence. This is different mtheinteractivesessi onof theprevi ous sectionwhere
34 I NTRODUCTI ON
MA 1. AB ndedto chconunand i mmedi atel y. The script fileis more useful when
therear many conun ds needtobe ex utedtoacmpl i shsome obj ecti ve like
runni ngan opti mi zati ontechni que. It is i mportant toremember that MATLAB aI l ows you
toswi tchback i nteracti vemode at any ti me by just typi ngcommands in eworkspace
wi ndowlikein ep vi ous secti on.
MATLAB provi des an edi tor for creati ngthesefiles on thePC pl atform. Thi s edi tor
uses col or toi denti fyMATLAB statements and el ements. It provi des ecurrent vaI ues
of the vari abl es (after eyareavai l abl eintheworkspace) when themouse is over
the vari abl e name intheedi tor. Thel t woways to access eedi tor through the
MATLAB Command wi ndowon the PC. 5tart MATLAB. Thi s wi l l open a MATLAB
Command or Workspace wi ndow. Inthis wi ndowtheedi tor can be startedby usi ng
the menu or e tool bar. On the Fi l e menu cl i ck on New and choose M- fiI c.
Al tematel y cl i ckon theI eftmost i conon thetool bar ( etool ti preads New Fi l e). The
i confor theedi tor can aI sobe pl aced on thedesktop inwhi ch case theedi tor can be
startedby doubl e-cl i cki ngthe i con. In this event a MATl AB Command wi ndowwi l l
not be opened. The edi tor provi des its own wi ndowfor enteri ng scri pt statements.
At this poi nt we areready touse theedi tor. Make sureyou ad comment s and
understand t hem as theseprocedu s wi l l be used often. Usi ng theedi tor i mpl i es we
wi l l be worki ng wi thm-fi l es.
The commands ar esame as in1h i nteracti vesessi onexcept thereis no MATLAB
prompt prefi xi ngtheexpressi ons. To execute thesecommands you wiIl have tosave
t hemtoafi l . Let us call thefi l escri ptl .m. The. mextensi onneed not be typed i f you
are usi ng MATLAB edi tor. You can save the file usi ng the Save or Save As
command f rommost edi tors. It is i mport t toknowthe full pa to is file. Let us
assume thepathfor efileis C: OpCbookChl scri ptl .m. Not e epa hereis
speci fi edas a PC pathdescri pti on. The reason we need is i nformati onis toi nf orm
MATLAB where tofi nd efile. We do is in eMATLAB Command wi ndow.
Thi s i mpl i es we shoul d have eMATLAB Command wi ndowopen. On PCs we start
MATLAB through the i cons. I nthis book we wi l l accompl i sh most tasks by typi ng in
theCommand wi ndowor through programmi ng. l nmany i nstances ereareal temate
ways usi ng menu i tems or e tool bar. Thi s is left to the reader to experi ment and
di scover. In the MATLAB Command wi ndowus e addpat h command to i nf orm
MATLAB of the l ocati onof thefile:
>> addpat h C: Opt _bookCh1
The scri pt th81wi l l be createdand saved inSCT tl .mcan be runby typi ng(notethat
eextensi on is omi tted)
>> scr i pt l
To understand and associ atewi th theprograrnmi ng concepts embedded in. thescri pt
parti cul arl y for persons wi thIi mi ted prograrnmi ng experi ence it is r ecommended to
run thescri pt after a bl ock of statements have been wri ttenra er thantypi ng efile
1.2 I NTRODUCTI ONTOMATLAD 35
inits enti rety. Pri or torunni ng thescri pt you wi l l have tosave thescri pt each ti me 50
the changes in f i J e are recorded and the changes are current. Another
recommendati on is todel i beratel ymi s'pesome statements and attempt todebug the
error generatedby eMATLAs debugger duri ng uti on.
Creati ng the Scr t M-.flle (The fol l owi ngwi l l be typedl sav ina fil .)
exampl e of usi ng scr i pt
A1 [1 2 31;
A2 [4 5 61;
t he commands not t er mi nat ed wi t h semi - col on wi l l
di spl ay i nf or mat i on on t he scr een
A = [ A1; A2]
B [A1' A2' ]
C A* B
now r e- cr eat e t he mat r i x and per f or mmat r i x
mul t i pl i cat i on as i n ot her pr ogr ammi ng l anguages
exampl e of f or l oop
f or i 1 3 var i abl e i r anges f r om1 t o 3 i n
st eps of 1 ( def aul t )
a1 (1 i ) i ;
end l oops mus t be cl osed wi t h end
a1
f or i 6: - 1: 4 not e l oop i s decr ement ed
end
a2
a2( 1i - 3) i ; f i l l i ng vect or f r omr ear
cr eat i ng mat r i x A and B ( cal l ed AA and BB her e)
f or i 1: 3
AA( l i ) a1 (1 i ) ; assi gn a1 t o AA.
AA( 2 i ) a2 (1 i ) ;
BB (i 1) a1 (1 i ) ;
BB ( i 2 ) a2 (1 i ) ;
end
t he same can be accompl i shed b y AA (1: ) a1
wi t hout t he f or l oop
AA pr i nt t he val ue of AA i n t he wi ndow
BB
who l i st al l t he var i abl es i n t he wor kspace
consi der code f or Mat r i x mul t i pl i cat i on
% whi ch Mat l ab does s o easi l y
36 I NTRODUCTI ON
mul t i pl y t wo mat r i ces ( col umn of f i r st mar i x must
mat ch r ow of second mat r i x)
szAA si ze( AA) si ze of AA
szBB si ze (BB) ;
i f ( szAA( l 2) szBB( l l ))
onl y i n col umn of AA mat ch t he r ows of BB
f or i l : szAA( l l )
f or j 1: szBB( l 2)
CC ( i j ) 0 . 0 ; i ni t i al i ze val ue t o zer o
f or k 1: szAA( l 2)
e nd
end
CC( i j ) c CC( i j ) + AA( i k) *BB( kj ) ;
k - l oop
j - l oop
end
end
CC
i - l oop
i f - l oop
Not e t he power of M T ~ B der i ves f r omi t s abi l i t y t o
handl e mat r i ces ver y easi l y
t hi s compl et es t he scr i pt sessi on
Save theabove fi) e (scri ptl. m). Add thedi rectory totheMATLAB path i ndi cated
before. Run thescript fileby typi ngscri ptJ at thecommand pr ompt . Thc commands
shoul d all execute and you shoul dfmal l y seetheMATLAB prompt inthc Command
wi ndow.
Note: You can al ways rcvert to the i ntcracti ve mode by di rectl y enteri ng
commandsi n eMATLAB wi ndowafter theprompt . IntheCommand wi ndow:
>> who
>> cl ear C % di scar ds t he var i abl e C f r omt he wor kspace
> > use wi t h caut i on. Val ues cannot be r ecover ed
>> hel p cl ear
>> exi t
This sessi onil1ustrated:
Usc of th edi tor
Creati nga script
Runni ng a scIi pt
Error debuggi ng (recommcndcd activity)
Programmi ngconcepts
. Loop constructs if and for l oops
L oopvari abl eand i ncrements
Array access
1. 2 I NTRODUCTI ONTOMA AD 37
Cl ear stal ement
1.2.5 Creatl ng a Code Snl ppet
Inthis secti on we will exami ne the0 er typc of m-fi l whi chis cal1ed thefuncti on
m-fi l e. For those fami l i ar wi l h other programmi ng l anguages Ii ke C J ava or
F'ORTRAN esefiles represent functi ons or subrouti nes. They arepri mari l y uscdt'O
handl e s'Ome speci fi c types of cal cul ati ons. Thcy al sopr'Ovi dea way for themodul ar
devel opment of code as wel l codereuse. These code modul es ar usedby bei ng
cal l cdor refcrredinother secti ons of thecodc saythrougha script filewe l ookedat
earlier. The code that calls thc functi onm-fi l eiscal l edthecal l i ngprogram/code. The
essenti al paramctcrs indcvel 'Opi ngthefuncti onm-fi l c arc (1) what i npu1is necessary
for the caI cul ati ons (2) what speci fi c cal cul ati ons must take pl ace and (3) what
i nformati onmust be retumed tothecal l i ngpr'Ogram. MATLAB requi res theSl ructure
of efuncti onmfiletofol l owa prescri bedf ormat.
We will use1heedi 10r 10devel 'OP a func1i onm-fi l etha1wiI1pcrforma pol ynomi al
curvefi t . II requi res a set of datarepresen1i nga curve that needs tobe fi1 10ge1her
wi the order of thepol ynomi al to be fi t. Thi s exerci sc is cal l cd curve fiuing. In
Chaptcr 6 such a probl emwill be i den1i fi cd as a probl emin unconstrai ned
opti mi zati on. For now the cal cul a1i ons necessary to accompl i sh the exerci se are
consi deredknown. It i nvol ves sol vi nga linear equati onwi th enomtal matrl x and a
ri ght-h dvcctor obtai nedusi ngthedatapoints. 'he output f romthem-fi l ewill be
ecoe icients representi ng epol ynorni al .
Before we start todevel opthecode thefirst lineof this filemust be f ormatted as
speci fi edby MATLAB. Inthefirst line thefirst w'Ordstartirig um efirst col umn is
c wordfuncti on. It is fol l owedby theset of rel umparameters (retumval ). Next
ename (mypol yfi t) 'Of thefuncti onwi ththeparameters passedtothc functi onwi thi n
p eses (XY N). lefilemust besavedas name. m(mypol yfi t.m). The comme
be1ween thefirs1lineand thefirs1executabl c statement wiI1appc if you 1ypehel p
lI ame (hel p mypol yfi t) in thc Command wi ndow. The reason for thc narne
mypol yfi t . m is that MATLAB has a bui1t infunclI on pol yfi t. Open theedi tor 10create
thefilecontai ni ngthefol l owi ngi nformati on:
f unct i on r et ur nval mypol yf i t ( XY N)
These comment s wi l l appear when t he user t ypes

hel p mypol yf i t i n t he Command wi ndow
Thi s space i s i nt ended t o i nf or mt he user how t o
i nt er act wi t h t he pr ogr am what i t does
what ar e t he i nput and out put par amet er s
Least squar e er r or f i t of pol ynomi al of or der N
x y - Dat a f ound i n XY
r et ur ns t he vect or of coef f i ci ent s st ar t i ng f r om
% t he const ant t er m
. i : j ' J S I
. .
: 1 '.
39
end end i 100p
i f t he x- poi nt s ar e di st i nct t hen i nver se i s not a
pr ob1em
1.2 I NTROOUCTI ONTOMAl 1. AB
l NTRODucnON
or i 1: N+1
a( i ) = 0. 0; i ni t i a1i ze t he coef f i ci ent t o zer o
end
38
r et ur nva1 i nv( A) *b' ;
Save thefileas mypol yfu. m. To usethefuncti onwe will need tocreatesome xydata
dthcncall themypol yfi t functi on. S MA1LABin c di rectory t mypol yfit. m
resi des in. or add thedi rectory to thepath. IntheM A11. AB Command wi ndow: The
fol1owi ng cod is typedinthecommand wi ndow.
J
A

X

*

3
x

*

x

x

*

n
U
2
J

+

.
o
X
2

4
=
=

1
0
J
)

u
z
1
1
a
-
4
r
y
y

o
x
x
x

a
e
sz si ze (XY) ;
NDATA sZ( l l ) ; number of dat a poi nt s - r ows 0
xy mat r i x
i f NDATA == 0
f pr i nt f ( ' Ther e i s no dat a t o f i t ' ) ;
r eur nva1 a; zer o va1ue r et ur ned
r et ur n; % r et ur n back t o ca11i ng pr ogr am
end
>>coef f mypol yf i t ( XY13)
> > a cubi c po1ynomi a1 was de1i ber at e1y cr eat ed t o
>> % check t he r esul t s. You shou1d get back
> > t he coef f i ci ent s you used t o gener at e t he cur ve
> > t hi s i s a good t est of t he pr ogr am
> > Let us cr eat e anot her examp1e
i f NDATA < 2* N
f pri nt f ' ( Too f ew dat a poi nt s f or f i t' )
r et ur nva1 a;
r eur n
end
>> XY2 ( : 1) XYl ( : 1) ; same f i r st co1umn
>> XY2 ( : 2) 20 + exp ( XY2 ( : 1) )
> > not e t he power of Mat 1ab - a vect or f i 11
> > a new set of xy- dat a
The pr ocessi ng st ar t s her e.
The coef f i ci ent s ar e obt ai ned as sol ut i on t o t he
Li near A1gebr a pr ob1em [A] [c] [b]
Mat r i x [A] i s t he Nor ma1 Mat r i x whi ch cont ai ns
t he sumof t he power s of t he i ndependent var i ab1e
[A] i s symmet r i c
u
s
m
u
a
d
u
>>coef f 1 mypol yf i t ( XY2

>> hel p mypol yi t
> > you shou1d see t he st at ement s you set up
> > I n t he next exer ci se we wi 11 cr eat e a scr i pt
> > f i 1e t hat wi 11 r un a pr ogr amf or po1ynomi a1
> > cur ve- f i t t i ng we wi 11 save XY2 so t hat i t can
> > be used agai n
agai n cubi c po1y. 3)
100p over al 1 dat a poi nt s
var i ab1e i s "11" (e1) (e)
XY( 112) *XY( 111) ^( i - 1) ;
11
f or i 1: N+1;
b( i ) 0. 0;
f or 11 l : NDATA;
b( i ) +
100p
b( i )
end
save C: Opt _bookChl XY2. dat - asci i doub1e
t hi s wi 11 save t he f i 1e as an asci i t ext f i 1e
wi t h doub1e pr eci si on va1ues
Thi s conc1udes e exerci se where a code s p W written to lc ate E

Cfficients of thepol ynooual usedtofit a curv tosome data. etypeof filc is thc
f or j 1: N+1;
i f j >= i ca1cu1at i ng upper di agona1 t er ms
power (i - 1) + (j 1) ;
A( i j ) 0. 0 i ni t i al i ze
f or k l : NDATA; sumover dat a poi nt s
A( i j ) = A( i j ) + XY( kl ) ^power ;
end k l oop
end c10se i f st at ement
A i ) = A( i j ) exp10i t i ng Mat r i x symmet r y
end end j 100p
40 I NTRODUCTI ON
fi mcti onm-fi le. It n beusedina certainway. lecode was tedusi nga cubi c
pol ynomi al . Nonpol ynomi al datawcre a1sotested. The datawere savedfor later use.
1. 2. 6 Creatl ng a pr ogr am
h is secti ona program will ad dat a curvefit thedatausi nga pol ynomi al and
mpare original and fi dda pru llywill bedevel oped. The tawill beread
usi nga User I n( UI ) wi ndowand a di al ogbox. The code will be devel oped a
script filc. This gi ves us anopportuni ty torevisit most of thecode we used rlier for
i nforcement . Wc will alsousethcfuncti onm- fiI e createdin previ ous exercise.
ler several ways for you or theusers tointeract wi ththecode you devel op. 1
basi c method is toprompt users for i nformati onat prompt intheCommand wi ndow.
is equi ckest . Thi s is probabl y what you will usewhen devel opi ngthecode. Once
thedehas beentested depencl i ngon useful ness it mi ght berel evant consi der usi ng
more sophi sti catedcustomeI ements I i ke i nput boxes and filesel ecti onboxes. lis book
will conti nuetousetheseel ements throughout as appropri ate. Whilethei nput el ements
us in is code arcncwcommands therest of theprogramwill mostl y usecommands
t havc bccni ntrodu d rlier. Ins uential order theevcnl s in sprogramare: (1) to
readthe datasavedearlier usi nga filcscl ecti onbox (2) toread eorder of fit usi ng
an Input di al ogbox (3) tousethemypol yfi t functi ondcvel opedinthelast sectionto
obta thec fficients(4) toobtai n ec ina of thefittcdc ve(5) tograprucal l y
compare original dfitt dat a and (6) ton efittedauracyon efi m
itsel newscript filewill be cal l edpr og...J1j m.
Start thecdi tor tocreatethefilc cal l edp'
pr ogr amf or f i t t i ng a po1ynomi a1 cur ve t o x y dat a
f r om
App1i ed Opt i mi zat i on usi ng Mat 1ab
Dr . P. Venkat ar aman
Chapt er 1 Sect i on 1. 2. 6
The pr ogr am100ks f or a f i 1e wi t h t wo co1umn asci i
dat a wi t h ext ensi on . dat . The or der of t he cur ve i s
obt ai ned f r omuser . The or i gi na1 and f i t t ed dat a
ar e compar ed wi t h r e1evant i nf or mat i on di sp1ayed on
t he same f i gur e. The pr ogr amdemonst r at es t he use
of t he f i 1e se1ect i on box an i nput di a10g box
cr eat i ng speci a1 t ext st r i ngs and di sp1ayi ng t hem
[ f i 1epat h] =ui get f i 1e( ' *. dat ' ' A11 Fi 1es' 200200) ;
ui get f i 1e opens a f i 1e se1ect i on box
checkout he1p ui get f i 1e
t he st r i ng var i abl e f i 1e wi l l hol d t he f i l ename
1.2 I NTRODUCTI ONTOMATLAB 41
t he st r i var i ab1e pat h wi 11 have t he pat h
i nf or mat i on t he def au1t di r ect or y poi nt ed wi 11
di f f er dependi ng on t he p1at f or m
i f i sst r ( f i l e) i f a f i 1e i s se1ect ed
10adpat hf i 1e [ ' l oad' pat h f i 1e] ;
10adpat hf i 1e i s a st i ng var i ab1e concat enat ed
wi t h t hr ee st r i ngs "10ad " pat h and f i 1e
not e t he space af t er 10ad i s i mpor t ant
eval ( 10adpat hf i 1e) ;
eva1uat es t he st r i ng enc10sed - whi ch i nc1udes t he
Mat 1ab command 10ad. Thi s wi 11 i mpor t t he xy- dat a
t he dat a wi 11 be avai 1ab1e i n t he wor kspace as a
var i ab1e wi t h t he same name as t he f i 1ename
wi t hout t he ext ensi on ( t hi s assumes you se1ect ed
t he xy- dat a usi ng t he f i 1e se1ect i on box)
newname st r r ep( f i 1e' . dat ' " ) ;
newname i s a st r i ng var i ab1e whi ch cont ai ns t he
st r i ng f i l e st r i pped of t he . da ext ensi on. Thi
i s a st r i ng r ep1acement command
x eval ( newname) ; assi gns t he i mpor t ed dat a t o x
j ust f or conveni ence. The above st ep i s not
necessar y
NDATA l engt h( x( : 1) ) ; number of dat a poi nt s
cl ear pat h 10adpat hf i 1e newname
get r i d of t hese var i ab1es t o r ecover memor y
end
Not e: i f a f i 1e i s not se1ect ed not hi ng i s bei ng
% done
Use of an i nput di a10g box t o get t he or der
of po1ynomi a1 t o be f i t t ed
PROMPT {' Ent er t he Or der of t he Cur ve' };
PROMPT i s a st r i ng Ar r ay wi t h one e1er
not e t he cur l y br acket s
TI TLE ' Or der of t he Po1ynomi a1 t o be Fi t t ed' ;
a st r i ng var i ab1e
LI NENO 1; a dat a var i ab1e
get va1 i nput dl g( PROMPT TTLE LI NENO) ;
42 I NTRODUCTI ON
t he i nput di al og capt ur es t he user i nput i n get val
ge val i s a st r i ng Ar r ay
check hel p i nput dl g f or mor e i nf or mat i on
no st r 2num( get val {l l }) ;
t he st r i ng i s conver t ed t o a number - t he or der
cl ear PROMPT TTLE LNENO del et i ng var i abl es
% cal l f unct i on mypol yf i t and obt ai n t he coef f i ci ent s
coef f mypol yf i t ( xno) ;
gener at e t he f i t t ed cur ve and obt ai n t he squar ed
er r or
er r 2 0. 0 i'
f or i 1 : NDATA f or each dat a poi nt
f or j l : no + 1
a( l j ) = x( i l ) "( j - l ) ;
end
y( i ) a*coef f ; he dat a f or he f i t ed cur ve
er r 2 er r 2 + ( x( i 2) - y( i ) ) *( x( i 2) - y( i ) ) ;
he squar e er r or
end
pl ot t i ng
pl ot (x ( : 1) x ( : 2) ro' x ( : 1) y b- ' ) ;
% or i gi nal dat a ar e r ed o' s
f i t t ed dat a i s bl ue sol i d l i ne
xl abel ( x' ) i
yl abel ( y' ) ;
st r or der set st r ( num2st r ( no) ) ;
conver t t he or der of cur ve o a st r i ng
same as get val i f you have not cl ear ed i t
set st r assi gns t he st r i ng t o st r or der
t i t l est r [ ' pol ynomi al cur vef i t or or der '
s r or der ' of f i l e' f i l e) i
t he t hr ee dot s at t he end ar e cont i nuat i on mar ks
t he t i t l e wi l l have t he or der and t he f i l e name
t i t l e ( t i t l est r )
l egend( ' or i gi nal dat a' ' f i t t ed dat a' ) ;
er r st r l num2st r ( er r 2) ;
er r st r 2 [ ' squar ed er or er r st r l ]
gt ext ( er r st r 2) ;
1.2 I NTRODUCTI ONTOMATl A9 43
t hi s pl aces t he st r i ng er r st r 2 whi ch i s obt ai ned
by combi ni ng t he st r i ng ' squar ed er r or ' wi t h
t he st r i ng r epr esent i ng t he val ue of t he er r or
wher ever t he mouse i s cl i cked on t he pl ot .
movi ng t he mouse over t he f i gur e you shoul d
see l ocat i on cr oss- hai r s
cl ear st r or der t i t l est r er r st r l er r st r 2 a y x i j
cl ear NDATA no coef f XY2 f i l e er r 2 get val
gr i d
Thi s f i ni shes t he exer ci se
Run theprogra byfirst runni ngMATLAB in di rectorywhere these es are. or
addi ng the p to 1ate At e command prompt type pr oLP . The
programshoul d EXEcute requi ri ng user i nput through the tile sel ecti on bOLi nput
di al og and final1y di spl ayi ngFi gure 1.11. leappe ancemay be slightly di fferent
dependi ng on pl a' ormMATLAB is bei ngrun.
M Thi s fi ni shes the MATLAB secti on of the chapter. Tl msecti on lIas i ntroduced
ATLABin a robust manner. A broad mng of pr ogmmmi ng experi ence has been
ml tl ated in this chapter- AH mw commands have hen i derui ncd wi th a bdef
expl anati on in comments. It is i mportant youusetheOPPOl tuni ty totypein
ECodE yoursel f;That is theonl y way theuseof MAT1AB will become fami l i ar. The
practi cea1sowi l l l eadtofewer syntax erro Thewri ti ngof code will si gni fi cantl y
i mprove your abi1i ty to debug and troubl eshoot. While is chapter empl oyed a
Pol ynomi al curve fIt of order 3 of fi l eXY2.dat
5
x
Fl gure1.11 Origlnal andfllleddala.
t2J!.: ;l; ' f. l i 44uu 1ui k J t : i
44 I NTRODUCTI ON
seoarate secti on on the use of MATLAB out of n sitysub uent c hapmMe
CMa i zedby a compl rati onof the us m of MAT .TLAB
opti mization.
PROBL EMS
uruuz;ztm: ; alr; ::;t;
mathemati cal modeJ.
1.2 I denti fy several possi bl eopti mi zati onprobl ems rel atedan ai rcraft. For each
probl emi denti fy all e di sci pl i nes wi l l hel p establ i sh the mathemati cal
model.
1.3 Id nt sev possi bl e 0 pti mi za
pm 0 bl ide a11 t he d is cipl i ns w ill h 1 elp es isl h ma la na atiL
modeJ..
14 I denti fy severa1possi bl opti mi zati onprobl ems rel atedtoa mi crosysternused
for control . ForBach probl emi denti fyall thEdi scI Pl i nes that wi l l hel pestabl i sh
emathemati cal model .
1.5 Def i ne a probl emwi t1I respect toyour i nvestment inthestock market. Descri be
enatureof themathemati cal modd-
1.6 Def i ne probl emand establ i sh mathemati ca1 model for the I - beam
hol di ng up an i ndependent si ngl e-fami l y home.
1. 7 Def i ne the probl emand i denti fy the mathemati cal model for an opt i mum
overhangi ng traffic l i ght.
1.8 Def i ne e probl emand i denti fy a mathemati ca1 model for schedul i ng and
ooti mi zati onof thedai l yrouti neactivity.
Dne e p e mfor a l ami n
iv gi venaspe :ei c p ump.
110 Def i ne a chemi cal engi neeri ng probl emto mi x v i ous mi xtures of 1i mi ted
avai l abi l i ty tomake speci fi edcompounds tom t speci fi edd mands.
2
GRAPHI CAL OPTI MI ZATI ON
P
U
1
2
Thi s book i ncl udes a l argenumber of cxampl es wi th t wo vari abl es. Two- vari abl e
probl ems can be di spl ayed graphi cal l y and th sol uti on obtai ned by i nspecti on.
Al ong wi th the useful ness of obtai ni ng the sol uti on wi thout appl yi ng
mat hmati cal condi ti ons thegraphi cal representati onof theprobl emprovi des an
opportuni ty to expl ore the geometry of many of the numeri cal techni ques we
exami ne later. Thi s is necessary for i ntui ti vel y understandi ng theal gori thmand its
progress toward the sol uti on. I n practi cal si tuati ons whi ch usual l y i nvol ve over
t wo vari abl es thereis no opportuni ty for neat graphi cal correl ati onbecause of the
Ii mi tati ons of graphi cal representati on. It wil1be necessary torevi ewthenumbers
to deci de on convergence or thel ack thereof. What happens wi thmany vari abl es
is an extensi onof thegeometri c features that areobserved inprobl ems i nvol vi ng
t wo vari abl es. A good i magi nati on is an ess ntial tool for these probl ems wi th
many vari abl es.
Contour pl ots provi de the best graphi cal presentati on of the opti mi zati on
probl emint wo vari abl es. The poi nts on any contour (or curve) have thesame val ue
of the functi on. Several sowarepackages areavai l abl e to createand di spl ay these
plots. MATLAB. Mathemati ca Mapl e and Mat hcad are among e general -puqJ Ose
softwarepackages candrawcontour plots. Inthis book we will use MATLAB for
graphi ca1opti mi zati on.
1
9
U
1m
2.1 PROBL EM DEFI NI TI ON

The standard format for opti mi zati on probl ems was establ i shed in Chapter 1. lt is
rein oduc herefor conveni ence:
46 GRAPHI CAL OPTI M' ZATI ON
Mi ni mi ze fl.. xJ X2 . Xn)
Subj ect to: h(x" X2 Xn) = 0
h
2
(xX2.. x ) =O
ht(XI X2.. . X) =0
gl ( X"X2... Xn) o
g2(XI . X2.. Xn) SO
gl l l (X"X2.. . x ) SO
x: gxi X7.
i =I 2... n
(2.1)
(2.2)
(2.3)
(2.4)
In is chapter whi l eadheri ngtotheformat. thenecessi tyfor zeroon theri ght-hand
sideis rel axed. is is bei ngdone for conveni ence and10aidcomprehensi on. le
ri ght-hand side can also have numeri cal val ues other 1han zero. Th first exampl e
chosenfor illustrationis a si mpl eonEusi ng el ementary functi ons whose gmp12i cal
natureis wel l known. Thi s si mpl eexampl e will permi t exarni nnti onof theMATLAB
code at will gencrate ecurves andthesolution. It will alsoal l owus todefi ne E
format for thedi sp1ayof sol uti ontothegraphi cal opti mi zati onprobl cm.
2.1.1 Exampl e 2.1
The first exampl e Exampl e 2.1 will have two equal i tyconstrai nts andtwo i nequal i ty
constramts:
Mi ni mi ze fl.. XI'
X2) = (XI -
3)2
+ (X2-
2)2 (2.5)
Subj cct to: hl (xl . X2): 2x
1 + x2= 8 (2.6a)
h
2(x"
XU: (XI -
1)2
+ (
- 4) 2=4 (2.6b)
gl (X" X2): Xl + x2S 7 (2.7n)
g
2(x . XU:
(2.7b)
o SXI S 10; OSX2 S 10 (2.8)
Intheabove dcfmi 1i on we have two straight lines. two circles nnda parabol a. Note
that two equal i ty cons ints and two vari abl es i mpl y t E is no scope for
o
zation. leprobl emwill be detcrmi nedby uali1yconstrai nts provi dedthe
twocons ints areI i nearl yi ndependent whi ch is trueinthis exnr le. This exampl e
was 1ed10hel punderstand codci nMA' AB at will beusedtodrawgraphi cal
sol uti ons inthis book.
2.1 PAOBLEMDEFINfTl ON 47
Fi gure2.1iII ustrates thegraphi cal sol uti onto i sprobl em. lefigurealsodi spl ays
addi ti onal i nformati on related to the i nequal i ty cons aintspl aced on it after the
principal graphi c i nformati onwas generatedand di spl ayed.
2.1.2 For mat for the Graphl cal Dl spl ay
The graphi cal sol uti on to Exampl e 2.1 as seen in Fi gure 2.1 is generated usi ng
MATI . AB [1.2] ex:pt for theidentifi tionof thei nequal i tyconstraints. Al l i nequal i ty
constrai nts aredi sti ngui shedby hashmarks. The hashedsi deindicates theinfeasible
region. InMA' 1. AB5.2 ereisno featuretoinsert thesemarks throughprogrammi ng.
An add-onprogramcalledMat dr awwas usedtocreatethehashI i nes inthefigure. It
canbe downl oaded f romtheMathworks site. Begi nni ngwi Versi on5.3 thereis a
figure(plotedit) ditor will al l owyou toinsert addi ti onal graphi c el ements to E
fig e. lestudent canalsop cil in ehashtines after obtai ni ngthepri ntout It is
tobe noted at thegraphi cal sol uti onis i ncompl eteif thei nequal i ty constrai nts are
not di sti ngui shed or if thefeasibleregi onhas not beenestabl i shedinsome manner.
10
g
B
3
2 3 4 5 6
x1valu
7
guro2.1 Graphical solutionfor Exampl e2.1.
s 10
48 GRAPHI CAL OPTI MI ZATI ON
In the graphi cal di spl ay of sol uti ons onl y e obj ecti ve functi on is drawn for
several contours sothat edi recti onfor emi ni mumcan be i denti fi ed. Each cqual i ty
constr nt is drawn as a si ngl ecurve for enumeri cal val ueon theri ght-handside.
They shoul dbe i denti fi ed. Each i nequal i tyconstraI nt is si mi l arl ydrawn for a val ueon
eri ght-handside. They need tobe i denti fi cdtoo. The hash marks ar l rawn/penci l ed
inon thefi gurc or a pri ntout of efi gu . The extent of thedcsi gnregi onisestabl i shed
and thesol uti oni denti fi edon thefi gure. The rel evant regi oncan be zoomed for better
accuracy. datafor epl ot areobtai nedusi ngMATLAB' S matri x operati ons after
establ i shi ng the pl ol ti ng mesh. This expl oi ts MATI . AB' S nal ural speed for matri x
operati ons.
2. 2 GRAPHI CAL SOLUTI ON
MATLAB poss s a powerf ul vi sual i zati on engi ne that permi ts the sol uti on of e
two-vari abl e opti mi zati on probl emby i nspecti on. There are three ways to l ake
advantage of the graphi c features of MATI . AB. e first is l he use of MATI . AB'S
hi gh-I evel graphi ngrouti nes for data vi sual i zati on. This will be pri mary way to
sol ve graphi cal opl i mi zati on probl ems in this book. Thi s wil1 al so be e way 10
i ncorporate graphi cal expl oral i on of numeri cal techni ques in e book. For mo
preci se control over the di spl ay of data MATLAB al l ows us r i ntcracti on through
programmi ng usi ng an obj ect-ori ented systemi denti fi ed in MATI . AB as Handl e
Graphi cs. The thi rduseof the MATLAB graphi cs engi ne is touse theHandl e Graphi cs
systemtodevel op a Graphi cal User I nterface(GUI ) for theprogramor m-fi l e. This
creal es a facility for sophi sl i cateduser i nteracti on. MOSl of thepl otti ngneeds can be
mel by thehi gh-I evel graphi cs functi ons avai l abl einMATLAB.
2. 2. 1 MATLABHi gh- Level Gr aphl cs Funct l ons
'here are three useful wi ndows duri ng a typi cal MATLAB scssi on. The first is e
MATLAB Command wi ndowthroughwhi ch MATLAB recei ves i nstructi onand di spl ays
al phanumeri c i nformati on. le second wi ndowis the texl -edi tor wi ndowwhere
m- fiI es are coded. The third is the Fi gure wi ndowwhere the graphi c el ements are
di spl ayed. There can be mor e anone fi gurewi ndow. The fi gurewi ndowis targel
of thehi gh-l evel graphi cs functi ons.
The graphi cs functi ons in MATLAB al l owyou to pl ot in 2D or 3D. They al l ow
contour pl ots in2D and 3D mesh and suapl ot sbar area pi echarts hi stograms
ani mati on and gradi ent plots. Subpl ots can al sobe di spl ayedusi ngthesefuncti ons.
In addi ti on they permi t operati on wi thi mages and 3D model i ng. They al l owbasi c
control of thc appear ce of the pl ot through col or line styl e and markers is
ranges and pect ratioof thegraph. 'hey permi t annotati onof thegraph inseveral
ways. Some of thesefuncti ons will be used inthenext secti onwhen we devcl opthe
2.2 GRAPHI CAL SOLUTI ON 49
:.L
m-fi l efor efirst exampl e. The fol l owi ngi ntroduces some more useful I nf orm on
about theuseof MATLAB8 hi gh-I evcl graphi cs functi ons.
The t wo mai n graphi cal el ements arc typi cal l ycontrol l edusi ngthehi gh-I evel
graphi cs functi ons arethefi gu andtheaxes. Usi ng Handl e Graphi cs you can control
most of theother graphi cal el ements whi ch i ncl udeel emnts usedintheGUI. These
other el ements are typi cal l y chi l drcn of e axes or fi gure. The fi gure functi on or
command creates a fi gurewi ndowwi tha number starti ngat one or will creat anew
fi gure wi ndowi ncrementi ng the wi ndowcount by one. Normal l y all graphi cs
functi ons aretargetedtothec rent figur wi ndowwhi ch is sel ectedby cl i cki ngit
wi th the mouse or executi ng the command fi gure (number) where number is the
number of thefi gure wi ndow al will have l he focus. AIl commands arei ssuedin
command wi ndow. Graphi cs functi ons or commands will automati cal l y create a
wi ndowif none exists.
Invoke hel p commandname at the MATLAB prompt to knowmor e about the
functi ons or comrnands. The wor d handl e app on m y pl aorms and inmany
appl i cati ons parti cul arl ythosethat deal wi thgraphi cs. They arewi del y preval ent in
obj ectori ented programmi ng practi ce. MATI . ABs vi sual i zati on systemis obj ect
ori ented. Most graphi cal el ements areconsi deredas obj ects. The handl e" inMATI . AB
is a sy oftwarecreated number can i dent'y thespeci fi c graphi c obj ect . If
this handl e is avai l abl e enproperti es of theobj ecl such as linesize marker type
col or and soon can be vi ewcd set or reset if neccssary. InMATLAB Handl e Graphi cs
is eway tocustorni ze egraphi cal el ements. Inhi gh-I evel gr hi cs functi ons s
is used ina l i mi tedway. Inthis chapter we will use it ina mi ni mal way tochange E
charactcri sti cs of some of thegraphi cal el ements on thefigure.
To understand the concept of "handl cs" we wi l l run e fol l owi ng code
i nteracti vel y. It deal s wi thcreati nga pl ot whi ch was i ntroduced inChapter 1. The
bol dface comment s in the code segment i ndi cate the newfeal ures that are bei ng
emphasi zed.
Sta MATLABand i nteracti veLyperf ormthefol l owi ng:
>>X O: pi / 40: 2*pi ;
> > y X. si n(x) ;
>> pl ot ( xy' b- ' ) ;
>> gr i d;
>>h pl ot ( xy ' b- ' ) h i s t he handl e t o t he pl ot
> > a new pl ot i s over wr i t t en i n t he same f i gur e wi ndow
> > a numer i cal val ue i s assi gned t o h
> > We can use t he var i abl e name h or i t s val ue
> > t o r ef er t o t he pl ot agai n
>>
>> set ( h' Li neWi dt h' 2) ; t hi s shoul d make your pl ot
> > t hi cker bl ue
>> set ( h' Li neWi dt h' 3' Li neSt yl e' ' :' ' Col or ' ' r' )
r

o

c

e

r
h

0
D
t
r

c
o
n

e
t
-
-
v
c

e
x

x
v
s

e
y
v

L
a
e
y

e
r
a
t

c
e
o

r
l

c
p
50 GAAPHI CAL OPTI MI ZATI ON

> > The handl e i s used t o r ef er t o t he obj ect
> > whose pr oper t y i s bei ng changed
> > Usual l y Pr oper t y i nf or mat i on occur s i n pai r s of
> > pr oper t y- namej pr oper t y- val ue
> > pr oper t y- val ue can be a t ext st r i ng or number
>>
>> get ( gca) t hi s wi l l l i st t he pr oper t y of t he axes
> > of t he cur r en pl ot . Not e t her e ar e a si gni f i cant
> > amount of pr oper t i es you can change t o cust omi ze
> > t he appear ance of t he pl ot
>>
>> set ( gca' yt i ck' [- 5 2. 502. 55] )
> > you have r eset t he yt i ck mar ks on t he gr aph
>> set ( gca' Font Name' ' Ar i al ' ' Font Wei ght ' ...
>> ' bol d' ' Font Si ze' 14)
>>
> > Changes t he f ont used f or mar ki ng t he axes
>> set ( gca' Xcol or ' ' bl ue' )
> > changes t he x- axi s t o bl ue
>> % concl udes t he demonst r at i on of handl es
Fr omthe sessi onabove it is evi dent to fi ne-tune thepl ot you create you
need to first i denti fy the obj ect handl e. You use the handl e to access eobj ect
prop y. You change a property by setti ngits val ue. We will seemor e graphi cs
functi on as we obtai n the graphi cal sol uti on to Exampl e 2.1 in fol l owi ng
sectl on.
2. 2. 2 Exampl e 2.1- Graphi cal Sol uti on
Fi gure 2.1 is thegraphi cal representati onof Exampl e 2.1. The range for thepl ots
matches thesi deconstrai nts for theprobl em. The i ntersecti onof thet wo equal i ty
constrai nts i denti fi es t wo possi bl sol uti ons to theprobl em approxi matel y (2.6
2.8) and (1 6). The i nequal i ty constrai nt g2 makcs the poi nt (2.6 2.8)
unacceptabl e. Poi nt (1 6) is acc ptabl e wi th respect to both constrai nts. Thc
sol uti on is therefore at (1 6). Whi l e the obj ecti ve functi on was not used to
determi ne thc sol uti on contours of theobj ecti vefuncti onaredrawn i ndi cati ng
di recti onfor emi ni mumof theobj ecti v functi on. The sol uti oncan be i denti fi ed
by i nspecti onof theass mbl edpl ots wi thout reference to thetermi nol ogy or the
techni qucs of opti mi zati on. Recal l however. that this is onl y possi bl ewi thone or
t wo desi gnvari abl es intheprobl em.
The code for is ex npl ei nvol ves sixm-fi l es: a scri pt m-fi l e will incude
the MATL AB statements to create and di spl ay edata and fi vem- fi 1 onefor
cach functi on i nvol ved in Exampl e 2. 1. We begi n wi th the script m- f i l e-
Ex2_1. m.
2.2 GAAPHI CAL SOLUTI ON 51
File: Ex2 1. m
Chapt er 2: Opt i mi zat i on wi t h MATLAB
% Dr . P. Venkat ar aman
Exampl e 2_1 ( Sec 2. 1- 2. 2)
gr aphi cal sol ut i on usi ng MATLAB ( t wo desi gn var i abl es)

t he f ol l owi ng scr i pt shoul d al l ow t he di spl ay
of gr aphi cal sol ut i on
Mi ni mi ze f ( x1x2)
h1( x1x2)
h2( x1x2)
gl ( X1x2)
gl ( x1x2)
(x1 3) **2 + (x2 2) **2

2x1 + x2 8
( x1- 1) ^2 + ( X2- 4) ^2 4
x1 + x2 <= 7
x1 - 0. 25x2^2 <= 0. 0
% 0 <= x1 <= 10 0 <= x2 <= 10
NOTE: The hash mar ks f or t he i nequal i t y

const r ai nt s must be det er mi ned and dr awn out si de
of t hi s exer ci se and on t he pr i nt out
- - - - - " " - -
x1=0: 0. 1: 10i t he semi col on at t he end pr event s
% t he echo
x2=0: 0. 1: 10i t hese ar e al so t he si de const r ai nt s
x1 and x2 ar e vect or s f i l l ed wi t h number s st ar t i ng
at 0 and endi ng at 10. 0 wi t h val ues at i nt er val s
of 0. 1
[X1 X2] meshgr i d( x1x2) i
gener at es mat r i ces X1 and X2 cor r espondi ng t o
% vect or s x1 and x2. I t i s a mesh of x1 and x2 val ues
at whi ch t he f unct i ons wi l l be eval uat ed
f 1 obj _exl ( X1X2) i t he obj ect i ve f unct i on i s
eval uat ed over t he ent i r e mesh and st or ed i n f 1
MATLAB wi l l comput e t he val ues f or t he obj ect i ve
t hr ougha f unct i on m f i l e cal l ed obj _exl . m
i neq1 i neconl ( Xl X2) i t he i nequal i t y gl i s
eval uat ed over t he mesh
i neq2 i necon2( Xl X2) i t he i nequal i t y g2 i s
eval uat ed over t he mesh
eq1 = eqcon1( Xl X2) i t he equal i t y h1 i s eval uat ed
over t he mesh
eq2 = eqcon2(X1X2) i t he equal i t y h2 i s eval uat ed
over t he mesh
[C1han1] cont our ( 1 x2 i neq1 [77) ' r- ' ) i
a si ngl e cont our pl ot of gl ( or i neql ) i s dr awn f or
t he val ue of 7 i n r ed col or as a cont i nuous l i ne
dupl i cat i on of t he cont our val ue i s necessar y
f or dr awi ng a si ngl e cont our
han1 i s t he handl e t o t hi s pl ot . Thi s handl e
can be used t o change t he pl ot di spl ay char act er i st i cs
Cl cont ai ns t he val ue of t he cont our
cl abel ( C1han1) i
l abel s t he cont our wi t h t he val ues i n vect or C1
cont our ( x1x2i neq1[77] ' r- ' ) wi l l dr aw t he cont our
wi t hout l abel i ng t he val ue
hol d on al l ows mul t i pl e pl ot s i n t he same f i gur e
wi ndow
gt ext ( gl ' ) i
wi l l pl ace t he st r i ng ' gl ' on t he pl ot at t he spot
sel ect ed by a mouse cl i ck. Thi s i s a t ext l abel
t hi s pr ocedur e i s r epeat ed f or r emai ni ng
const r ai nt s
[C2han2] cont our ( x1x2i neq2 [00] ' r- ) i
cl abel ( C2han2) i
gt ext ( ' g2' ) i
[C3 han3] cont our (x1 x2 eq1 [88] ' b- ' ) i
cl abel ( C3han3) i
gt ext ( h1' ) i
[C4han4] cont our ( x1x2eq2 [44] ' b- - ' ) i
cl abel ( C4han4) i
gt ext ( h2' ) i
[Chan] cont our ( x1x2f 1' g' ) i
cont our of ' f ' i s dr awn i n gr een col or . The number
of cont our s ar e deci ded by t he def aul t val ue
cl abel (C han) i
xl abel ( ' x1 val ues' ' Font Name' ' t i mes' ' Font Si ze' 12.
' Font Wei gh ' bol d' ) i l abel f or x- axes
yl abel ( ' x2 val ues' ' Font Name' ' t i mes' ' Font Si ze' 12
' Font Wei ght ' ' bol d' ) i
gr i d
hol d of f
The functi onm- fiI es are:
2.2 GAAPHI CAL SOLUTI ON 53
on1. m
f unct i on r et val i necon1( X1 X2)
r et val X1 + X2i
X1 X2 ar e mat r i ces
r et val i s t he val ue bei ng r et ur ned af t er t he
comput at i on
Si nce X1 and X2 ar e mat r i ces r et va1 i s a1so a
mat r i x
By t hi s way t he ent i r e i nf or mat i on on t he mesh
i s gener at ed by a si ng1e cal l t o t he f unci on
i necon1
2 . m
f unct i on r et va1 i necon2( X1X2)
r et val Xl - 0. 2S* X2. ^2i
Not e t he use of ^ oper at or f or el ement by e1ement
oper at i on. That i s each e1ement of t he X2 mat r i x
i s squar ed. Wi t hout t he dot t he i mp1i cat i on i s a
mat r i x mul t i pl i cat i on - bet ween mat r i ces whose
i nner di mensi ons must agr ee. Si mi l ar oper at or s ar e
def i ned f or el ement by el ement mul i pl i cat i on
and di vi si on
L m
f unct i on r et val eqcon1( X1X2)
r et val 2. 0*Xl + X2i
2. m
f unct i on r et val eqcon2( X1X2)
r et val (X1 - 1) . *( X1 1) + (X2 - 4) . * (X2 - 4) i
U
f unct i on r et val obj _ex1( X1X2)
r et val = (X1 - 3) . *( X1 - 3) +( X2 - 2) . (X2 2) i
2. 2. 3 Di spl ayl ng t he Gr aphi cs
Al1of lhc files requi redfor graphi cal di spl ayof theprobl emhave been crcal cd. Inthe
Command wi ndow l ypeaddpat b fol l owedby ecompl el e pathfor thcdi rectory at
hol ds these files. At the prompt type thc name of thc script file for the exampl e
wi thout the. meXl ensi on
>> Ex2 1
54 GRAPHI CAL OPTI MI ZAnON 2.2 GRAPHI CAL SOLUTI ON 55
The first contour pl ot will appear. Move themouse over to epl ot and a cross hai r
appears. Cl i cki ngon thepl ot wi1l pl ac thestri ng"gl " at thecross hair. The second
contour pl ot shoul dappear and thereis a pause topl ac etext l abel . After thefour
cons aints eobj ecti vefuncti onispl ottedfor several contour val ues. Fi nal l ythepl ot
shoul d app r i nFi gure 2.1 wi thout thehashmarks (andwi your choi ce of B
l ocati onfor constrai nt labels). You can insert thehashmarks throughedi ti ngthepl ot
di rectl y ( Versi on5.3 onw ds). sol uti onfor theprobl emis at ( 1 6) where four
pl ots i ntersect . The val ueof theobj ecti vefuncti onis 20 esol uti on.
If thepl ot is acceptabl e you canpri nt thei nformal i onon thefi gureby expo ngit
(usi ng an appropri ate extensi on) 10 a file and later i ncorporati ng it in another
document . You can al so send it to epri nter f romIhe pri nt command on theFi l e
menu. Typi ng hel p pri nt intheCommand wi ndowshoul dI i st a set of commands you
can use tosave efile.
10
2 4 6
X
1
values
s 10
>> pr i nt - depsc2 pl ot _ex_2_1. eps
wi1l c ate the l evel 2 col or postscri pt file cal l ed pl ocex_2_l. eps in the worki ng
di rectory. A compl ete pathname shoul dsavethefileto especi fi eddi rectory. You
can then execute standard operati ng systemcommands 10 dump the filc 10 e
appl i cabl printcr. You can al so save the fi gure (ex si on .fig) and later open it in
MATLAB usi ngthefilc open command f rommenu b ortool bar.
2. 2. 4 Cust oml zl ng t he Fl gure Fl gure2. 2 Customi zed U forE mpl e2.1.
Inthis secti onthc basi c fi gureand pl ots creatcdabov wilI be custorni zcdusi ngthe
Handl e Graphi cs commands first expl o di nScti on2.2. 1. Al l ofthe pl ots descri bi ng
econstrai nts will have a l i newi dthof 2 poi nts. leobj ecti vefuncti onwi l l have a
l i newi dthof 1 poi nt. The text l abel s wi1l be inthe"Ti mes" font. It wilI be inbol dface
wi tha font of si ze 14and in ecol or of thecons int . Thex dY tickmarks will be
changed to establ i sh a broader grid. The x dy labe1s wi l l i ncl ude a subscri pt . A
two- rowtext wi1l be i nsertedat thepoi nt sel ectedthroughthemousc. The ncwfi gure
is shown inFi gurc 2.2.
First py thefilein eprevi ous exerci seand rename it. lerenamed filewill be
used maket hen essarych gesto ecode. Thesechanges inc1ude bothnewcode
as weIl as repl acement of exi sti nglines of code. In efoIl owi ng onl y echanges to
codearei1I ustrated. The ori gi nal code is shown in the italic styleand i denti fi ed
wi ththreeellipsis poi nts (. . .) at begi nni ngof theline. The code tobe i nsertedor
repl acement code is shown innor m styl . If no repl acemcnt is apparent thentheol d
code is used 10defi nethel ocati ons whe ewdeis appended.
g t t ( ' gl ' ) ;
k1 gt ext ( gl ' ) ; k1 i s t he handl e t o t he t ext el ement
set ( k1' Font Name' ' Ti mes' ' Font Wei ght ' ' bol d'
Font Si ze' 14' Col or ' ' r ed' )
. . cl abel ( C2han2}i
set ( han2' Li neWi dt h' 2)
k2 = gt ext ( ' g2' ) ;
cl abel ( C3han3) ;
set ( han3' Li neWi dt h' 2)
k3 gt ext ( ' h1' ) ;
' Font Si ze' 14' Col or ' ' bl ue' )
{Cl hanl } cont our ( xl x 2i neql {77}' r - ' };
set ( han1' Li neWi dt h' 2) ; set s 1i ne wi dt h of t he gl
cont our t o 2 poi nt s
cl abel ( C4han4) ;
set ( han4 Li neWi dt h' 2)
k4 gt ext ( ' h2' ) ;
2.3 ADDl TI ONAL EXAMPLES 57
set ( k4' Font Name' ' Ti mes' ' Font Wei gh ' bo1d'
Font Si ze' 14' Co1or ' ' b1ue' )
c: l abel (C han) ;
set ( han' Li neWi dt h' 1)
.. xl abel ( xl val ues' Font Name' ' t i mes'
x1abe1( ' x 1 va1ues' ' Font Name' ' t i mes' Font Si ze' 12.
Font Wei ght ' ' bo1d' ) i
y1abe1( ' x 2 val ues' ' Font Name' ' ti mes' '
Font Si ze' 12 ' Font Wei ght ' ' bo1d' ) i
set ( gca xt i ck' [0 2 4 6 8 10] ) set xt i cks
se ( gca' yt i ck' [0 2. 5 5. 0 7. 5 10] ) set yt i cks
k5 gt ext ( {' Chapt er 2: Examp1e 1'
' pr et t y gr aphi cal di spl ay' })
t he above gt ext descr i bes a st r i ng ar r ay
st r i ng ar r ay i s def i ned usi ng cur l y br aces
set ( k5' Font Name' ' Ti mes' ' Font Si ze' 12
Font wei ght ' ' bo1d' )
!(xI' X
2
) cos(qx
2
) -1- c + d
wi th
a = 1 b = 2 c = 0.3 d = 0.4 p = 3 q=4
(2.9)
Fi gurcs 2.3-2. 7ar egraphi cal di spl ay of sol uti ons is section. The fi gures in
thebook bei ngres ctedtobl ack and whi te will not nvey ei mpact of col or i mages
you will see on the screen. Thcre are two files associ atcd wi ththeplots: thescript
m- fiI e (ex2.-2. m) and thc functi onm- fiI e (obLex2. m).
. . z. m
Examp1e 2. 2 Sec. 2. 3. 1
( at t he end)
c1ear C C1 C2 C3 C4 h h1 h2 h3 h4 k1 k2 k3 k4 k5
get r i d of var i ab1es f r omt he wor kspace
Run theprograrntosec efi gureinFi g 2.2. Once thepl ot is customI zed toyour
sati sfacti on you can make it standardfor other pl ots you will pr U i nMAAs .
Thi s filecoul dal sobe a templ atefor general contour plotting. Note at a1l functi on
i nformati onis obtai nedthrough functi on m- fiI es whi ch arecoded outsi dethescript
filc. Sctti ng up ncwprobl ems or exampl cs onl y requi res addi ngl changi ng new
functi onm- fi1.
11 I AbclodConl g
defaull cGlcr
2. 3 ADDI TI ONAL EXAMPLES
The f01l 0wi ng addi ti onal ex npl es will servetoiIl us tebo opti mi zati onprobl ems
wel l asaddi ti onal gmphi cal fcatures of M A TLA at will be useful ind vel opi nggraphi cal
sol uti ons toopti mi zati onprobl ems. lCgrphi cal routi nes inMATLAB arepowerful d
SYtouse. They cangr phi cal l ydi spl aytheprobl ems vem1ways wi verysi mpl c
mm ds. The useful di spl ayis however detcnni nedby euser. The first exampl e in
this secti on Exampl e 2.2 is a probl emin uncons i ned opt tion. second
cxampl e is a structum1engi neeri ngprobl cmof reasonabl c compl cxi ty. lC exar Ic
demons sopti mi zati oninthcareaof heat Iransfcr dcsi gn.
2.3.1 Exampl e 2. 2
Thi s cxampl c iII us tes scvcral di ffcrent ways of gmphi caUy di spl ayi nga functi onof
two vari abl es. The probl emwas uscdtoiII usl rategl obal opti mi zati oninReference 3.
The si ngl c obj ccti vefuncti onis
-0. 5 O
X
J
val u
0.5
gu 2.3 Filledconl ours withcol orbar . Exampl e2.2.
59
2. 3 AOOmONAL EXAMPLES
2DCon
wi thGrndi .nl V: 10
GRAPHI CAL OPTI MI ZATl O N 58
8
2
-
g
J

X
1
val ues
-1 X:! V ues
0.5 o
X
1
values
- 0.5
3Dconl our wilhlorbar . Ex le2.2.
t he obj ect i ve f unct i on i s
eval uat ed over t he ent i r e mesh
f i l l ed cont our wi t h def aul t col or map
hel p gr aph3d gi ves you t he choi ces f or col or map
[C1han11 cont our f ( x1x2f 1
[0 0. 1 0. 6 0. 8 1. 0 1. 2 1. 5 1. 8 2. 0 2. 4 2. 6 2. 8 3. 01) ;
Fl gure2. 5
f 1 obj _ex2( X1X2) ;
1y' t.
t i cks
t i cks
speci f i c cont our l evel s i ndi cat ed above
cl abel ( Cl hanl ) ;
col or bar i l 1ust r at es t he def aul t col or scal e
set ( gca' xt i ck' [- 1 - 0. 5 0. 0 0. 5 1. 01) cust om
set ( gca ' yt i ck' [ 1 - 0. 5 0. 0 0. 5 1. 01) cust om
gr i d
xl abel ( ' X_1 val ues' ' Font Name' ' t i mes'
' Font Si ze' 12) ;
% l abel f or x- axes
yl abel ( ' x 2 val ues' ' Font Name' ' t i mes'
Unconst r ai ned f unct i on i l l ust r at i ng gl obal mi ni mum
Exampl e wi l l i nt r oduce 3D pl ot s 3D cont our s f i l l ed
2D cont our s wi t h gr adi ent i nf or mat i on
- - - - - - - - - - - - - -
x1= 1: 0. 01: 1; t he semi - col on at t he end pr event s
" t he echo
x2=- 1: 0. 01: 1i t hese ar e al so t he si de const r ai nt s
x1 and x2 ar e vect or s f i l 1ed wi t h number s st ar t i ng
at - 1 and endi ng at 1. 0 wi t h val ues at i nt er val s of
0. 01
Contour wlthgradlent vectors: Exampl e 2.2. gure2. 4
[Xl x21 = meshgr i d( x1x2) ;
gener at es mat r i ces X1 and X2 cor r espondi ng
vect or s x1 and x2
r emi nder T ~ i s case sensi t i ve
CO!U' SC Mesh P101
l ormap 1
4
3.5
3
2.6
2
1.5
a
u
m
w
0.6
X
2
val ues
-1 -1
X
1
val u
Fl gure2.6 3Dmesh plot wlthcolorbar: Exampl 2.2.
Font Si ze' 12) ;
t i t l e( {' Fi l l ed Label l ed Cont our ' ' ...
def aul t col or map' }' Font Name' i mes' ' Font Si ze' 10)
a new f i gur e i s used t o dr aw t he
basi c cont our pl ot super i mposed wi t h gr adi ent
i nf or mat i on
al so i nf or mat i on i s gener at ed on a coar ser mes h t o
keep t he f i gur e t i dy. gr i d i s r emoved f or cl ar i t y
f i gur e a new f i gur e wi ndow i s dr awn
y1 - 1: 0. 1: 1. 0;
y2 1: 0. 1: 1;
[ Y1Y2] meshgr i d( y1y2) ;
f 2 obj _ex2( Yl Y2) ;
[C2han2] cont our ( y1y2f 2
[0 0. 5 0. 75 1. 0 1. 5 1. 75 2. 0 2. 5 3. 0] ) ;
2.3 ADOmONAL EXAMPl ES 61
Coa SurfacePlot
lonnnp-jcVdefnult
4
3.5
3
2.5
2
1.5
0.5
0
1
0.5
0.5
5
.0.5
X
2
val ues
-1 -1
H
M

-
a v

x

Fl gure2.7 3Dsu pl ot withdefault col orbar : Exampl e 2.2.
2
1.5
cl abel ( C2han2)
[ GX GY] gr adi ent ( f 20. 2) ;
% gener at i on of gr adi ent i nf or mat i on see hel p gr adi ent
hol d on mul i pl e pl os on t he same f i gur e
qui ver ( Y1Y2GXGY) ;
dr aws t he gr adi ent vect or s at gr i d poi nt s
see hel p qui ver
hol d of f
set ( gca' xt i ck' [ - 1 - 0. 5 0. 0 0. 5 1. 0] )
set ( gca' yt i ck' [ - 1 - 0. 5 0. 0 0. 5 1. 0] )
xl abel ( ' x_1 val ues' ' Font Name' ' t i mes'
Font Si ze' 12) ;
yl abel ( ' x 2 val ues' ' Font Name' ' i mes'
' Font Si ze' 12) ;
t i t l e( {' 2D Cont our ' ' wi t h Gr adi ent Vect or s' }
Font Name' ' t i mes' ' Font Si ze' 10)
62 GRAPHI CAL OPTI MI ZATl ON
A f i nal exampl e of cont our i nf or mat i on i s a 3D
% cont our pl ot - or a st acked cont our pl ot
f i gur e
col or map( spr i ng)
cont our 3( x1x2 f 1
[0 0. 3 0. 6 0. 8 1. 0 1. 5 1. 8 2. 0 2. 4 2. 6 2. 8 3. 0] ) ;
set ( gca' xt i ck' [- 1 0. 5 0. 0 0. 5 1. 0] )
set ( gca ' yt i ck' [- 1 - 0. 5 0. 0 0. 5 l . 0])
change col or map and set col or bar
changi ng col or f or di spl ay i s easy by sel ect i ng
among some col or maps pr ovi ded by MATLAB
col or bar
xl abel ( ' x 1 val ues' ' Font Name' ' t i mes'
Font Si ze' 12) ;
yl abel ( ' x 2 val ues' ' Font Name' ' t i mes' . . .
Font Si ze' 12) ;
t i l e( {' St acked (3D) Cont our ' : col or map - spr i ng' }
gr i d
he next t wo f i gur es wi . l l di spl ay 3D pl ot s
he f i r st i s a mesh pl ot of t he f unct i on
once mor e coar se dat a i s used f or cl ar i t y
f i gur e
col or map (cool ) anot her col or map
mesh (y1 y2 f 2) usi ng i nf or mat i on gener at ed ear l i er
set ( gca ' xt i ck' [- 1 - 0. 5 0. 0 0. 5 l . 0])
set ( gca' yt i ck' [- 1 0. 5 0. 0 0. 5 1. 0] )
col or bar
xl abel ( ' x 1 val ues' ' Font Name' ' t i mes'
Font Si ze' 12) ;
yl abel ( ' x_2 val ues' ' Font Name' ' t i mes' . . .
Font Si ze' 12) ;
t i t l e( {' Coar se Mesh Pl ot ' ' col or map - cool ' }
gr i d
he f i nal pl ot i n t hi s ser i es
sur f ace pl ot wi t h def aul t col or map
f i gur e
col or map( j e}
sur f ( y1y2f 2) usi ng ol d i nf or mat i on
col or bar
xl abel ( ' x 1 val ues' ' Font Name' ' t i mes' .
2.3 AOOrrtONAL EXAMPLES 63
Font Si ze' 12) ;
yl abel ( ' x_2 val ues' ' Font Name' ' t i mes'
Font Si ze' 12) ;
set ( gca ' xt i ck' [- 1 - 0. 5 0. 0 0. 5 l . 0])
set ( gca' yt i ck' 1- 1 - 050. 0 0. 5 l . 0])
t i t l e( {' Coar se Sur f ace Pl ot ' col or map - j et /
def aul t ' }' Font Name' ' t i mes' ' Font Si ze' 10)
gr i d
Q s . m
Opt i mi zat i on wi t h MATLAB
Chapt er 2. Exampl e 2. 2
f ( x1x2) a*xl ""2 + b*x2^2 c cos ( aa*x1)

d*cos ( bb*x2) . . .
+ c + d
a ' " 1; b 2 c 0. 3; d ' " 0. 4; aa
3. 0*pi ; bb 4. 0*pi ;
not e mat r i x oper at i ons need a dot oper at or
r et val a*X1. *X1 + b*X2. *X2 - c*cos( aa*X1)
d*cos( bb*X2) + c + d;
The brief comments inthecode shoul dprovi deanexpl anati onof what you seeon
figure. Fi gures canbe further custorni zedas secnin eprevi ous section. Froman
opti rni zati onperspecti ve Fi gure2.4provi des thc best i nfonnati onabout thcnatureof
theprobl em. The 2D contour curves identifythenei ghborhood of the10 l rni ni mum.
legradi ent vectors i ndi catc di recti onof efuncti ons steepest risea1 poi nt
so s and val l cys canbc di sti ngui shcd. The contou thernsel ves canbecol ored
wi thout bei ngfillcd.
Thc qui ver plp1shown in Fi gure 2.4also provi des a mechani sm10i ndi cate e
feasible regi on when deal i ng wi th i nequal i ty constrai nts si ncc they i ndi catc e
di recti oninwhi ch constrai nt functi onwiII incr se. If several functi ons arebci ng
wn enthc cI utter produ dby earrows may dfuse ecI arity. The user is
encouragcd10usethe.powerful graphi cal features of MATLAB tohisbenefi t at all ti mes
wi thout l osi ngsight of eobj ecti veof his cffort. MATLAB graphi cs has many more
features thanwill becoveredin is chaptcr. The expos inthis chapter shoul dbc
sufficient for E derconfi dentl yexpl oremany other useful graphi mmands.
Thc 3D mesh and surfaceplots have l i mi teduseful ness. Theseplots canbeuscdto
rei nforcesome of thefeatures foundinFi gure2.4. The i nfonnati onintheseplots may
bei mproved by choosi nga camera angl e emphl ZCS measpect of thegraphi I
descri pti on. Thisexpl orati onisleft to C ader asanexerci se. Usi nghel pvi ewinthe
MATLAB Command wi ndowshoul dget you startcdin is di ction.
2. 3. 2 Exampl e 2. 3
lenext exampl e is a compl ex one f romstructural engi neeri ngdesi gn is rel evant
inci vi l l mechani cal l aerospacecngi n ri ngappl i cati ons. It appeared as a probl cmin
Refercnce 4. It is dcvel oped indetail here. The probl emis toredesi gnthe basi c tall
fl agpol c invi ewof thephenomenal i ncreasc inwi nd spceds duri ngextreme weather
condi ti ons. In recent catas' Ophi c events. the wi nd sp ds in tomadocs have bccn
measured at over 350 mi l es per hour. Thesehi ghspecds app r tobe thenor mrather
unusual cvent .
Desl gn Pr obl em: Mi ni mi ze the mass of a standard IO- m tubul ar fl agpol c to
wi thstandwi nd gusts of 350 mi l es per hour. The fl agpol ewil be made of structural
steel . Use a factor of safetyof 2.5for c structuraJ desi gn. lCdefl ecti onof thetop
of thefl agpol eshoul dnot exceed 5 cm. The probl emis descri bedinFi gure 2.8.
Mat hemat i cal Model : The mathemati cal modcl is devel oped in detail for
compl etencss and to provi de a rcvi ewof useful s ctural (5) and aerodynami c
rel ati ons [6). The rel ati ons areexprcssedinori gi nal symbol s rather thaninstandard
format of opti mi zati onprobl ems toprovi de an i nsi ght intoprobl emformul ati on.
Desl gn Paramet ers: Thc stnl ctucal stcel [5] has thefol1owi ng materi al constants:
E (modul us of clasticity): 200 E 9Pa
011 (al l owabl c nor maJ s) : 250 E+06 Pa
1:
0
1 1(al l owabl eshcar stress): 145 E+06 Pa
8 m
A
10m
gure2.8 Fl agpol edeslgn: Exampl a 2.3.
s8ctl on A A
2.3 ADDI TI ONAL EXAMPLES 65
'Y(materi aJ densi ty): 7860 kgl m
3
FS (factor of safety): 2.5
g (gravi tati onal accel erati on) = 9.81 ml s2
For theacrodynami c cal cul ati ons thefol l owi ngareconsi dered:
p (standardair densi ty): 1. 225kgl m
3
C
d
( coeffi ci ent of cyl i nder): 1.0
Wp (flagwi nd l oadat 8 m): 5000 N
Vw (wi nd speed): 350 mph (156.46 ml s)
Thegeome cparameters ar
Lp: thel ocati onof flagwi nd l oad(8m)
L: l eng of thepol ( 10m)
OlIl l : permi Ued defl ecti on(5cm)
Desl gn Vali bl es: The desi gnvari abl es shown inFi g2.8are
do: outsi dedi ameter (XI ) [ Note: x's arenot usedin emodel ]
dj : i nsi dedi ameter (.x2)
Geomet ri c Re uons: The fol1owi ng rel ati ons will be useful inlater caJ cul ati ons:
A: areaof cross cti on= 0. 25 * d ~ - d f )
1: di ame ical moment of inertia= * d ~ - dt)/ 64
Qlt: first momcnt of areaabove theneutral is di vi dedby thi ckness
= d ~ + dodi + dr ) / 6
Obj ecti ve Functi on: The obj ecti ve functi on is the w i ght of the I O- muni form
fl agpol e:
Wei ght:j{xl' X
2
): L * A * 'Y* g
(2.10)
Constra t Functi ons: le wi nd l oad per uni t l ength ( F
D
) on the fl agpol e is
caI cul atedas
Fo = 0.5* P * ~ * C
d
* d
o
Thc bendi ng moment at thebase of thepol edue to is uni f ormwi nd l oadon theenti
pol eis
Mw=0. 5 * FoL * L
The bendi ng moment due tothewi nd l oadon thef1ag is
Wn"' L_
F- " " " P A JD
Bendi ng (nonnal ) stress at thebase of epol eis
'bcnd= 0.5 ( Mw+M
F
) di l
Nonnal s ess due tothewei ght is
weight = 'Y g L
Tota1 nonna1 sU sses to be resi sted for desi gn is e sumof nonna1 stresses
comput ed above. Incorporati ng the factor of safety and the a1l owabl e stress f rom
materi a1va1ues thefirst in ua1i tynsU int can be set up as
+wci ght S nU
I FS (2.11)
The maxi mumshear l oadinthecross secti onis
s= WF +F
o
' " L
The maxi mumshear stress inthepol eis
=S * QI (I t)
The second i nequa1ity constrai nt bed on handl i ng e r stresses inthef1agpol e
18
(2.12)
le third practi ca1coostrai ot is based 0 0 the def1ecti oo of the topof e pol e. 'hi s
dfl ecti oodue toa uni f onn wi nd l oadon thepol e is
ow= Fo 01<L
4
/(8 E *
The def1ecti onat thetopdue tothef1ag wi nd l oadat is
O
F
= (2 W
F
'" L
3
_ W
F
L ' " L ' " Lp)/(E'"

The thi rdcoostrai ot transl ates to
C3( X
1
X
2
) ow+ O
F
S O.
u
(2. 13)
f
/
2.3 AODI TI ONAL EXAMPLES 67
To di scourage sol uti ons where d
o
< dj we wi l l i ncl udea geometri c constrai nt:
84( X
1
X
2
) d 0. 001 (2.14)
Si de Constrai nts: lis defi nes thedesi gnregi onfor thesearch.
2 c mS d
o
S 100 cm; 2 c mS d
i
S 100 c m (2.15)
MATLAB Code: The m-fi l es for this exampl e are gi ven bel ow. An i mportant
observati onin is probl em and structural engi neeri ngprobl ems inparti cul ar is
order of magni tude of the quanti ti es in the constrai ni ng uati ons. le stress
coos nts areof eorder of 108+06 whi l e thedi spl acemeot l enns areof eorder
of 108-02. Most numeri ca1techni ques s ggl etohandl e this range. Typi ca1ly such
probl ems 0 d to be nonna1i zed before bei og soI ved' It is essenti a1 in appl yi ng
numeri cal techni ques used inopti mi zati on.
Thi s exampl e is pl ottedint wo p 15. Inthefirst part each i nequa1ityCOI int is
i nvesti gateda1one. Two curves areshowlI for each cons nt. Thi s avoi ds clutter. 'he
si defor thel ocati onof thehash mar k 0 0 theconstrai nt is detenni ned by drawi ng these
C Vi n col or. le bl ue col or i ndi cates the feasi bl e di recti on. Al tematel y qui ver
pl ots can be used. The second part is econsol i datedcurve shown. The consol i dated
curve has to be dr awn by removi ng the comment s on some of the code d
comment i ng thede at is not needed. Si nce eopti ma1sol uti oncannot be c1earl y
establ i shed zoomfeatureof MATLAB is used tonarrowdown thesol uti on.
2. : . J . . . . ( t he mai n scr i pt f i l e)
Chapt er 2: Opt i mi zai on wi t h MATLAB
Exampl e 2. 3 Sec. 2. 3
gr aphi cal sol ut i on u6i ng MATLAB ( t wo desi gn var i abl es)

Opt i mal desi gn of a Fl ag Pol e f or hi gh wi nds
Ref . 2. 4
' - - - - - - - - - "
gl obal st at ement i s used t o shar e same i nf or mat i on
% bet ween var i ous m- f i l es
gl obal ELAS SI GALL TAUALL GAM FS GRAV
gl obal RHO CD FLAGWSPEED L P L DELT
- - - - - - - - - - - " - - - - - "
I ni t i al i ze val ues
ELAS 200e+09i Modul us of el ast i ci t y - Pa
SI GALL = 250E+06i % al l owabe nor mal st r ess - Pa
TAUALL = 145e+06i al l owabl e shear st r ess - Pa lI l i 7 III 1
68 GRAPHI CAl OPTI MI ZATI ON
GAM= 7860;
FS 2. 5;
GRAV 9. 81;
-
R H O l . 225;
CD l . 0;
FLAGW 5000;
SPEED 156. 46;
" -
LP 8;
L 10;
DELT 0. 05;
- -
gl val SI GALL/ FS
g2val TAUALL/ FS
g3val DELT
g4val 0. 001
densi t y of mat er i al kg/ m3
% f act or of saf et y
gr avi t at i onal accel er at i on
densi t y of ai r kg/ m3
% dr ag coef f i ci ent
concent r at ed l oad on f l ag - N
I / S
l ocat i on of dr ag l oad on f l ag - m
l engt h of pol e
al l owabl e def l ect i on - m
r i ght hand si de val ues
f or t he const r ai nt s
- - - - ' - - - - - -
x 0. 02: 0. 01: 1; t he semi - col on at t he end pr event s
t he echo
x2=0. 025: 0. 01: 1; t hese ar e al so t he si de consr ai nt s
t x1 and x2 ar e vect or s f i l l ed wi t h number s
not e a way t o avoi d x1 x2
[ X1 X2] meshgr i d{x1x2) ;
vect or s x1 and x2
f 1 obj _ex3( X1X2) ;
t he obj ect i ve f unct i on i s eval uat ed over t he ent i r e
mesh
Const r ai nt s ar e eval uat ed
i neq1 i neq1_ex3{X1X2) ;
i neq2 i neq2_ex3( X1X2) ;
i neq3 i neq3_ex 3 {X1X2) ;
i neq4 i neq4_ex3( X1X2) ;
[ C1han1] cont our {xl x2f 1 [01000050000
100000 150000 200000 250000 300000] ' g- ' ) ;
cl abel ( C1han1) ;
set ( gca ' xt i ck' [0 0. 2 0. 4 0. 6 0. 8 l . O] )
set ( gca ' yt i ck' [0 0. 2 0. 4 0. 6 0. 8 1. 0) )
xl abel ( ' out si de di amet er ' ' Font Name' ' t i mes'
Font Si ze' 12) ;
E
yl abel {' i nsi de di amet er ' ' Font Name' ' t i mes'
Font Si ze' 12)
gr i d
hol d on
f i gur e a new f i gur e wi ndow
cont our {x1x2i neq1 [ gl val gl val ] ' r - ' ) ;
hol d on
dr aw anot her cont our at 10 t he const r ai nt boundar y
cont our {x1x2i neq1 [ O. l *gl val O. l *gl val ] ' b- ' ) ;
set {gca ' xt i ck' [0 0. 2 0. 4 0. 6 0. 8 l . O] )
set {gca ' yt i ck' [0 0. 2 0. 4 0. 6 0. 8 l . O] )
xl abel {' out si de di amet er ' ' Font Name' ' t i mes' ' . . .
Font Si ze' 12) ;
yl abel {' i nsi de di amet er ' ' Font Name' ' t i mes' ' .
Font Si ze' 12) ;
hol d of f
gr i d
t he f ol l owi ng code may be usef u1 f or t he consol i dat ed
f i gur e. Not used her e _ _ uncomment bel ow
[C2han2] cont our {x1x2i neq1 [ gl val gl val ] ' r - ' ) ;
cl abel {C2han2) ;
set {h2' Li newi dt h' 1)
k2 gt ext ( ' gl ' ) ;
set {k2' Font Name' ' Ti mes' ' Font Wei ght ' ' bol d' .
Font Si ze' 14 Col or ' r ed' )
f i gur e
cont our ( x1x2i neq2 [ g2val g2val ] ' r - ' ) ;
hol d on
cont our ( x1x2 i neq2 [0. 1*g2val 0. 1 *g2val ] b- ' ) ;
set {gca' xt i ck' [O 0. 2 0. 4 0. 6 0. 8 l . 0) )
set {gca ' yt i ck' [0 0. 2 0. 4 0. 6 0. 8 l . 0) )
xl abel {' out si de di amet er ' ' Font Name' ' t i mes'
Font Si ze' 12) ;
yl abel {' i nsi de di amet er ' ' Font Name'
' t i mes' ' Font Si ze' 12) ;
hol d of f
gr i d
[C3han3] cont our ( x1x2i neq2 [ g2val g2val ] ' r - - ' ) ;
cl abel {C3han3) ;
set ( h3' Li neWi dt h' 1)
k3 gt ext ( ' g2' ) ;
se ( k3' Font Name' ' Ti mes' ' Font Wei ght ' ' bol d'
f i gur e
cont our ( x1X2i neq3 [ g3val g3val ] ' r - ' ) ;
hol d on
cont our ( xl x2i neq3 [ 0. 1*g3val 0. 1*g3val ] ' b- ' ) ;
set ( gca ' xt i ck' [0 0. 2 0. 4 0. 6 0. 8 1. 0] )
set ( gca ' yt i ck' [0 0. 2 0. 4 0. 6 0. 8 1. 0] )
Font Si ze' 12) ;
yl abel ( ' i nsi de di amet er ' ' Font Name' ' t i mes'
Font Si ze' 12) ;
hol d of f

[ C4han4] m cont our ( x1x2i neq3 [ g3val g3val ] ' b- ' ) ;
cl abel ( C4han4) ;
k4 = gt ext ( ' g3' ) i
Font Si ze' 14' Col or ' ' bl ue' )
f i gur e
cont our ( x1x2i neq4 [ g4val g4val ] ' r - ' ) i
hol d on
cont our ( x1x2i neq4 [ 0. 001*g4val 0. 001*g4val ] ' b- ' ) i
set ( gca ' xt i ck' [0 0. 2 0. 4 0. 6 0. 8 1. 0] )
set ( gca' yt i ck' [0 0. 2 0. 4 0. 6 0. 8 1. 0] )
Font Si ze' 12) i
yl abel ( ' i nsi de di amet er ' ' Font Name' ' t i mes'
Font Si ze' 12) ;
hol d of f
gr i d
[ C5han5] cont our ( xl x2i neq4 [ g4val g4val ]
b- - ' ) i
cl abel ( C5han5) i
k5 gt ext ( ' g4' ) i
Font Si ze' 14' Col or ' ' bl ue' )
2.3 AOOI TI ONAL EXAMPLES 71
t he equal i t y and const r ai nt s ar e
not wr i t t en wi t h 0 on t he r i ght hand si de.
I f you do wr i t e t hemt hat way you woul d have t
i ncl ude [00] i n t he cont our commands
1neal ex3..... m ( t he f i r st const r ai nt )
f unct i on r et val i neq1_ex3( Xl X2)
bet ween var i ous m- f i l es
gl obal ELAS SI GALL TAUALL GAM FS GRAV
gl obal RHO CD FLAGWSPEED LP L DELT
AREA 0. 25* pi *( X1. ^2 - X2. ^2) i mat r i x
I NERTI A = pi *( X1. ^4 X2. ^4) ! 64i mat r i x
FD 0. 5*RHO*SPEED*SPEED*CD*X1i
MW 0. 25*FD*L*L. *X1. ! I NERT1Ai
MF 0. 5*FLAGW* LP*Xl . ! I NERT1Ai
SI GW GAM GRAV*Li
r et val MW+ MF + SI GWi SI GWi s added t o al 1 mat r i x
% el ement s
1nea2 ex3. m ( t he second const r ai nt )
f unct i on r et val i neq2_ex3( X1X2)
gl obal ELA8 8I GALL TAUALL GAM FS GRAV
AREA 0. 25* pi *( X1. ^2 - X2. ^2) i
I NERTI A pi *( X1. ^4 - X2. ^4) ! 64i
FD 0. 5*RHO*SPEED*SPEED*CD*X1;
8 FLAGW+ ( FD L) i
o ( X1. *X1 + X1. *X2 + X2. *X2) ! 6. 0i
r et val 8. *0. / NERTI A;
i nea1t : L. m ( t he t hi r d const r ai nt )
f unct i on r et val i neq3_ex3( X1X2)
gl obal st at ement i s used t o shar e same i nf onat i on
gl obal ELAS. SGALL TAUALL GAM FS GRAV
72 GRAPHI CAL opnMI ZATI ON
AREA 0. 25* pi *( Xl . ^2 - X2. ^2) ;
I NERTI A pi *( Xl . ^4 X2. ^4) / 64;
FD = 0. 5*RHO*SPEED*SPEED*CD*Xl ;
dw FD*L^4. / ( 8*E S I NERTI A) ;
df ( 2. 0 FLAGW*L^3 - FLAGW*L*L*LP) . / ( ELAS*NERTI A) ;
r et val dw + df ;
i n 4 ex3. m ( t he f our t h const r ai nt )
f unct i on r et val i neq4_ex3( Xl X2)
r et val Xl X2;
Fi gure 2. 9di spl ays thegraphi cal or consol i datedsol uti onto probl cm. l ecodc
is avai l abl e l hem.fi l eabove (by comment i ng thecodes creal c thenewfi gure
wi ndows and removi ng thecomment s on thecodc is currcntl ycommentcd). Thc
opti mal sol uti on is not very c1ear. Fi gu 2.10 is obtai nd by zoomi ng in near e
nei ghborhood of 0.6. Sec hel pzoomfor in ucti onon its use. lt can be achi cvedby
F1llgPolc blcm
0.81-
a
o
a
u
n
u
U
E
S
-
0.2
0.2 0.4 0.6 0.8
outsidedi amel er
Fl gure2.9 Graphl cal solulion: Exampl e 2.3.
2.3 ADDl TI ONAL EXAMPLES 73

0.58 0.6 0.62 0.64 0.66 0.68 0.7 0.72 0.76 0.8
outsidedi amCl cr
Fi gure 2.10 Graphi cal solutlon(zoomed): :ampl e2.3.
typi ngzoomat theworkspace prompt and usi ngthemouse todraga rectangl earound
e regi on that n ds to be enl arged. Fr omFi gure 2. 10 c sol uti on is around the
outsi dc di ameter of 0. 68 m d tllc i nsi dedi ameter of 0. 65 m. Typi ng zoomin
workspace agai nwill toggl e back tononnaJ state. InFi gure 2. 10 tick
marks arepl acedon thefi gurethrough ecommand wi ndowfor bcttcr inte tati on
of thesol uti on.
The graphi cs in e above ex npl e were created usi ng the same statemenl s
encountered previ ousl y. Col or contours were used toestabl i shthefeasi bl eregi on. he
zoomfeaturewas empl oy toobtai na bettcr esti matc of thesol uti on.
2. 3. 3 Exampl e 2. 4
This exampl e is u m earcaof heat transfer. The probl emis todesi gna tri angul ar
fin of smal l est vol ume that wi l l at least del i ver speci fi cd fin effi ci enci es. The
graphi cal t i tu of this code is very si mi l ar to Exampl e 2.3. ln is cxampl e the
i nequal i tyconstrai nts arecomput ed and retumed f roma si ngl efuncti onm.fi l erather
an separatc files consi deredin the previ ous exampl e. Another ncwfeal ure in this
exampl e is to i nvoke speci al mathemati cal functi ons the sesseI functi ons that a
74 GRAPHI CAL OPTl MI ZATI ON
avai l abl einMATLAB. sexampl e al soiI1ustr s a probl emwhere theopti mi zati on
probl emcan be adequatel y defi ned but the sol uti on is easi l y determi ned f romthe
equal i ty and si de cons aints. Inother words the probl emcan easi l y accommodat e
addi ti onal demandi ng constrai nts. Inl argecompl cx mathemati cal model s wi thmany
dcsi gnvari abl cs it is not easy toensure that at least one of thc i nequal i ty constr nts
I S active.
Des 'nPr obl em: M ze amount of materi al used in edi of ascri cs
of i denti cal tri angul ar fins that cover a gi vcnareaand operateat or abovc thespeci fi ed
cffi ci cnci .
Mat hemat l cal Model : Thi s type of probl emshoul d be abl e to accommodat e
several dcsi gn vari abl es. In this secti on e probl emis set up to have t wo desi gn
vari abl . 1efinmateri al is al umi num.
Desi gn Parameters: For al umi num
h = 50 W/ m
2
[convecti oncoeffi ci cnt]
k = 177 W/ m- K [thermal conducti vi ty)
N = 20 [ number of fins]
W=O. 1 m [wi d of thefins)
Fi ngap is th samc as thebase l engthof thetri angul ar fin.
Desi gn Vari abl es
b: base of thetri angul ar fin
L: hei ght of thc tri angul ar fin
Fi gurc 2.11 iJl us at egeom yof fins.
Geomet ri c Rel atl ons: Th fol l owi ngaresome of th areacal cul ati ons that areuscd
later inthedcvel opmcnt of ccons aints:
A
f
= ( 2N- 1)*b V footpri nt of efinand gap
Ac = 2*W*[L2+ (b!2)2] 112 finareafor heat transfer
Ab = ( N- l)*b*W gap ar
At = N*Ac + Ab total areafor heat transfer
Obj ecti ve Functi on: The total vol ume of ma ial for thefinis
j(b L) = O.5*N*W*b*L (2.16)
b b
FJ gure2.11 Flndesl gnfor heat transfer: Exampl e 2.4.
Constrai nt Funct ns: Thc heat transfer equati ons for the fin arc avai l abl e in
Reference 7. Fi ns aretypi cal l ymount ed on a speci fi ed . 1efirst cons int is one
on area:
h(b L): Af = 0. 015m
2
(2.17)
Note: is constrai nt essenti al l y fi xes theval ueof b. In egraphthis constrai nt wil
be a strai ght lineparal l el to cLaxi s. l f is wcre tohappen ina probl cmwi thscveraI
desi gnvari abl es it woul d be prudent toel i mi nate is vari abl c f romthemathemati cal
model by i denti fyi ngit as a desi gnpar leter.
eeffi ci ency of a si ngl efincan be establ i shed
r = (l / mL) 1(2mL)l I
o
(2mL)
where
m= (2h / )112
d el' s areBessel equati ons of thefirst ki nd. The first i ncqual i tyconstrai nt is
g(b : f ~ 0. 95
The overal l cffi ci ency for theheat transfer is
TJ
o
= l - N* 4
1
) (
The final i nequal i tyconstrai nt is
g2(b L): TJ
o
0. 94
Si de Constrai nts: The si decons ai nts
0. 001 SbSO. 5
0.01 S L S 0. 03
(2.18)
(2.19)
(2.20)
MATLAB Code: Si mi l ar to e previ ous exampl e. the hashmarks on the i nequal i ty
cons aint canbe establ i shedby pl otti ngt wo vaI ues of theconstrai nt: thelimit val ue
and a val uehi gher. Once yhav beendetermi ned. thentheconsol i datedfi gurecan
be establ i shed. Th codeshown herewill drawaI l thefuncti ons inseparatewi ndows.
For theconsol i datedplot this code necds to be edited. It is left to thestudent m
exercl se.
mThe mai n script file
% Exampl e 2. 4 Sec. 2. 3

Mi ni mumf i n vol ume f or ef f i ci ent heat t r ansf er
mat er i al Al umi num
- - - - - - - - - - - - - - - - - - -
gl obal N H K W AREA
" - - - - - - - - -
ni t i al i ze val ues
N 20 number of f i ns
W 0. 1 wi dt h of f i ns
50. 0 convect i on coef f i ci ent W/ m*m
K 177. 0 t her mal conduct i vi t y W/ m- K
AREA 0. 015 avai l abl e f i n f oot pr i nt ar ea
- - - - - - -
r i ght hand l i mi t s f or t he f unct i ons
h1val AREA;
gl val 0. 95;
g2val 0. 94;
- - - -
x1=0. 001: 0. 0001: 0. 005;
x2=0. 01: 0. 001: 0. 03; x1 and x2 ar e vect or s f i l l ed wi t h
number s
[X1 X2] meshgr i d( x1x2) ;
vect or s x1 and x2
f 1 = obj _ex4( X1X2) ;
t he obj ect i ve f unct i on i s eval uat ed over t he ent i r e
mesh
Const r ai nt s ar e eval uat ed
!
I't'
2.3 ADDI TI ONAL EXAMPLES n
eq1 eq1_ex4( X1X2) ; t he equal i t y const r ai nt
[ i neq1 i neq2]
i n t he above not e t hat t he t wo i nequal i t y const r ai nt s
ar e obt ai ned si mul t aneousl y. Keep i n mi nd
t hat each const r ai nt i s a mat r i x
r eady f or t he pl ot s
[C1han1] cont our ( x1x2f 1 [ 0. 00001 0. 00002...
0. 00004 0. 00006 0. 00008 O. OOOl ] ' g' ) ;
cl abel ( Cl han1) ;
set ( gca xt i ck' [ 0. 001 0. 0015 0. 002 0. 0025 0. 003
0. 0035 0. 004 0. 0045 0. 005] )
set ( gca ' yt i ck' [ 0. 010 0. 015 0. 02 0. 025 0. 03 0. 125] ) ;
xl abel ( ' f i n l engt h' ' Font Name' ' t i mes' ' Font Si ze' 12) ;
yl el ( ' f i n wi dt h' ' Font Name' ' t i mes' ' Font Si ze' 12)
gr i d
% hol d on
f i gur e open a new f i gur e wi ndow
p10t t i ng t he equal i t y const r ai nt
cont our ( x1x2eq1 [ h1val h1val ] ' r - ' ) ;
set ( gca' xt i ck' [ 0. 001 0. 0015 0. 002 0. 0025 0. 003
0. 0035 0. 004 0. 0045 0. 005] )
set ( gca ' yti ck' [ 0. 010 0. 015 0. . 02 0. 025 0. 03 0. 125] ) ;
yl abel ( ' f i n wi dt h' ' Font Name' ' t i mes' ' Font Si ze' 12)
gr i d
pl ot t i ng t he f i r st i nequal i t y const r ai nt
f i gur e
cont our (x1 x2 i neq1 [0. 968 0 . 968] r - ) ;
hol d on
cont our ( x1x2i neq1 [ 0. 9690. 969] ' b- ' ) ;
set ( gca' xt i ck' [ 0. 001 0. 0015 0. 002 0. 0025 0. 003
0. 0035 0. 004 0. 0045 0. 005] )
set ( gca ' yt i ck' [ 0. 010 0. 015 0. 02 0. 025 0. 03 0. 125J ) ;
yl abel (' f i n wi dt h' Font Nar ' t i mes' Font Si ze' 12)
hol d of f
gr i d
pl ot t i ng t he second i nequal i t y const r ai nt
f i gur e
cont our ( x1x2i neq2 [ g2val g2val ] ' r - ' ) ;
hol d on
cont our ( x1x2i neq2 [ 1. 01*g2val 1. 01*g2val ] ' b- ' ) ;
set ( gca' xt i ck' [ 0. 001 0. 0015 0. 002 0. 0025 0. 003
0. 0035 0. 004 0. 0045 0. 005] )
set ( gca' yt i ck' [ 0. 010 0. 015 0. 02 0. 025 0. 03 0. 125] ) ;
yl abel ( ' f i n wi dt h' ' Font Name' ' t i mes' ' Font Si ze' 12)
hol d of f
gr i d
nb. Le leobj ecti vefuncti on
vol ume of t he f i n
r et val 0. 5*N*W*X1. *X2
ql J . mThe equal i ty int
f unct i on r et val eql _ex4( Xl X2)
t he equal i t y const r ai nt on ar ea
r et val ( 2. 0 N - 1) *W*Xl ;
The constrai nts
f unct i on [ r et l r et 2]
r et ur ns bot h t he i nequal i t y const r ai nt s
c 2*sqr t ( 2. 0) *sqr t ( / K) *X2. / sqr t (Xl ) ;
r et l ( bessel i ( 1c). . / ( ( 0. 5*c) . *bessel i ( Oc) ) ) ;
Ae 2. 0*W*sqr t ( ( X2. *X2 + . 25*Xl . *X1) ) ;
Ab ( N- 1) *W*X1;
At N*Ae + Ab;
Ar Af . / At ;
r et 2 ( 1. 0 - N *Ar . (1 - r et 1) ) ;
The above script files shoul ddi spl ay each functi onina scparatefi gure wi ndow.
The i nequal i ty constnl i nt fi gures shoul dcontai nt wo contours wi th c bl uecontour
i ndi cati ng efeasi bl eregi on. Fi g 2.12di spl ays a composi te pl ot The sol uti onis
detcnni nedby theequal i tyconstrai nt and thel ower Ii mi t on efi nl eng di scussed
previ ousl y.
1m
1 l
E
. .
11<'
."'
2.4 AoomONAl MATl .AD GAAPHI CS
0.045
iO
0.025
1.5 2 2.5 3 3.5
finbase(m)
gure2. 12 Graphl cal solulion: Exampl e 2.4.
2. 4 ADDI TI ONAL MATLAB GRAPHI CS
79
S
x10
-3
This secti on pr ovi da bri ef exposure to addi ti onal features of Handl e Graphi cs
inc1udi ng devel opment of a GUI in MATLAB. The coverage is very modest and is
inc1uded hcre for thc sake of compl cteness as we have accompl i shcd our goal of
graphi cal opti mi zati on e previ ous secti ons. Thi s secti on can be avoi ded
postponed or emphasi zed for the proj ects in e course as it has no addi ti onal
i nformati on for graphi cal opti mi zati on. It does contai n reasonabl c i nfonnati on in
undcrstandi ngMATLAB graphi cs and programmi ng. The reader is encouragedtotake
somc ti megetti ngfami l i ar wi th eversi onof MATI . ABi nstal l edon his machi ne. There
are usual l y si gni fi cant enhancements in newcr versi ons of MATLAB. For exampl e
MATLABVrsi on5. 3( is book is basedon Versi on5.2) al l ows you tocustomi ze your
plot and add annotati ons toit roughaddi ti ona1edi ti ngt i 1 of wi ndow.
You do not have tocustomi ze your fi gurethroughcode al one. Inthis secti on we will
a1so be devel opi ng the GUI usi ngMATLAB Handl e Graphi cs so we have more
experi en wi MATI . ABHandl e Graphi cs. MATI . ABdoprovi dc a GUI creati ontool
gui de at theuser is recommended toexpl ore.
Intheprevi ous secti ontherewere onl y two desi gn vari abl es. An cql I y spaced
vector was obtai nedfor eachvari abl c basedon si dc constrai nt. A mshgri dwas
engenerated. Al I functi ons werc eval uatedon this grid. Contour pl o15were drawn
for thefuncti ons representi ng obj ecti veand constrai nts. Except for theobj ecti ve
functi on whi ch di spl ayed several contours the constrai nts were onl y drawn for a
si ngl eval uc. Thi s procedure suggests thepossi bi l i tyof automati on. User i nput will be
necessary for sel ccti ng ngc for thevari abl es as wel l as thc functi ons at will be
pl otted.
Inthis sccti on we will createpl otti ngfacility toi ncl udea GUI that wi l l
Obtai n enumber of pl o15
. The range and i ncrcment of thedesi gnvari abl es
Use a filesel ecti onbox/ utility at theuser will us to i denti fy efuncti onto
be pl otted
Prompt theuser for thenumber of contour for each pl ot
Al l owsel ecti onof contour l evel s
Creatc thepl ot
leexcrci sewi l l requi rc t wo m-fi l es: one 10set up eGUI and eo r toassist in
thepl otti ng. The user will be abl etotweak thc pl ot intheCommand wi ndowafter it
is drawn. The fol l owi ngsubsecti ons psent a bri ef i ntroducti ontoHandl c Graphi cs
and GUI control s inVersi on 5.3.
2. 4. 1 Handl e Gr aphi cs
MATI.. AB Handl e Graphi cs refer to a col l ecti on of l ow-l evel graphi c routi nes that
actual l ygeneratc or carryout graphi cal changes you s i nt hcMA . ABfi gurewi ndow.
For themost part theserouti nes aretransparent dtypi cal l yusers nced not be awarc
of them. Infact tounderstandopti mi zati onand toprogramthetechni ques inMATI.. AB
it is not necessary to knowabout Handl e Graphi cs si nceusi ngthehi gher-l evel pl ot
commands such as pl ot and contour is suffi ci ent toget thej obdone. "Thesc commands
actual l y ki ck in scvcral of e Handl e Graphi cs routi nes 10 di spl ay e pl ot on the
fi gure. A noteworly feature of MATI ..AB graphi cs is it is i mpl emented
obj ect ori entedmanncr. What this means is that most of the es you seeon the
fi gurc Ii ke axes l abel s tcxt and lines arc all obj ec15. This mcans that most of the
graphi cal i tems on the fi gurehave properti es C bechanged roughprogram
code. Intheprevi ous secti ons we used this tomani pul ate col or font si ze lin wi d
and theIi ke.
Inorder to change eprope es of obj ects it is nec sary to i denti fy or refer to
thcm. Obj ects inMATLAB arei denti fi edby a uni que number ref erredas a handl e.
Handl es can be assi gnedtoall obj ec15when they arc crcatcd. Some obj ec15 partof
other obj ccts. Obj ects that contai nother obj ec15arc referredtoas contai ner or parent
. There is a defi ni tc hi erarchi s cture among obj ects. 'he root of all
MATLAB graphi c 0 15is efi gureobj t . The fi gure0ect for exampl e can hol d
sevcral GUI control s whi ch areknown as ui control obj ec15. It 31socontai ns the es
2.4 AOOI TI ONAL MATAB GRAPHI CS 81
obj t . obj t in t umhas the line text and i mage obj ect. Al l 0ects in
MATLAB can be associ atedwi aset of properti es at is usual l y based on i15functi on
or useful ncss. Di fferent obj ect types have di fferent sets of properti es. Each of thcs
prope es is descri bcdthrougha pai r of rel atedi nformati on(name value). efirst
el ement of this pai r is the namc for the property and the second is the val ue
correspondi ngto property. When eoect is created it i nheri ts all of thedefaul t
prope es. For exampl e:
pl ot ( si n( 2*) ' - go'
Li neWi dh' 2
Mar ker EdgeCol or ' ' k'
Mar ker FaceCol or ' [ 0. 49 1 0. 63]
Mar ker Si ze' 12)
The last four 1i nes abovc represent four properti es of e pl otted line that is bei ng
changed. The property narn on e Icft can be easi l y understood. leformat si ze
uni ts areinpoi nts. The col or val ue is a r owvcctor of col or val ues between zero
and onc representi ngthered green and bl ueval ues respecti vel y. There aresevcral
othr prope es of the pl otted line whi ch will be sct at thcir defaul t val ues. Thc
MATLAB onl i nercferenceshoul d Ii st all obj ec15and thcir prope es.
In general . obj ect properti es cllnbe changed by usi ng theser functi onto change
their val ues. Si mi l arl y obj ect properti es c bercadby theuse of theget functi on. To
i nqui reabout theproperty of an 0t :
get ( handl e' Pr oper t y Name' )
ge ( h_1' Col or ' )
Intheabove h_l is ehandl e of theobj ect whose col or is desi rcd. Si mi l arl y
se ( handl e ' Pr oper t y Name' ' Pr oper t y Val ue' )
set ( h_l ' Col or ' ' r ' ) or set ( h_1' Col or ' [1 0 0] )
intheabove val ueof Col or for theobj cct represcnt by thehandl c h_l is set 10
rcd in t wo ways. The sccond speci f i the col or red through the vector val uc. It is
expectcdthat theobj ect i denti fi edthroughh_1 shoul dhave a Col or property. MATt. AB
wi l l i nf ormyou of thc error incase you areassi gni nga property/val ue tisnot val i d
for theobj ect.
2. 4. 2 Graphl cal User I nterf ace
The most si gni fi cant featurc of software desi gn and devel opment today is the
obj ect-ori enteddesi gnparadi gm was bri efl y iII ustrated in eprevi ous secti on.
The combi nati on of theabove wi thevent-dri venprogrammi ng is botha natural and a
qui redfcaturc incurrent softwarc programs. I nsteadof theprograms control l edby
a predetermi ned processi ng s uence (batch processi ng) today' s processi ng is
govemed by the user through some dirl ted mouse cl i cks and buttons. These user
sel ecti ons cause certai nevents that i nstruct thesoftwaretocarryout some acti on. The
o 1fire e evenar e usual l y referred 10 user inte el cmc . It is
i mportant to understand that thc same evcnts can be tri ggered by obtai ni ng user
responsc to sc en prompts al the command linc (unattracti vc). Usi ng buttons to
achi eve the same resul t i5 more gl amorous al though it uses mor e compu ti on
sources. l nfact for commerci al success it is n essarytoi ncl udethis mechani smof
user inte. racti oninthesoftware product . Al so. users expect theseel ements 10behave
in a standard way. There are many el ements the user can use to i nteract wi th
program5. Many appl i cati ons across several di sci pl i nes havc nowdefi neda mi ni mum
5tandardcol l ecti onof eseel ements. MATLAB 50provi dcs them. The organi zati on
of eseel emenl s and their presentati ontotheuser ina graphl cal manner is i denti fi ed
as egrphl cal user i nterfa . A number of GUI - based t1s. ncti ons. avai l abl e
at theMATLAB site.
Thi s secti onprovi des onl y a bri ef i ntroducti onto eel ements used intheexampl e.
Thseel cments arepri mari l y usedthroughMATLAB Handl c Graphi cs programmi ng.
Consi der thecasc of a button at theuser can push tocause some acti on. referredto
as a push buttoninthc subsequent di scussi on. At theoutset. thereru two pri nci pal
requi remcnts at arecxpectl of thepush button. Fi rst the er must be abl etosee
it. Second. whcn euser cl i cks on it (si ngl eversus doubl e cl i cks can be di sti ngui shed
inmany software devel opment codes) thebuttonmust spondwi someexpected
acti on. InMA TLABthepush buttonis a graphi cal ott. It has many prope thatcan
be used todescri beits app ranceon thc screcn. InMA'LAB. all graphi cal obj ects can
onl yappe in a fi gurc wi ndow. Hencc a fi gure wi ndowis essenti al to di spl ay thc
push button. is fi gure wi ndowact sa contai ner and can contai n many user
i nterface el cmcnts. Consi deri ng the presentati on of the button on thc scrccn it is
nessary todcci de where thebuttonshoul dapp r on ef i gw wi ndowhow bi git
shoul dbe what l abel it shoul d have (typi cal l y i ndi cates ki ndof acti onit is l i kel y
cause) d so on. Si ncc the push button is a graphi cal obj ect much of these
prope esc beset if therc is a handl e avai l abl efor I hepush-buttonobj cct. Once the
graphl 1appearance is tak
Hpush ui cont r ol ( Hf i g_1' St yl e' ' push'
' Uni t s' ' i nches'
EJ
11
2.4 ADDI TI ONAL MATLABGRAPHI CS 83
' Posi t i on' [ 2. 50. 10. 7 . 2]
' Backgr oundCol or ' ' r ed'
St r i ng' EXI T'
' Cal l back' ' cl ose' ) ;
Intheabovc Ii nes of codc. Hpush is thehandl e tothepush-buttonobj ect . It is created
by usi ngthefuncti onui control . Thi s is thefuncti onthat wi l l createtheuser i ntcrface
el ements. The same functi onis used topl aceseveral di erent user i nterfaceel ements
through thesel ecti onof thestyleproperty. Intheexampl e bel ow. thesei ncl udelabel
boxcs and user edi tabl etexl boxes. Several parameters i denti fy Lhe typ of l emcnt
UEat is createdand theproperti es UEat need to beset. Each propeny is set througha
propertynamel propertyval ue pair. Hf i l i n I i ncs of code abovc to the
handl e of thefi gurewi ndowinwhi ch this el ement wiUbe pl aced. The typeof el ement
createdis a push-buttonel ement cstabl i sh through Styl eproperty s ngname
push. " The Uni ts used to si zeand posi ti onthepush buttonis i nches. The Posi ti on
property is a r owvector of fo val ues gi vi ngtheLeft. Bottom. Wi dth. Hei ght of the
el ement intheUni ts scl ectedearlier. The defaul t l ocati onof theori gi nis thebottom
lcft comer of thefi gu wi ndow. The BackgroundCol or for thebuttonis sct tored. Thc
l abel that wilbe on thefaceof thepush buttonis EXI T. Thi s is theTCXl property of
thepush-buttonobj ect . Thc Cal l Back property of thepush buttonis estri ngcl ose."
This stri ngwill be thc parameter intheeval functi on at is cal l edwhen pushl cl i ck
event is tri gge d. MATLAB command cl ose wi l l be executed in the Command
workspace when the push button is p ssed. Thl s will cause l he fi gure wi ndow
di spl ayi ngthepush buttontoc1osc.
Onl y a f ewel ements areused in is sccti on. The Cal1Back stri ngs areal soqui te
s i ghor w . Bef ore dcvel opi ng thc code. it is useful to observe hl erarchi cal
order in whi ch thegraphi cal obj ects in MATLADareorgani zed(reproducedf rom e
MATLAB documentati on) inFi gure 2.13. The root i mpl i es thestarti ngresourcefor aIl
graphi c di spl ays inMATLAB. Its handl e usual l y has a val ueof O. The root al soserves
as a contai ner for all of thefi gurewi ndows. Another exprcssi onfor is rel ati onshi p
is that a fi gurewi ndowis thechi l dof theroot . Fr omthehi erarchy it isessenti al not e
e user i nterfaceel ements. ui conr ol or ui menu c onl y exi st wi in a
Ro+ot
Ul control Axes Ul menu
I mage Ll ne Pat ch Surf ace Text l i ght
F1gure 2.13 Hlerarchical slruclurelor MATl AB graphl cal objecls.
84 GRAPHI CAL OPTI Ml ZAnON
figurewi ndow. 80 al50the e5obj ect . Inthefol l owi ngcode eachinterfaceel ement
is first associ atedwi ahandl e50 epropertyof theel ement canbe fen to
if necessary. Thi s is especially trueof thetext entry boxes to i nvok theCal l 8ack
s ng5.
2.4.3 GUI Code
lefol l owi ngcodewil1generatetheGUI. The ui wai t quires thal is wi ndowbe
cl osedbeforesubsequent processi ngcantakepl ace(or ui r esume beuSedin E
de) .
. Gl l . . yl
GUI f or Pl ot t i ng f aci l i t y
D . P. Venkat ar aman Appl i ed Opt i mi zat i on Usi ng MATLAB
% Ch. 2 Sec. 2. 4
GUI i s cr eat ed wi t hi n a f i gur e wi ndow

- - - - - - -
Hf i g_1 ' " f i gure; Hf i g_1 i s t he handl e t o t he
f i gur e wi ndow
set ( Hf i g_l ' Col or' [0. 30. 60. 5] . . . set col or
Number Ti t l e' ' of f ' no wi ndow t i t l e
Name' ' Set Range f or Desi gn var i abl es'
' Uni ts' ' i nches' i nches used t o l ayout
% ot her cont r ol s
' Menubar' ' none'
' Posi t i on' [444. 41. 5]);
Posi t i on pr oper t y i s [l ef t bot t om wi dt h hei ght] i n
Uni t s sel ect ed
Ht _1 ui cont r ol ( Hf i g_1
St yl e' t ext '
' Uni ts' ' i nches'
' posi t i on' [0. 11. 12. 4 .2]
' St ri ng' ' Number of Funct i ons t o Pl o ) ;
]

J
'
n
u
-
-
1
q
d
'
l
s
L
'
'

H
'
8
6

f
h
U
L
o
-
-
C
I

r
G
n
'
t
e
-
-
'

n
'
'
n

o
'
'
o

c
'
'
1
e
s
u
l
t
i

y
i
s

z
t
n
o

s
u
p

4
'
'
'
t

H

2.4 ADOI TI ONAL MATLADGRAPHI CS 85
Backgr oundCol or ' ' whi te'
St ri ng' '
' Cal l back' ' Np st r 2nUm( get ( Ht _2" St r i ng" ) ) i ' ) ;
HL X1L ui cont r ol ( Hf i g_l
' St yl e' ' text'
' Posi t i on' [0. 10. 80. 5 .2]
' St ri ng' ' x1( mi n) ' ) i
HT_XI L ui cont r ol ( Hf i g_l
' Styl e' ' edi t '
' Posi t i on' [0. 70. 80. 7 .2]
' Backgr oundCol or ' ' whi te'
' St ri ng' "
' Cal l back' ' xl mi n st r 2num( get ( HT XI L'
St r i ng' )) i ' ) j
HL_XI M ui cont r ol ( Hf i g_1
' Styl e' ' text'
' Posi t i on' [1. 50. 80. 6 .2]
' St ri ng' ' i ncr emen' ) i
HT X1M ui cont r ol ( Hf i g_l
' St yl e' ' edi t '
' Posi t i on' [2. 20. 80. 7 .2]
Backgr oundCol or ' ' whi te'
St ri ng'
' Cal l back' ' x1i nc st r 2num( get ( HT_X1M'
St r i ng' )) j ' ) j
HL_X1U ui cont r ol ( Hf i g_1
St yl e' t ext '
' Posi t i on' (3. 00. 80. 5 .2]
' St ri ng' ' xl ( max) ' ) j
HT_X1U ui cont r ol ( Hf i g_l
' Styl e' ' edi t '
' Posi t i on' (3. 60. 80. 7 .2]
86 GRAPHI CAl OPTI MI ZATI ON
BackgroundCol or' ' whi te'
St ri ng'
' Cal l bacl ' x1max st r 2num( ge ( HT_X1U
Stri ng' ) ) ; ) ;
E _X2L ui cont r ol ( Hf i g_l
Styl e' text'
' Posi t i on' [0. 10. 50. 5 .2)
' stri ng' ' x2( mi n) ' ) ;
HT_X2L = ui cont r ol ( Hf i g_l
St yl e' edi t '
' uni ts' ' i nches'
' Posi t i on' [0. 70. 50. 7 .2)
' BackgroundCol or' ' whi te'
St ri ng'
' Cal l back' ' x2mi n st r 2num( get ( HT_X2L'
St ri ng' ) ) ; ) ;
HL_X2M ui cont r ol ( Hf i 9_1
St yl e' text'
' Posi t i on' [ 1. 50. 50. 6 .2)
' Stri ng' ' i ncrement ' ) ;
HT_X2M ui cont r ol ( Hf i g_1
Styl e' ' edi t'
' posi t i on' [2. 20. 50. 7.2)
' Stri ng' '
' Cal l back' ' x2i nc st r 2num( get ( HT_X2M'
St ri ng' ) ) ; ' ) ;
HL_X2U ui cont r ol ( Hf i g_1
' Styl e' ' text'
' Posi t i on' [3. 00. 50. 5.2]
' stri ng' ' x2( max) ' ) ;
HT_X2U ui cont r ol ( Hf i g_1
' Styl e' ' edi t'
2.4 ADDITIONAl MA11AI i GRAPHI CS 87
' Posi t i on' [3. 60. 50. 7 .2]
St ri ng' '
Cal l bacl ' x2max st r 2num( get ( HT_X2U'
St ri ng' ) ) ; ) ;
Hl abel ui cont r ol ( Hf i g_1
Styl e' text'
' posi t i on' [0. 10. 12. 2 .21
' BackgroundCol or' ' y'
' Stri ng' ' Af t er ent er i ng val ues pl ease pr ess
EXT' ) ;
Hpush ui cont r ol ( Hf i g_1' Styl e' ' push'
' uni ts' ' i nches'
' Posi t i on' [2. 50. 10. 7 .21
Backgr oundCol or ' ' red'
St ri ng' ' EXI T'
' Cal l back' ' cl ose' ) ;
ui wai t
Intheabovecode estatemenlSshoul dbecY tointerpret . Much of thecodec be
edit after copy andp teoperation. The last command ui wai t instruclSMATLAB
tohol doff executi onuntil theuser closes eGUI wi ndow. Fi gu 2.14 shows the
i mage of theGUI i tapp rsinMATLAB. letext boxes arcinitiali z.edwi thno val ues
forcing e user to set values ereby causi ng al1 of the Cal l Back strings to be
evaluated. Inaddition th dataenteredby theuser isa s ing. It has tobeconvertedto
a number. The dehe is basic. It does not verify at theuser didenter a number.
is verification can be done by extendi ng e Cal l Back string over mul ti pl e
statemenlS. Thi s is left anexerciseinprogrammi ng for theuser.
The script m-fileusedtoselect dcreatethepl0lSis as follows.
m. m
Scr i pt f i l e t o go wi t h Cont our pl ot t i ng
Dr. P. Venkat ar aman Appl i ed Opt i mi zat i on Usi ng MATLAB
-
El 1l i E

E
Flgure2.14 Imageof GUI.
88 GRAPHI CAl OPTI MI ZATl ON
Ch. 2 Sec 2. 4
scr i pt M f i 1e f or 20 Cont our P10t t i ng of f unct i ons

% i nvo1ved i n opt i mi zat i on
A11 f unct i ons ar e expect ed t o be avai 1ab1e as
MATLAB f unct i on m- f i 1es
Funct i ons wi l 1 be se1ect ed t hr ough t he i nput se1ect i on
box
F unct i on i nput par amet er s wi 11 be meshed mat r i ces
based on t he r ange of par amet er s avai 1ab1e f r omGU
The Number of p10t s and r ange f or t he p10t

ar e obt ai ned f r omr unni ng t he
user i nt er f ace scr i pt GUI 2_4 pr i or
t o t he ca11 t o P1ot 20. m
GU2 4 cal l t he GUI

Xnf o Yl nf o ar e i nt r oduced f or i nt r oduci ng def aul t
val ues
Xi nf o [ xl mi n x1max x1i nc1 ;
t he va1ues x1mi n x1max x1i nc ar e avai 1ab1e i n
t he Command wor kspace due t o Ca11Back
i f i sempt y( Xi nf o)
Xi nf o [- 4 4 0. 051 def au1t
end
f pr i nt f ( ' n' )
Yi nf o [ x2mi n x2max x2i nc1 ;
i f i sempt y( Yi nf o)
Yi nf o 4 4 0. 051
end
f pr i nt f (' n' )
xvar Xi nf o( 1) : Xi nf o( 3) : xi nf o( 2) ;
yvar Yi nf o( 1) : Yi nf o( 3) : Yi nf o( 2) ;
[X1 X21= meshgr i d( xvar yvar ) ;
% xl vect or
% x2 vect or
mat r i x mesh
set def au1t number of p10t s t o 1
i f i sempt y( Np)
Np 1;
end
c1 ; c1ear f i gur e
rl1:
2

1
1

2.4 AODI TI ONAl MA11AB GRAPHI CS 89

f or Mp10t l : Np
t ext 1 [ ' The f unct i on whi ch i s bei ng p10t ed must
' nbe a MATLAB'
' nf unct i on M - Fi 1e. Gi ven a meshed mat r i x i nput '
' ni t must r et ur n a Mat r i x'
' nP1ease se1ect f unct i on name i n t he di a10g box .
and hi t r e t u r n n ' 1 ;
% t hi s pr i nt s t he t ext t o t he scr een i n t he command
wi ndow and ser ves as a pr ompt
t he n i s a new 1i ne command
f pr i nt f ( t ex1)
usi ng t he ui get f i 1e di a10g box
[ f i 1e pat hl ui get f i 1e( ' c: *. m' ' Fi 1es of t ype
MATLAB m- f i 1e' 300300) ;
check i f f i 1e i s st r i ng
st r i p t he . mext ensi on f r omt he f i 1e so i t can be
ca11ed by t he pr ogr am
i f i sst r ( f i l e)
f unct name st r r ep( f i l e' . m' " ) ;
e1se
f pr i nt f ( ' nn' )
t ext 2 [ You have chosen CANCEL or t he f i 1e was
' nnot accepab1e'
' nThe pr ogr amneeds a Fi 1e t o Cont i nue'
' nP1ease ca11 P1ot 20 agai n and choose a f i 1e OR '
' npr ess t he up- ar r owbut t on t o scr o11 t hr ough
' npr evi ous commands nn'
Bye 1' ] ;
er r or ( t ext 2) ;
end
c1ear t ext 1 t ext 2; c1ear s t he var i ab1es t ext 1 .
and t ext 2 f or r euse
c1ear Fun maxva1 mi nva1 st r con conva1ue oneva1ue
1abcont 1abcont U
F un f eva1( f unct nameXl X2) ;
maxva1 max( max( Fun) ) ;
mi nva1 mi n( mi n( Fun) ) ;
f pr i nt f (' The cont our r anges f r omMI N: 12. 3f MAX.
12. 3f ' mi nva1maxva1) ;
f pr i nt f ( ' n' ) ;
st r con i nput ( ' DO you want t o set cont our va1ues ?
90 GRAPHI CAl opnMI ZATI ON
[ no] : ' ' S' ) ;
st r conU upper ( st r con) ;
i f st r cmp( st r conU' YES' ) I st r cmp( st r co ' Y' )
f pr i nt f ( ' I nput a vect or of cont our l evel s ) ;
f pr i nt f (' n' )
f pr i nt f ( ' bet ween 10. 2f and 10. 2f ' mi nval maxval ) ;
f pr i nt f (' n' )
conval ue i nput (' I nput cont our l evel as a Vect or : ' );
l abcont i nput ( ' Oo you want l abel l ed cont our s ?
[ no] : ' s' ) ;
el se
l abcont U upper ( l abcont ) ;
i f st r cmp( l abcont U' YES' ) I st r cmp( l abcont U' Y' )
[Ch] cont our ( xvar yvar Funconval ue) ;
cl abel ( Ch) ;
el se
cont our ( xvar yvar Funconval ue) ;
end
ncon i nput ( ' I nput number of cont our s [20] : ' ) ;
i f i sempt y( ncon)
ncon 20;
l abcont i nput ( ' Oo you want l abel l ed cont our s ?
[ no] : ' s' )
i f st r cmp( l abcont U' YES' ) I st r cmp( l abcont U' Y' )
[Ch] cont our ( xvar yvar Funncon) ;
cl abel ( Ch) ;
el se
cont our ( xvar yvar Funncon) ;
end
el sei f ncon 1
oneval ue i nput (' I nput t he si ngl e cont our l evel ' ) ;
l abcont i nput (' 00 you want l abel l ed cont our s ?
[ no] : ' s' )
l abcont U upper {l abcont ) ;
i f st r cmp( l abcont U ' YES' ) I st r cmp( l abcont U' Y' )
[Ch] = cont our ( xvar yvar Fun [ oneval ueoneval ue] ) ;
cl abel (C h) ;
el se
end
el se
cont our ( xvar yvar Fun [ oneval ueoneval ue] ) ;
REFERENCES 91
l abcont i nput ( 00 you want l abel l ed cont our s ? [ . . .
no] :' ' s' ) ;
i f st r cmp( l abcont U' YES' ) I st r c ( l abcont U Y' )
el se
end
end
[Ch) cont our ( xvar yvar Funncon) ;
cl abel ( Ch) ;
cont our ( xvar yvar F l ncon)
end
i f Np > 1
hol d on;
end
Hf gcf ;
end
f i gur e( Hf ) i
gr i d
hol d of f
In the above the pl otti ng commands have been used before. The newMATLAB
commands arei sempt y cl f i sst r st r r ep max mi n upper st r cmp and
f pr i nt . Use thehel pcommand 10l earnmore about these. Run thescript file d
understandthes uenceof acti ons as wel l eprompts regardi ngthecontour. In
isexampl e thefuncti ons creal ethepl ot do not need gl obal val ues for cal cul ati on
sfe' edf rom ecommand workspace. Inother words thefuncti ons aretobe
pl ottedcanbe cal cul atedi ndependentl y.
REFERENCES
1. MATLAB The Langooge of Tecl l1l ical Compl ui ng: Usi l l g MATLAB Versi on 5
Ma Wo sInc. 1998.
2. MATLAB The Language of Techni cal CO ul i ng: Usi ng MATLAB Graphi cs Versi on5
MathWorks Inc. 1996.
3. Bohachevsky 1. 0. 10hn80n M. E. andSlein M. L. General i zedSi mul atedAnneal i ng
for FuncL i onOpti mi zati on Tecl mometri c Vol. 28 No. 31986.
4. Arora J. S. l ntroducl i on10 Opl i l l l umDesi gn McGraw- Hi I I NewYork 1989.
5. Ber ' F.P. andJ0 s E. R. . Jr
York1992.
6. Fox R. W. andMcDonal d A. T. l ntroducl i on10 Fl ui dMechani cs 4 ed. Wi l ey New
York1992.
7. Arpaci V. S. Conducti ol l Heat Tral l Sfer Addi son-Wesl ey Readi ng MA 1966.
8. Hansel man D. andLi ttletield B. Masteri ngMatl ab5 A Comprehensi veTutori al and
Reference. MA' ' LABCurri cul umSeries Prenti ce-HaJ l Engl ewood Cliffs. NY.
1996.
9. Bui l di ngGUI s wi MATLAB. onl i nereferenceinpdf format. Math Works Inc.. 1998.
PROBL EMS
( Many of thegraphi ca1enhancements arenowpossi bl ethroughpl otedi t features in
la r rel eases of MA1LAB. The fol owi ng probl ems arej ust suggesti ons. The suggested
probl ems use addi ti onal graphi cal el ements. and a1sor ui redel eti nghandl es. Pl eas
consul t MATI.. ABdocumentati on. )
2.1 Pl ot theba1lis aj ectory o msi mpl e two-di mensi ona1m hani cs.
2.2 Produc anani mated di spl ayof t11 j ectory.
2.3 Createa two-di mensi onal programthat will di spl ay a randomfi ri ngl ocati on
and a randomtarget l ocati on. Al l ow the user to choose initi a1 vel oci ty
magni tude and angl e. Pl ot his trectoryand ca1cul atehis error.
2.4 Createa thre di mensi onal program at wi l l di spl ay a randomfiringl ocati on
and a randomtarget l ocati on. Al l ow e user to choose initi a1 vel oci ty
magni tude and di recti onangl es. Pl ot his traj ectoryand cal cul atehis error.
2.5 Dr awtheboundary l ayer profi l eof l ami nar l10w over a l1at plate. Dr awlines
i ndi cati ngvel oci typrofi l eat tenpoi nts intheboundary layer.
2.6 Create a program at will pl ot vel oci ty profi l e for user-speci fi c i nputs.
AlI ow theuser toexpress his i nputs througha di a10g box.
3
LI NEAR PROGRAMMI NG
The maj or part of this book deal s wi thmnthemnti ca1model s nt nrechnrncteri zedby
nonl i near equati ons. Ii ke most of exampl es used for iII ustrati on us far. The
presence of a si ngl e nonl i near equati on in the model is suffi ci ent to i denti fy the
probl emf romthe cl ass of nonl i near programmi ng ( NLP) probl ems. Mathemati cal
programmi ng is ano er termthat is used todescri besuch model s and eir sol uti on
chni ques. Most of engi neri ngdesi gnfalls inthiscategory. Havi ng representedsuch
probl ems graphi ca11y in the previ ous chapter. it app rs the gradi ent and th
C va re of ncti ons i n the mathemati cal model had a si gni fi cant i mpact on
i denti fyi ng e sol uti on even though no effort was made to poi nt out is feature
sp ifically. In fact graphi cal sol uti on was obtai ned by i nspecti on rather than
determi nedby mathemati cal rel ati ons or entities.
There isan equal l y i mpo tcI ass of probl ems whos mathemati cal model is made
up excI usi vel yof functi ons that areonl y Ii near such as Exampl e 1.3 model edin
Equ ons 1. 21- 1. 24. The same exampl e w modi f i edinEquati on51. 25-1. 2850 t
a graphi cal sol uti oncoul dbe di 5cusSed. It is app ent o m graphi ca1descri pti on
in Fi gure 1. 6 at is no curvature evi dent in e fi gure. These probl ems are
md l i near programmi ng (LP) probl ems. They are natural in e subj ect of
operati ons research whi ch covers a vast vari ety of model s used for several ki nds of
deci si onmaki ng. Ex npl e1.3does represent a deci si on-maki ngprobl em. Typi cal l y
L P canbe a course by itself (or several courses for tmatter). A si gni fi cant porti on
of such a co woul dbe to devel op mathemati ca1model s f romdi fferent areas of
appl i cati ons as there is usua11y one numeri cal techni que at is commonl y used to
sol veL P probl ems. It is cal1ed thesi mpl ex met hod and is based on theal gori thmby
D ig [1]. The met hod i nvol ves mostl y el ementary r owoperati ons typi cal1y
encountered in Gauss-el i mi nati on type m ods that are part of the numeri cal
93
94 Ll NEAR PROGRAMMI NG
techni ques used inl i near al gebra. I n is chapter onl y a l i mi l ed but useful di scussi on
i sp sented. First it isi mportant tounderstandL P probl ems model i ngi ssues and
their sol uti on as ey are di fferent f rommost of e other probl ems in this book.
S ondm Yof current numeri cal techni ques for NL P probl ems obtai n their
sol uti on by l i neari zi ng the sol uti on at the current desi gn poi nt. These l i neari zed
uati ons n be sol vedby emet h S of this chapter. le ader is dir tedto y
of thebooks on L P for a mor e detai l eddescri pti onof themodel s and thetechni ques
as thep sentati onin is book is si mpl e and brief. References 1- 3 areuseful for this
pu ose.
Exampl e 1.3wi l l be used todefi nethetermi nol ogy associ atedwi LPprobl em. le
modi fi edversi oDof theprobl emis used 10devel op eformat of theL P probl emso
at we canfol l owthroughwi th egraphi cal sol uti on. The mathemati cal model has
t wo desi gn vari abl es xJ X2 thal represent e number of Component Pl acement
Machi nes of typeA and B. respecti vel y. The obj ecti veis 10maxi mi ze thenumber of
boards tobe manufactured. Constrai nt gl repr ts eacqui si ti ondoJ] ars avai l abl e.
Constrai nt g 2 represen15 e fl oor space constrai nt . Constrai nt g3 represen15 e
numbcr of operators avai l abl c.
Maxi mi ze j(x): 990 XI + 900 X2 + 5250 (3.1)
Subj ect to: gl(x): 0.4XI + 0. 6X2 S 8.5 (3.2)
g2( X) : 3 XJ - X2 S 25 (3.3)
g)(x): 3 XJ + 6 X2 S 70 (3.4)
xJ ~ O ; X 2 ~
le probl em. defi ned in Eq ti ons (3. 1) (3.4) is a natural representati on of
mathemati ca1 model in engi neeri ng desi gn or y other di sci pl i ne. By al l owi ng
ual i ty (=) constJ 015 dgr ter anor eq 1() constrai n15theprobl emcoul d
accommodat e any mathemati cal model at is characteri zedby l i near functi ons. Thi s
represental i on can be consi dered the i nequal i ty f ormof the mathemati cal model
al thoughit is not exp ssedin estandardf ormat for theL P probl em.
3.1.1 S ndar dFor mat
The standardformat of theLP probl emi ncl udes onl y equal i tyconstrai n15. It is sct up
as a mi ni mi zati on probl em(somc authors prcfer a maxi mi zati on probl em). ln
addi ti on. all of the vari abl es in e model e expected to be semi posi ti ve ( 0) or
nonnegati ve [4]. Fi nal l y thecons int l i mi 15 equanti ty on theri ght-handsi deof
t
}
3.1 PR08LEMDEFI NI TI ON 95
econstrai nt arerequi redto be posi ti ve(>0). In eprobl emdefi nedinEquati ons
(3.1)-(3.4) the changes at must be carri ed out i ncl ude converti ng the obj ecti ve
functi onto opposi tetype and transformi ngtheconstrai n15to equal i ty type.
Thev i abl esXJ and X2 representi ngthenumber of Component Pl acement Machi nes
of model A and B respecti vel y can be expected tobe semi posi ti veP- 0) isei ther
some machi nes wi l l be orderedor no machi nes wi l l be ordered. Not e thedi sete
mr e of theprobl emis bei ngi gnor as 3. 75machi nes of Type A is not somethi ng
that woul d be ordered. The sol uti onwill be rounded to a sui tabl ei nteger. 1tereare
extensi vedi scussi ons indi sc andi nteger programmi ng litera to suggest
is may nol be e opt i mumval ue. The j usti fi cati on for roundi ng off is it is
conveni ent 10do soat thisj uncture.
In the obj ecti ve functi onj(x) coeffi ci en15 990 and 900 are cal l ed the cost
coe. cien15. Each coeffi ci ent is associ atedwi adesi gnvari abl e:presen15
i ncrease(dec ase) incost per uni t change intherel atedvari abl e. lesi mpl est way to
l sform oecti vefuncti onto erequi redformat is tomul ti pl y all of theterms
by -1. Therefore enewobj ecti vefuncti onis
Mi ni mi ze j(x): - 990 XJ - 900 X2 - 5250 (3.5)
InEquati on (3.5) constant termon the right (- 5250) can be abso ed i nto the
l efl -handsi deby defi ni nga newobj ecti vef uncti onL(x) wi thout affecti ng eopt al
va1ues for thedesi gnvari abl es. lt can be observed at i ncreasi ngtheval ues of the
vari abl es will make theobj ecti ve mor e negati ve whi ch is good as we aretryi ngto
makej(x) as l owas possi bl e. If a parti cul ar cost coeffi ci ent had a posi ti vesi gn then
i ncreasi ngtheamount of thecorrespondi ngvari abl ewoul d l eadtoan i ncreaseof the
obj ecti vefuncti on whi ch is nol desi rabl eif efuncti onneeds tobe mi ni mi zed. Thi s
i deais expl oi tedin eSi mpl ex met hod presentedlater.
The qual i ty constrai nts have to be transformedto an ual i ty co ai nt. The
si mpl est wayachi eve thischang i stoi n ducean addi ti onal semi posi ti vevari abl e
for each i nequa1ityconstrai nt. Thal wi l l create E pondi ngequal i tycons nt
ina strai ghtforwardway. Consi der efirst cons nt:
gJ(x): 0.4x
J
+ 0. 6 S 8.5 (3.2)
is canbe transformedto
gJ(x): 0.4x
J
+ 0. 6 + x
3
= 8.5 (3.6)
The newvari abl e X3 is referred to as th sl ack vari abl e. The defi ni ti on is qui l e
appropri atebecause it takes up thesl ack of theori gi nal cons ai nt terms when it is less
8.5. 1tesl ack wi1l be posi ti veas efrrst t wo terms wi l l be less or ual to
8.5(for constrai nt sati sfacti on) f rom eori gi nal defi ni ti oninEquati on (3.2). Inother
words it is thedi fferencebetwecn constrai nt limit (8.5) and first t wo terms of
theconstrai nt . If tbeval ues of XI and X2 cause theconstrai nt tobe at i15l i mi t val ue.
enX3 will be zero. Thi s is thereasonit is deti nedtobe semi posi ti ve. By defi ni ti on.
9 6 UNEAR PROGRAMMI NG
therefore X3is si mi l ar totheoriginaJ vari abl es intheLP probl em. Transformi ngthe
remai ni ngconstrai nts ina similar manner thecompl ete equati ons instandardformat
are
Mi ni mi ze j(x): - 990 x -900 X2- 5250 (3.5)
Subj t to: 8'(X): 0.4x + 0.6X2+ XJ = 8.5 (3.6)
82(X): 3 X - X2 +X4 = 25 (3.7)
83(X): 3 x + 6 X2+ Xs = 70 (3.8)
x 0; ~ ~ 0; x
3
0; x
4
0; X
s
0 (3.9)
Equati ons (3.5)(3.9) exprl theLP probl eminstandardformat. An i nconsi stency
canbeobservedwi resp tto eformat of theprobl ems previ ousl ydefi nedinthe
book. Namel y 8(") was reserveddefi ne i nequaJ itycons ints whi l eh() defi ned
uaJ itym aints. Here 8(X) is aJ sobei ngusedfor equaJ ityns aints. Thi s change
is acconunodatedin is chapter only. Insubsequent chapters theprior meani ng for
thesymbol s is restored. Actual l y symbol s arenot requi red econstrai nt under
disc si on na1sobe rred10by equati onnumber. The formu1ati oninEquati ons
(3.5)-(3.9) can be exp ssed succi nct1y by resorti ng to a matri x notati on. l i s
conversi on is qui te strai ghtforward. Before embarki ng on is transformati on the
representati onof theobj ecti vefuncti onneeds tobechang' dagain. The constant term
is brought totheri ghl -handsideso obj tivefuncti on nbeexpress wi
terms onl yi nvol vi ng edesi gnvari abl es on theri ght. Thi s is shown asL(x) bel ow.
Once agai nfor conveni ence thesame symbol j(x) is usedinpl aceof thisnewfuncti on
in subsequent di scussi ons. These extra symbol s have been thrown in to keep the
deri vati ondistinct. Thi s is a smal l pri cefor theability 10devel op ernathemati caJ
model ina natuml way and construct it toconf ormtothestandardrep sentati onin
most references. Therefore:
Mi ni mi ze L(x): j(x) + 5250 = - 990 x -900 (3.5)
Thfol l owi ngdefinitions arcusedtorewri ttI leprobl emusi ngmal ri ces. Let thecost
coefficients berepr ented by cC2 . c". The coefficients in econstraints are
representedby esym 1ai j (usi ngdoubl e subscripts). lefirst subscri pt identi
constrai nt number whi1e the secondconstrai nt is the same as the vari abl e it is
mul ti pl yi ng. It is aJ sousefu1toidentify rst subscri pt wi ththerownumber and
thesecondsubscri pt wi thecol umn number torrespondwi opemti ons t use
I i near al gebm. For exampl e the aint 8(x) wil1be rep sented
g(x): a" x + a' 2x
2
+ a' 3x
3
= 8.5
3.1 PR08LEMDEANmON 97
where all = 0.4. a12 = 0.6. al3 = 1. Because thereare five desi gn vari abl es in e
probl emdefinition addi ti onal terms wi fol l owi ngcoeffi ci ents a' 4 (=0) and
al 5(=0). arei ncl udedin eexpressi onfor g(x) wi thout changi ngits meani ng:
g(x): a
ll
X + a' 2x
2
+ a' 3x
3
+ a'4x4+ asxs = 8.5
leconstrai nt g2(X) canbe si mi 1arl yexp ssedas
g2(X): G z x + a
u
+a
23
x
3
+ a
24
x
4
+ sXs = 25
The coe cients intheabove equati oncanbe silyestabl i shedby compari sonwi th
Equati on(3.7). Letting eri ght-handsidevaJ ues of theconstrai nts berepresentedby
b"
b
m
e st rdformat inEquati ons (3.5)(3.9) c be slatedto
Mi ni mi ze j(x): cTx (3.10)
(3.11) Subj 0: g(x): A x=b
Si den ai nts:" ~ 0 (3.12)
x represents thecol umn vector of desi gnvari abl es i ncl udi ng esl ackvari abl es l x"
X2... x ] T m ents the ansposi ti onsymbol ). c presents thecol umn vector
of cost coefficients. [c" C2. . CJ T. b represents ecol umn vector of theconstrai nt
I i mits. [b" b
2
bm]T. A represents them x n matri xof constrai nt coefficients. In
is book all vectors col umnvectors unl ess ot I l erwi seidentified.
M
I
N
-
-
X

1
l
J
l
l
l
c
c
.
c

r
-
E
E
-
E
-
E
E
E
a
-
-
-
p
t

-
B
E
l
l
i
-
-
J

1
2
m

U
.
L
U
r
'
E
B
E
E
E
'
E
E
E
E
E
E
a
E
E
E
-
h
u

.
E
E
E
E
E
E
E
E
'
E
a
E
E
J
M
h
a
a
a

1
h
-
W
1
2
m

a
a
a
j

r
E
E
E
E
E
E
E
E
E
E
E
E
E
E
E
E
L

A

Compari ng Equati ons (3.10) and (3.11) wi thEquati ons (3.5) (3.8) evectors and
matri xare
c = [-990 - 900 0 0 O]T
b = [8.5 25 70]T
Azh?i :!
The f oUowi ng addi ti onaJ defi ni ti ons have beenusedinthestandardformat. n is E
number of desi gnvari abl es dmi s enumber of constraints. Intlteexampl e n = 5
andm = 3. Nol e t I lat thesl ackvari abl es arei ncl udedinthecount for n.
98 LI NEAA PAOGRAMMI NG
Negat l ve Val ues of Desi gn Varl abl es: The LP standard mathemati ca1 model
al l ows onl yfor nonnegati vedesi gnvariables. Thi s does not limit eappl i cati onof LP
inany way. Thereis a standardproceduretohandl enegati veval ues for variables. If a
desi gnvari abl e sayx.. is cxpectedtobeunreslrI ctedinsi gn enit is defi nedas e
di fferenc of two nonnegati vevari abl cs
11 ..1
x. =x-; - x
I - ^J ^J
(3.13)
where and areacceptabl eLP variables. If thelatter is hi gher thantheformer xJ
canhave negati vevalues. Equati on(3.13) is usedtorepl acex inthemath cal
model . Note that is procedurc alsoi ncreases thc number of desi gnvari abl i nthe
model .
Type of Constrai nts: The less an or equal to (::::) constrai nt was handl ed
naturallyindevel opi ngtheLP standardmathemati cal model. lestandardnumeri cal
techni quefor obtai ni ngthesol uti onrequi res that esl ackvari abl es be posi vetostart
and b also be posi ti ve throughout . If there is a constrai nt is requi red to be
mai ntai nedabove a speci fi edval ue for exampl e
g(x): a" x + a
J2
xz+ a' 3x
3
b (3.14)
thenit appe snatural toi ntroducea negati vesl ackvari abl e- X4 tochange Equ on
(3.14) toanequal i tyconstrai nt:
g(x): a
ll
X + a
l2
x
2
+ a' 3x
3
- x
4
b
J
Whileit is possi bl etoch ge esi gnon X4 by mul ti pl yi ngwi th- 1 roughout is
causes eri ght-handconstrai nt limit tobe negati ve whi ch shoul dbeavoi ded. Insuch
cases another vari abl e xs is i ncl udedintheexp ssion. Inthis case two addi ti onal
vari abl es arenowpart of themodel :
g(x): a" x + a' 2x2+ a
l3
x
3
- x
4
+ X
s
= b
J
'he vari abl eXs is called artificial vari abl e. Arti ficial vari abl es alsoi ntroduced
for chequal i ty constrai nt is naturally present inthemodel. The useof these
vari abl es wil1beillustratedlater throughanxampl e.
3.1.2 Model l ng I ssues
The LP programis characteri zedby emathemati cal model defi nd inEquati ons
(3.10)(3.12). Due tothelinear natureof a11 of thefuncti ons thedi scussi on
of the sol uti on needonl y consi der coefficients of the functi ons-the variabl
3.1 PAOBLEMOEFI NJ TI ON 99
themsel ves can be i gnored. These coefficients whi ch are present as vectors or a
ma xcanbemani pul atedtoyieldsol uti ons usi ngconcepts fromlinear algebra.
A l ook at thegraphi cal sol uti ontotheLP probl emsuggests that thesol uti ontothe
probl em if it exists must lieon theconstrai nt boundary becausethecurvatureof the
oectivefuncti oniszeroeverywhere (it is linear). Furthermore thesol uti onto LP
probl emis pri mari l yi nfl uencedby Equati on(3.11). Bei ng a I i near equati on. topics
fromlinear al gebra are necessary to underst d some of e i mpl i cati ons of e
mathemati cal model .
If n =m at is thenumber of unknowns is thesame as thenumber of variables.
therecanbeno scopefor opti mi zati onas Equati on(3.11) will determi ne solution.
provi dedone exists'. Inthe fol1owi ng. many of concepts arecxpl ai nedthrough
si mpl egraphi cal illustration or by consi derati onof si mpl eexampl es. They mostl y
deal wi thexampl es of two variables (x" X2). For exampl e consi dr twocons aints
gl: xl +Xz=2
g2: - x
1
+Xz = 1
Fi gure3.1 illustrates thegraphi cal sol uti ontotheprobl em. The solutionis at x = 0.5
andX2 = 1. 5. Thi s is i ndependent of theobj ecti vefuncti on(whatever it may be). ln
matri xform. A andb are
U
r
-
-
E
-
L
-
L
U

-
-
l
E
-
E
A
t
-

-
-
A
-
-
A

leexi stenc of a sol uti ondepends on therows of A (or col umns of A- we will deal
wi throws onl yforconveni ence). Iftherows of A arel i nearl yi ndependent thenthere
is a uni que sol uti onto esystemof equati ons.
ThDef i neanother probl emas fol l ows (onl y changi ngb omtheabove exampl e)
e newprobl emis
g : x + =2
g2: 2x
l
+2x
2
=4
The newfuncti ong2represents twi ce eval ueof gl Thi s means g2canbeobtai ned
m8J' Therefore 82is saidtodepend on 81 Techni cal l y et wo quati ons e
l i near dependent eac dependenceof thefuncti ons is not a concemasmuch
as thefact eyar not linearlyi ndependent . Both 81and82establishthesame line
graphically. What is thesol uti ontotheprobl em? There areinflnitesol uti ons as l ong
as the pair of (Xl X2) val ues satisfies 81In Fi gure 3. 1 any poi nt on the line
representi ng81 is a solution. Not onl y (0.5 1. 5) but (20) (02) arealsosolutions.
From opti mi zati onperspecti ve is is qui tegood for unl i ketheprevi ous caseof
u quesol uti on theobj ecti vefuncti oncanbe usedtofindthebest solution. 1t woul d
bebetter however if thechoi ces were finiterather thaninfinite.
100 LI NEARPROGRAMMI NG
4
n
H
R
M

u

H
M

u
-
e
e
a
u
y
l
o
u
u
R
M
R
H

U

4 2
m

-
i
g
.
f
-
n

e

F
M

3
21
)('" 1
1
-2
- 3
J o
X
1
2 - 2
Fl gure3.1 Uni quesol uti on-Unear i ndependence.
Whi l e severaJ cxampl es can be constructed to illustrate linear dcpendence it
appears reasonabl etoassume thatfora 11 ercise opti mi zati onthenumber of
constrai nts cannot equal thenumber of ulI knowns ( m n). Intheprevi ous illus n
n=2 dm = 1 (therewas onl y one effecti vecons aint). It is useful torecogni zethat
is corresponds tothecasen > m.
If m > n therearemore equati ons thanthenumber of variables. lis i mpl i es
thesystemof equati ons representedby Equati on(3.11) is i nconsi stent set or has a
redundant set of cquati ons. Consi der thefol l owi ngillustrationfor m = 3 and n = 2
whi ch uses thesame 8and g2as inthc first illustrationwhi l eaddi nga new83:
g: X. +X2=2
g2: - x
1
+X2 = 1
g3: X + 2 x2= 1
Fi gurc 3.2illustrates theset of equati ons is i nconsi stent si ncea sol uti ondonot
exi st. If one were toexist thenthethreelines must pass through esolution. Si nce
eyar all s ght lines erecanbe onl y one uni que i ntcrsecti ngpoi nt InFi gurc
3.2onl y two of th threelines inter ect at different points.
3
3.1 PROBLEMDEFIN 10N 101
F
V
v
b
/
/
K
A
A
m
d
/
/
/
/
4
Case n a 2; m a 3: Nosolutlon; Innslsten! 5e!
3
~ 1
1
- 3
4 -2 -1 o
X
1
2 3
Flgur 3.2 No solulion Inconslstent set 01equations.
Rcdefi neg3as
g3:XI +2 =3. 5
Thi s new83is edashedlineinFi gure3.3. Now a uni qu sol uti onto eprobl em
at (0. 5 1. 5) is establ i shed. lis is othesol uti onestabl i shedby consi derati onof 8
d g2 alone. This i mpl i es g3 is redundant. If 81 is mul ti pl i cd by 1. 5 82 is
mul ti pl i edby 0.5and bothadded. thc result is g3defi nedabove. That is
g3= 1. 5 gl + 0.5g2
canbeobtai nedby linearlycombi ni ng gand g2. is. addi ngconstant mul ti pl es
of nctions. Thi s is another exampl e of linear dependence. Thi s l i near
dependence and redundal l CYis alsoiII ustratedby theset
g: X + "2= 2
g2: -x =1
g3: 2x
1
. . 2x
2
= 4 (same as gl )
3.1 PROBL EMDEFI NI TI ON 103
For n ==m and wi thgl: xl + x2==2 and g2: X1+X2 = 1 ematri x A is
Casen a 2; m. = 3: UnlqU8solullon Unoar dependonc9 -
E
E
E
E
E
E
J
1
1

=

A

4
h

h

-
h

-
E-
a and its detenni nant is

det(A) = IAI = (1)*(1)(1)*( 1) =2
- "3. 5
Fr omtheorems inIi near al gebra[5] if det(A) is not zero inwhi ch case thematri x A
termed nonsi ngul ar a uni que sol uti onexi sts totheset of equati ons. This was uc for
eexampl e iII us tedabove as onl y sol uti onw l ocatedat (0.5 1. 5) shown
graphi cal l yinFi gure 3. 1.
Consi der theexampl e wi thgl: XI + X2= 2 dg2: 2x
1
+ 2x
2
=4. In is case
.r 1
det(A) = IAI = (1)*(2) ( 1) 2) ==0

- 3
- 3 - 2 O
X
1
2 3
lf det(A) is zero that is matri x A is si ngul ar reareei ther no sol Ul i ons or infinite
sollllions. For is exampl e ere were infinite sol uti ons. Dctcrmi nants of
hi gher rder squ 'e matri ces are eval uated by setti ng up l ower-order determi nants
until eyar e duced to a 2 x 2 determi nant whi ch is eval ua d as above. Any
textbook on engi neeri ngmathemati cs or l i near a1gebra shoul dillustrate istechni que.
It is not producedhere.
If n "" m di scussi onof theexi stencc of sol uti ons requi s addi ti onal concepts Ii ke
rank of a matri x and augmenl ed matri x. Si nce case n > m is of interest in
opti mi zati on onl y that case is empl oyed in subs uent ilI ustrati on. A useful
di scussi on wi l l need at 1east threevari abl es and t wo equati ons. Three vari abl es wi l l
deny theuse of graphi cs todevel op thefolI owi ng concepts. Usi ng theset
Fl gure3.3 Unl quesolutlon IInear dependence.
This suggests e concept of redundancy can be associ ated wi th l i near
dependence. The di scussi onof l i near dependence and i ndependence was establ i shed
g equati ons above. The same di scussi on can take pl acc by reasoni ng on the
coeffi ci cnts themsel vcs. In this case thecoeffi ci ent ma x A shoul d l ead us to e
same concI usi onrcgardi ng el i near i ndependence of a set of Ii near equati ons. e
concept of a del ermi nanl is n cssarytodevel op thecriteriafor l i near dependence.
gt: x
1
+X
2
+X
3
= 3
g2: - X +0.5x
3
= 1. 5
Determl nant: The determi nant is associ atedwi tha squarc matri x. For a general 2 x
2 matri x A where
thematri ces
E
E
E
E
B
E
E
-
J
.
a
a

-E
a
r
M
l
M
L

=

A

E
E
E
d
3
5

-
M
1
n
u
I
l

-
-
-
- A

E
E
E
E
E
E
J
3
5

--a
F
E
E
-
E
E
a
E
Z

h
u
-
-
E
E
E
.
J
I
I
l

-
-
a
-
-
a
f
l
L
=

A

(3.16)
a a
1= d d t( A)== a
a21 2
The newmatri x A. is cal l ed eaugmented mat ri x- t he col umns of b areadded toA.
Accordi ng 10 rems of l i near al gebra(presentedherewi thout proof):
edetermi nant is expres'i ed and eval uatedas
(3.15)
If theaugmented matri x ( A) and thematri x of coeffi ci ents (A) have thesame
rank r <n en er M many sol uti ons.
. If eaugmented ma x( A) d ematri xof coefficients (A) do not have the
same rank a sol uti ondoes not exist.
If theaugmented matri x(A) andthematri xof coefficients (A) have thesame
rank r = 1 1 where m = n thenthereis a uni qu solution.
Rank 01 a Matri x: Whi l e thereareformal definitions for defi ni ngtherank of e
ma xa useful way todetermi ne ranki sl ook edeten nt . 'herankof a
matri x A is theorder of elargest nonsi n lar square submatri x of A that is e
largest submatri x wi tha determi nant other anzero.
Intheexampl e above thelargest squaresubmatri x is a 2 x 2 matri x(sincem = 2
andm < n). Taki ng the subma ix whi ch i ncl udes the first two col umns of A e
determi nant was previ ous establ i shedto have a val ue of 2 thereforenonsi ngul ar.
Thus therank of A is 2 (r = 2). lesame col umns appear inA maki ngits rank also
2. lereforeinfinitely m y sol uti ons exist (r < n). One way de rmi ne E
sol uti ons is to assi gn(n - r) vari abl es arb val ues and use themtodetermi ne
val ues for theremai ni ngr variables. The val uen - r is alsooftenidntifiedas
deg reeof f reedom(DOF) for thesystemof equati ons. lntheparticular exampl e above
e DOF can be identified wi th a val ue of 1 (i.e. 3 - 2). For i nstance x 3 can be
assi gn da val ueof 1 inwhi ch caseXI = 0.5 dX2 = 1. 5. On th other hancJ. if X3 = 2
thenXI = 0.25and X2 = 0.75. The above cases illustratethat for a DOF of 1 once x 3 is
selected. Xl dX2 canbe obtai nedthroughsome further processi ng.
Another exampl e for iI1ustrating erank is ei nconsi stent systemof equati ons
in oducede lier where n = 2 dm = 3 reproducedfor conveni ence
gl: x
1
+x2=2
g2: - x
1
+ =1
g3: x
1
+ 2x
2
= 1
The matri ces for this systemare
A = H l }b=[ I } A ' =H i l ]
The rank of A cannot be greater 2sincen = 2 and is less th m= 3. Fromprior
cal cul ati ons edetermi nant of thefirst two rows of A is 2. Thi s is gr ter thanzero
and th reforetherank of A is 2. For A. thedetermi nant of the3 x 3 matri xhas tobe
exami ned. The determi nant of a general n x n squarematri xA Canbecal cul atedin
terms of thec'actorsof thematri x. The cofactor i nvol ves ede nni nant of a mi nor
matri x- whi ch isan(n- 1) x (n- 1) matri xob nedby del eti nganappropri atesingle
rowand a singlecol umn f romtheoriginal n X n ma ix A mul ti pl i edby 1 or - 1
dependi ngon eto of therowand col umn values. References 5 and6 c provi de
mor i nformati on. Here determi nant for the3 X 3 matri xisdefi nedfor iIl ustration
3.1 PROBLEMDEFI NI TI ON 105
in ebook. le (+1) d(1) bel owar obtai ned by usi ng( 1); wher i andj
represent erowand col umn val ues of thecoefficient mul ti pl yi ngtheval ue
al1 a al "l l -
A = 10;; lln iln( +l ) a111273+( l ) a1212J 2+( +l ) a13
1 ~ 2 a
33
1 I ~ . ~ I 1-- ~ I ~ : l l
(3.17)
lerefore
2
A
L
-
-
a
-
-
a
.
+

-
z
a
'
E
a
'
E
A

t
A

-
a
a
-
-
A
1
1

'
a
-
-
E
A
-
-
A
2
'
'
M
I
l
l

- A

= (1)(0) + (-1)(-2) + (1)( 3) =- 1
A. is nonsi ngul ar andits rank is 3 whi I e therank of A is 2. The systemof equati ons
will have no sol uti ons fromtheabove th orcmandcanbeseengraphi cal l yinFi gure
3.2. For the sake of compl eteness usi ng the previ ousl y defi ned dependent set of
equati ons:
gl: x
1
+x
2
=2
g2: - x
1
+x
2
= 1
g3: 2x
1
+ 2xa=4
whi ch yields thefol l owi ngma ces:
A =[-l l }b=[ ! } A ' =H 1 !]
The rank of A is2. Note usi ngthefirst andlast rowof A for thecal cul ati onof the
determi nant yields a si ngul ar 2 x 2 subma x. The determi nant of Ais
n
u
-
-
0
0

U

+

=

'
A
4
2
A
(

+

'
A
.
E
+

'
i
a
T
a
.
z
'
Z
''z
-
a
'
A
a
'
1
L
'
A
-
-
a
-
-
E
-
A

whi ch makes A. singular. However therank of Ais at least 2 due to presenceof
l esubma ixwhi ch determi nedtherank of A. Fr omtheabove theorem er k
of A isequal totherartkof A. whi ch is2 and thisisless m= 3. It canbe concl uded
t ereareinfillitesol ons for systemof equati ons.
A useful observati onof matri x A t hep cedi ngexampl e is t woof therows
are a constant mul ti pl e of each other. s makes e ma ces si ngul ar. The same
hol ds for t wo col umns at aremul ti pl es of each other.
Un/ t Vectors: Real -worl d opti mi zati on probl ems frequentl y i nvol ve a l arge
number of desi gnvari abl es (nvari abl es). Inreal LP probl ems. enumber of vari abl es
can be staggeri ng. For a successful sol uti on to e op zati on probl em. it is
i mpl i ci tl y assumed that the set of desi gnvari abl es is l i nearl y i ndependent . That is.
there is no di rect dependence among any t wo vari abl es of the sct. In is case. the
abstract p c e spawned by thedesi gnvari abl es is cal1ed en-di mefl si ofl al Eucl i dean
space. The word abstract isuscd herc tosi gni fythat it isa construct inthei magi nati on
as onl y threevari abl es can be accommodat ed in fami l i ar 3D physi cal space. le
desi gnvector is a poi nt in is space. The Eucl i deanspa is a di rect extrapol ati onof
the physi cal geome cal 3D space t surrounds us. Thi s space is termed
Cartesi anspace. For an i11ustrati on consi der thepoi nt (232) inCartcsi anspace or in
therectangul ar coordi nate system. InFi gure 3. 4 thepoi nt is marked P. The adof
numbers wi thi n parentheses denotes the val ues al ong the x" X2and X3 es
specti vel y.
The Cartesi anspace is identifi by a set of mutual l y pe endi cul lines or axes
markedx"x2' andx3Thepoi nt wherc theyi ntersect is theori gi n(0) of thecoordi nate
sysl em. The poi nt P can al sobe associ atedWitll thevector OP. drawn f romtlleori gi n
o to epoi nt P. Very often a poi nt and a vect or- t wo very di fferent enti ti es-are
used synonymousl y. Fr omel ementary vector addi ti onpri nci pl es thevector OP can
becons ctedby consecuti vel y j oi ni ngthreevectors al ongthethr coordi nate es.
Inthe fi gure this is shown by l ayi ngout a vector al ongthex axi s addi ng a vector
al ong e X2 axi s to it and finally addi ng thc third vector al ong thc X3 axis. The
addi ti onal smal l arrows inthefi gure are eLmi t vectors. l ey have a magni tude of
one and a di recti onal ongeach of the es. Theseuni t vectors arel abel ede" e2 and
e3 inthe fi gure. Mathemati cal l y poi nt P (232) and thevector addi ti oncan be
rcpresented(col umn vectors)
X3
2
3
X2
Fl gure3.4 Rectangul ar coordl natesystem.
3.2 GRAPHI CAL SOLUTl ON 107
d e

+

q
-
e

4
3

+

d
L
'
E
E
-
-
E
E
E
B
E
E
S
E
E
J
n
u
n
u
'
E

F
E
E
-
-
E
E
-
B
E
E
-
L
E
B
B
-
a
'
E
B
E
E
-
-
J
n
u
'
n
u
r
l
l
l
1
1
1
L

4
3

+

1
1
1
1
1
1
1
'
z
n
u
n
u

r
'
E
E
E
a
E
a
E
-
E
L
L
=

E
E
E
E
E
E
E
E
E
E
E
E
J
r
h
H
h
(3.18)
The advantage of uni t vectors is eycan cstabl i sh ypoi nt intheCar si anspace
through an approp te triad of numbers si g fyi ng coordi nate val ues or 10 tion.
Acal l y any poi nt in eCart i an space can be detenni ned by any set of thl
l i nearl yi ndependent vectors. Thesevectors aretermed thebasi s vectors. Inthis case.
the si mpl e connecti on between the poi nt and the el ementary addi ti on illustrated
above will not be avai l abl . Fu ermore oughthemethods of I i near al gebra these
three vcctors can be reduccd to the uni t vcctors through el ementary row/ col umn
opcrati ons al so cal l ed Gauss- J ordan el i mi nati on or reducti on. In ndi mensi onal
Eucl i deanspace spanned by then desi gnvari abl es epoi nt p"(p.. P2 . . . p") will
be representedby a col l ecti onof n val ues encl osedinpar n eses. The correspondi ng
uni t vectors aredfi nedas
o
o
e"=I.
(3.19)
levector f rom ori gi n(0) toP" is P sentedas
OP = pe + P2e2+ . . . + p"e
ll
These newdefi ni ti ons will be suffi ci ent tounderstandthetechni ques used tosol veLP
probl ems.
3.2 GRAPHI CAL SOL unON
le graphi cal di scussi on is agai n l i mi ted 10 t wo desi gn vari abl es in ori gi nal
probl emdescri pti on. It is usedheretodevel op certai ngeometri c i deas rel ated10LP.
I nLP th i ntroducti onof sl ack vari abl es wi l l easi l yexceed this limit of t wo vari abl es
for graphi cal ill ati on. so thegraphi cal sol uti on is illus ted wi th spect to e
ori gi nal vari abl es in the desi gn probl em. MATLAB is once agai n used to aw e
graphi cal sol uti on. The probl emis not reduced 10thestandardformat for thegraphi cal
sol uon. It is i mportant todo 50 for a numeri cal sol uti on. Si nce Ihepri mary graphi cal
el ement is e strai ght li 1Le a functi on for drawi ng 1i nes is devel oped through th
functi on m-fi l e drawLi ne. mbel ow. is functi on is al so used to provi de more
expos toMATLAB programmi ng.
dr awLl ne. m: is a functi on m file for drawi ng str ght lines. le 1i ne 10 be
drawn is representedas
108 Ll NEAA PAOGAAMMI NG
+by=c
Si ncex y areusedsymbol s for vari abl es xl represents thel ower rangeon x whi l e
X2 represents theupper rangc. Li mi ti ngIi nes/constrai nts can bc drawn by settingeither
a or b tozero. Col or (red) is uscdtodrawa p 'allel lineat 10%i ncrease/decreasc in
theori gi nal val ueof c (dependi ngon thedi recti onof theori gi nal i nequal i ty) i nstead
of hashmarks. Li mi t constrai nts areinmagenta and th obj ecti vefuncti onis inbl ue
dashed line. The newMA 11. AB command here is e line command. You are
enco agedtos k hel pon thecommand. The code:
dr awLi ne. m
Dr awi ng l i near const r ai nt s f or LP pr ogr ammi ng pr obl ems
Opt i mi zat i on Usi ng Mat l ab
Chapt er 3 l i near Pr ogr ammi ng
Li nes ar e r epr esent ed as: ax + by c ( c >= 0)

x1 x2 i ndi cat e t he r ange of x f or t he l i ne
t yp i ndi cat es t ype of l i ne bei ng dr awn 1 ( <=)
9 ( >=)
n ( none)
The f unct i on wi l l dr aw l i ne( s) i n t he f i gur e wi ndow

t he gr een sol i d l i ne r epr esent s t he act ual val ue
% of t he const r ai nt
t he r ed dashed l i ne i s 10 l ar ger or smal l er
( i n l i eu of hash mar ks)
t he l i mi t const r ai nt s ar e i dent i f i ed i n magent a col or
t he obj ect i ve f unct i on i s i n bl ue dashed l i nes
f unct i on dr awLi ne( x1x2abct yp)

r ecogni ze t he t ypes and set col or
i f ( t yp ' n' )
st r 1 ' b' ;
st r 2 ' b'
cmul t 1;
el se
st r 1 '" ' g'
st r 2 '" ' r' ;
end
val ues f or dr awi ng hash mar ks
dependi ng on t he di r ect i on of i nequal i t y
i f ( t yp -=' n' )
i f ( t yp == ' l ' )
cmul t +1;
3. 2 GAAPHI CAL SOLUTI ON 109
el se cmul t - 1;
end
end
set up a f act or f or dr awi ng t he hash const r ai nt
i f ( abs( c) >= 10)
cf ac 0. 025;
el sei f ( abs( c) > 5) & ( abs( c) < 10)
cf ac 0. 05;
el se
cf ac 0. 1;
end
i f (c 0)
cdum cmul t *O. l ;
el se
cdm (1 + cmul t * cf ac) *c;
end
i f b 0 t hen det er mi ne end poi nt s of l i ne x l i ne
i f (b -=0)
y1 (c - a*x1) / b;
y1n ( cdum- a* x1) / b;
y2 (c - a* x2) / b;
y2n ( cdum a*x2) / b;
el se
i dent i f y l i mi t const r ai ns by magent a col or
st r 1 ' m'
st r 2 ' m' ;
y1 x1; set y1 same l engt h as i nput x1
y2 x2; set y2 same l engt h as i nput x2
x1 c/ a; adj ust x1 t o act ual val ue
x2 c/ a; adj ust x2 t o act ual val ue
y1n ' " 0; set y 0;
y2n 0; set y 0
end
i f (a 0)
st r 1 ' m' ; set col or f or l i mi t l i ne
st r 2 ' m' ; set col or f or l i mi t l i ne
end;
dr aw axi s wi t h sol i d bl ack col or
hh l i ne( [x1x2] [00] ) ;
110 Ll NEARPROGRAMMI NG
set ( hh' Li neWi dt h' 1' Col or ' ' k' ) ;
hv 1 i ne ( [0 0] [xl x2] ) ;
set ( hv' Li newi dt h' 1' Col or ' ' k' );
st ar t dr awi ng t he l i nes
hl l i ne ( [ xl x2] [ yl y2] ) ;
i f (t p ' n' )
set ( hl ' Li neWi dt h' 2' Li neSt yl e' "' ' Col or ' st r l ) ;
el se
set ( hl ' Li neWi dt h' 1' Li neSt yl e' '' ' Col or ' st r l ) ;
end
i f ( b - = O) &{a - = 0)
t ext ( x2y2num2st r ( c) ) ;
end
i f ( b ==0) I ( a ==0) I ( yp ' n' )
gr i d
r et ur n end
gr i d;
h2 l i ne ( [ xl x2] [ yl n y2n] ) ;
set ( h2' Li neWi dt h' 0. S' Li neSt yl e' ' : ' ' Col or ' st r 2) ;
gr i d
hol d on
3.2.1 Exampl e 3.1
The probl emf rompage 94 is reproduced0 0 morefor cooveoi eoce.
Maxi mi ze j{X): 990 XJ + 900 X2+ 5250 (3.1)
Subj tto: g. (X): 0.4x. + 0. 6x2 S 8.5
(3.2)
g2(X): 3xJ - X2S 25
(3.3)
) : 3x. +6x2S70
(3.4)
x. 0; .i 0
Th graphi ca1sol uti oois shown inFi gure 3.5. The sol uti on rcadf romthc fi gu (you
C Zm earcaof sol uti oo) i sxj = 10 andxi = 7. It is ci ntersecti onof theacti ve
constrai nts (3. 3) d (3.4). leactual val ues arexj = 10.48 dxi = 6.42. Si nce an
i ntegra1 number of machi nes have to be ordered the sol uti on is adj ust eda
nei ghbori ngi nteger va1u at satisfies l hecontrai nts.
8Or '
60
3
- 40
0
3.2 GRAPHI CAL SOLUTI ON 111
5

. .

-- - . .
.
. .
.
.

. .
. . .
h
. . .
-
. J - 5250
5 10 15 20 25 30
Numbor 01machlnas01IypeA
Fl gure3.5 Graphl cal sOlulion Exampl e 3.1.
3. 2. 2 Characterl stl cs of t he Sol utl on
beglmetry evi dnt inthegraphi cal sol uti onof Exampl e 3.1isusedtoexpl ai nsome
of the concepts associ ated wi th LP and ilS numeri ca1 techni que. Fi gure 3.6 is E
graphi caJ rcp senl ati onof constrai nts i nvol vedinExampl e 3.1 (same as Fi gure
3.5wi thout theobj ecti vc ncti on). Inorder to late egcomet r yLPconceplS thc
stan rdforrnat of LP is n S y. The standardforrnat establ i shedbeforeis
Mi ni mi ze j{X): - 99Ox
J
- 900x
2
- 5250 (3.5)
Subj ect to: g. (X): 0.4x. + 0. 6x
2
+ x) = 8.5 (3.6)
g2(X): 3x. - X2+ X4= 25 (3.7)
g3(X): 3xI + 6x
2
+ Xs = 70 (3.8)
-x.
(3.9)
In Fi gure 3.6 the constrai nts x. 0 and X2 0 are addcd to the three functi onal
constrai nt Ii nes (3.6)-(3.8). The ha.'!hedareais thc feasi bl eregi on at is th desi gn
112 Ll NEAR P OGRAMMI NG
50
40
30
! 20
i 10
2
- 30
0 5 10 15
Numbor 01Machl n0801TypoA
20 25
Fl gure3.6 Feasi bl eregion Exampl e 3.1.
space inwhi ch all of theconstrai nts aresatisfied. The circles exaggerate thepoi nts of
i ntersecti onof fi veconstrai nts takent wo at a ti me. Many of themarenumbered Pl
P2 . . . P6. AI I of thesepoi nts of inter tioncan be associ atedwi tha certai nproperty
concemi ng theval ues of thevari abl es. Note therearefi vedcsi gnvari abl es (n= 5) and
threc functi onal constrai nts ( m= 3). For e npl c:
PI : (x
1
= 0 x2= 0 X) = 8. 5 x
4
= 25 X
s
= 70)
P2: (x
1
= 0 = 11. 67 X) = 1. 5 x
4
= 36. 67 X
s
= 0)
P5: (x
J
= 10. 7 x2= 7. 05 X) = 0 x
4
= 0 X
s
= -4. 4)
P6: (x
1
= 0 x
2
= 14. 17 x
3
= 0 x
4
= 39. 17 Xs = 15.42)
The val ues of evari abl es areobtai nedas ei nte ti onofthe constrai nts takent wo
at a ti me. I n eabove Ii st for each poi nt exactl y t wo of thevari abl es arezero. The
number 2 corresponds totheval ue of n - m. n represents thenumber of vari abl es and
m the number of constrai nts. Poi nts P5 and P6 are i nfeasi bl e because one of the
vari abl es has a negati veval ue.
Basl c Sol utl on: A basi c sol uti onis one obtai nedby setti ngcxactl y n - m vari abl es
to zero. I n Fi gure 3.6 aI l of the poi nts i denti fi ed by the ci rcl es rcprcscnt basi c
sol uti ons. The poi nts choscn above are all basi c vari abl es. In general for n desi gn
vari abl es and m cons ints numher of basi c sol ul i ons is gi vcnby thecombi nati on
3.2 GRAPHI CALSOLUTI ON 113
()=
This woul d yi el d10poi nts for our ex npl ewhi ch is shown inFi gure3.6.
Basl c Varl abl es: The set of vari abl es inthebasi c sol uti onthat have nonzero val ues
arecal l edbasi c variables. Correspondi ngl y theset of vari abl es inthebasi c sol uti on
havetheval ue of zeroarecal l ednonbasi c vari abl es. For thepoi nt Pl XJ dX2
arenonbasi c vari abl es whi l e X3X4andxs ar basi c vari abl es.
Basi c Feasl bl e Sol utl on: This is a basi c sol uti onthat is al sofeasible. 'heseare
epoi nts Pl P2 P3 and P4 inFi gurc 3.6. InLP thesol uti ontotheprobl em if it is
uni que must be a basi c feasi bl esol uti on. The basi c sol uti oncan aIso be consi dered
geomecaI lyas a corner poi nt or an extremepoi nt of thefeasi bl eregi on.
Convex Pol yhedron: This is a bounded regi onof thefeasi bl edsi gnspace- e
rcgi ondefi nedby quadri l ateral compri si ng thepoi nts PI P2 P3 and P4 inFi gure
3.6. The termcOl l vex set represents a col l ecti on of poi nts or vectors havi ng the
fol l owi ngproperty: For any l wo poi nts inthesct (or wi thi na rcgi on) if all ofthe poi nts
on thel i ncconnecti ng twopoi nts al soIi e in csame regi on theregi onis a convex
set. I magi n y linedrl1wn inl hc regi ontl finetl by equadri l atendwhose comer
poi nts arePl P2 P3 and P4. Fr omFi gure 3. 6it is c1ear thelinewilI still be wi in
theregi onestabl i shedby quai J ater - maki ng it a convex set.
Opt l mumSol utl on: Thi s is a feasi bl e sol uti on at mi ni mi zes the obj ecti ve
f uncti on- poi nt P4 in Fi gure 3.6. In LP the opt i mumsol uti on must be a basi c
feasi bl esol uti on.
Basl s: 'he basi s P sents ccol umns of thecoeffi ci ent matri x A coespond
to the basi c vari abl es. They f ormthc basi s of the m- di mensi onaI space. They are
termed thebasi s vectors.
/
/
J
6
4
2
2
p
l
z
/
4
1
2
4
h
r
Y
Y
Canonl cal For m: 'he basi s vector duc to uni t vectors throughrow/ col umn
operati ons (or Gauss-J ordan eIi mi nati on). The basi c feature of e numeri cal
techni que for L P (Si mpl ex method) is a peti ti veproced e starti ngf roman initial
basi c feasi bl esol uti on and determi ni ng ebest nei ghbori ngbasi c feasi bl esol uti on
that i mproves theobj ecti ve. The procedure is carri edon until theopt i mumsol oni s
reachdor if it is determi ned nosol uti onis possi bl e. Thc canoni cal f ormis used
for rapi d1y i denti fyi ngthesol uti on. Each i terati onin procedurecan be descri bed
mathemati caI ly as fol l ows.
The starti ngconstrai nts ' c organi zedas
[A]II [ 1= [b]
After theGauss-J ordan el i mi nati on (3.20) is assembl ed as
(3.20)
114 Ll NEAAPAOGAAMMI NG
m[ X]m+[R] n-m)[X](s) =[b]
The set of m desi gnvari abl es inthefirst term thebasi c vari abJcs. The sct of (n-
m) dcsi gnvari abJ es in csccond tcrmare nonbasi c val ucs. In cSi mpJ ex met hod
thc nonbasi c vari abl cs aresummari l y set tozcro.
3.2.3 Di nt Sol onTyp
Thcrea at most four di fferent resuJts at can be expcctedfor thesoJ uti onof theL P
probJ em: (1) a uni que sol uti on (2) i nfi ni teJymany soJ uti ons (3) unbounded sol uti on
and (4) thepossi bi l i ty at thereis no soJ uti on.
(1) Uni que sol uti on: le exampl e used for di scussi on has a Usi que sol uti on
(Fi 3.5). le condi ti on necess y for is to occur is e obj ecti ve
functi on and theconstrai nts have di ssi mi l ar sJ opes and thefeasi bJe regi onis
boundcdl cJ osed. Geometri cal l y thiscan be vi sual l yexpJ ai nedas themovement
of elinereprescnti ngthc obj ecti vefuncti on p ullcJ 10itseJf ina favorabJ e
di recti on until it remai ns j ust incontact wi thone of thefeasi bJecomers (bi c
feasi bJ e soJ uti on) of the feasi bl e regi on. The fe ibJe regi on is i denti fi ed
ui sti nctl y in Fi g 3.6. A si mpl rcisc uf i magi ni ng diffe obj ecti v
functi onJines shoul dconvi nce thevi ewer that several di fferent obj ecti ves n
be defi nedtol ocatetheuni que sol uti onat P 1 P2 P3 or P4. Thi s 20 geome
and constructi oncan be used tounderstandtheextensi onton di mensi ons. He
theI i nes wilJ be representedby lI yperpl anes.
(2) lnfinile sol uti on: In order for is to occur e obj ecti ve functi on must be
paral l el to one of thecons ints. For exampl e in Fi gure 3.6 let theori gi nal
probl embe redefi nedso theobj ecti vefuncti onis paral l el totheconstrai nt
h Anypoi nt on theconstrai nt and l yi ngbetween thelinesegment dcfi nedby
epoi nts p 2 and P4 is an opti mal sol uti ontotheprobl em. They wi l l yi eJ d e
sarneval ue of ve.
(3) Unbounded sollllion: I n is case the feasi bl e regi on is not bounded. In
Fi gure 3.6 if theconstrai nts gl and g2 were not part of theprobJ emformuJ ati on
en the feasi bJ eregi onis not bound on thetop. Referri ngto Fi gurc 3.5. E
obj tivefuncti oncan be shi f tedunl i lTUtedhi gher val ues. Inpracti ce there
wi l l be an upper bound on rangeof thedesi gnvari abJ es (not p tof stand d
format) that wilJ be used tocJ ose is regi on. inwhi ch case sol uti ons of type (1)
or (2) can be recovered. The prcsence of anunbounded sol uti ona1sosuggcsts
t the formul ati on of probl emmay be l acki ng. Addi ti onal meani ngf ul
constrai nt(s) c beaccommodat ed todefi nethesoJuti on.
(4) No sol ul i on: Fi gure 3.6is used to expl ai n is possi biIi ty. Consi der E
di recti onof i nequal i ty in81 is changed to thc opposi tetype(). The feasi bl c
regi onwi rcspect totheconstmi nts gl and 83is totheright of XI = J 5.0. Thi s
3.3 NUMERI CAL SOLUTI ON-THE SI MPLEXMETHOD 115
poi nt is not feasi bl ewi threspect to econstrai nt 82Thereforethereisno poi nt
is feasi bl e. Thereis no sol uti onto eprobl em.
In the above di scussi on. e two-vari abl e si tuati ons provi ded an obvi ous
cJ assi fi cati onof thesol uti ons. Inpracti ce. LP model s arel argewi thover hundreds of
vari abJcs. Model i ng and transcri pti on errors may gi ve rise to many of the above
si tuati ons. The genemti on of sol uti on is based on numeri cal tcchni qucs of l i near
al gebra whi ch is oftensensi ti veto equal i ty of thematri x of coeffi ci ents. Fi Jteri ng
out errant datais usual l y a si gni fi cant exerci seinthesearchof opti mal sol uti ons.
3. 3 NUMERI CAL SOLUTI ON -T HE SI MPLEX METHOD
The standard numeri cal procedure is based on e al gori thmdue to Oantzi g
menti oned earlier. It is refe 10as eSi mpl ex method. The procedure is rel ated
to sol uti on of a systemof Ii near equati ons. The actual appl i cati on of
procedure can be associ atedwi th theGauss- J ordan met hod f roml i near al gebra.
where thecoeffi ci nt rows aretransf ormed through e1ementary mul ti pl i cati onand
addi ti on. Most mai nf rame computcr i nstal l ati ons usual l y carry software that wi l l
hel psol ve L P probl ems. MATLAs al so provi des procedures 10sol veL P probl ems
in its Opti mi zati on TooJ box. In this sccti on the Si mpl x met hod is appl i ed to
si mpl e probl m pri mari l y tuundcrstunu thc pl ' Ogr ummi ng and geomctri c features.
This wi l l be used when we di scuss di recI techni ques for nonl i near probl ems. I nthe
next secti on the Si mpl ex met hod is i ntroduced in detai J wi th expl anati ons. In
subsequent secti ons MATLAB. or spreadsheet programs I i ke Excel can al sobe used
toi mpl ement theSi mpJ ex method.
3.3.1 Feat ur es of t he Sl mpl ex Met hod
Inthis secti on. themachi ne seJ ecti onex npJ ei ntroducedearJier is set up for appl yi ng
theSi mpl ex mc od. It is i nstructi vetonotc themet hod is iterative. Gi ven a starti ng
poi nt it wi l l march f orward throughi mprovi ng desi gns un it has f ound thesoh on
or cannot proceed further. For compl eteness theori gi nal probl emis rewri ttenhere:
Maxi mi zc j(X): 990x
l
+ 90Ox
2
+ 5250 (3.1)
Subj ect to: 81(X): O.4xl + 0. 6X2:s; 8.5 (3.2)
(3.3)
(3.4)
4
5
F
i
r
-
2
T
I
z
-
-
g2(X): 3xI - X2:s; 25

83(X): 3xI + 6x
2
:S; 70
x
1
O ; x 2 ~
The probl emwas aJ sotransformed10thestandardformat as fol l ows.
Exampl e3. 1
Mi ni mi ze J(X): - 99Ox
I
- 90Ox
2
- 5250
Subj ect to: g(X): 0.4xl + 0. 6x
2
+ x3= 8.5
(3.5)
(3.6)
g2(X): 3x - x2 +X4 = 25 (3.7)
g3(X): 3xI + 6x2 + Xs = 70 (3.8)
x
1
0; x
2
O ; x
3
0; x
4
0; X
s
o (3.9)
X3 X4 and Xs ar esl ack vari abl es.
The Si mpl ex met hod is used on the probl embei ng expressed in the standard
f ormat. Thef ol l owi ng i nformati onis useal l i norgani zi ngthecal cul ati onas wel l as
cogni zi ng the moti vati on for subsequent iterations. 1ese refcr to a probl emfor
whi ch a uni que sol uti on exists. While m y of the i tems bcl owwcrc in duced
earl i er they arereferencedhere for compl eteness.
Thc number of vari abl es inthc probl emisn. i ncl udes thesl ack and surpl us
vari abl es.
1enumbcr of constrai nts is m( m< n).
1eprobl emis a1ways tomi ni mi ze theobj ecti vc functi onf
The number of basi c vari abl es is m (same as number of constrai nts).
Thenumber of nonbasi c vari abl cs is n - m.
The set of uni t vectors expected is e" e e
m
The col umn of theri ght-handsi de(b) is posi ti veand greater anor equal to
zero.
T heca1cul ati ons areorgani zedina tabl e whi ch is cal l eda tabl eau or Si mpl cx
tabl eau. It wi l l be referredto thetabl einthis book.
Onl ytheva1ues of thc c i ci ents ar necess y for thecal cul ati ons. The tabl e
crefore contai ns onl y coeffi ci cnt va1ues the matri x [A] in previ ous
di scussi ons. These thecoeffi ci ents intheconstrai nt equati ons.
1eobj ecti vefuncti onis el astrowi n etabl e. 1econs int coeffi ci ents are
wri ttent.
Row operati ons consi st of nddi ng (subtracti ng) a defi ni temul ti pl eof thepi vot
r owtoothEr rows of thetabl e. The pi votrowi denti fi es ther owinwhi ch theuni t
vcctor (ina.ccrtai ncol umn) will havc theval ueof 1.
Row operati ons areneedcd toobtai nthecanoni cal fonn. Practi cal l ythis pl i cs
youshoul d bEabl e tospot thEuni t vectors i nthetabl e. I f this happens
then the current i terati on is compl te and consi derti ons for the next onc
must begi n.
3.3 NUMERI CAL SOLUTl ON -THE SI MPLEXMETHOD 117
Wi t h this inmi nd theSi mpl ex met hod is appl i edtothemachi ne sel ecti onprobl em.
Thc exampl e wi l l use and cJ arifythei tems intheabove Ii st .
3. 3. 2 Appl l cat l on of Sl mpl ex Met hod
Si mpl ex Tabl e 3. 1: InTabl e 3.1 (Si mpl ex Tnbl e 1) c first r owi ndi catcs the
vari abl enames. The last r owis eobj ecti vefuncti on. Spreadsheets arean effi ci ent
way to pro ss the i nfonnati on. it is bei ng cmpl oyed en symbol f can be
removed f rom elast col umn and last r owtoal l ownumeri cal ca1cul ati ons ( shown).
Thel nstcol uD1ni s ri ght nndsi d val ues. The rest of thc cntri es thecoeffi ci ents
of the cons ai nt equati ons.
The current i tcrati onis over if thetabl edi spl ays thecanoni cal fonn. Inpracti ce e
canoni cal f onn compri sspotti ng them uni t vcctors inthetabl e as wel l maki ng
sure theen es under theb col umn except for the rows representi ng eobj ecti ve
functi on(s) are nonnegati ve P-Q). A gl ance at Si mpl ex Tabl e 3.1 i ndi cates t E
canoni cal f onn is present.
Thc uni t vectors in c tabl eal soi dcnti fythosem vari abl cs at wi l l bel ong tothe
b is. Those vari abl es will have a nonzero val ue for is iteration. The remai ni ng
n- m nonbasi c vari abl es aresct tozero. Thi s sol uti onisdi rectl yi ntcrpreted omTabl e
3. 1. The sol uti onthereforeis
x = 0. 0 xz= 0. 0 =8.5 x
4
= 25 X
s
= 70 df =- 5250
Has theOpti mal Sol uti onBeen Ob i ned? The opti mnl sol uti onrequi res that f
be reduced much as possi bl e. For thecurrent sol uti on val ueof XI is O. Fr om
Equati on(3.5) if x were i ncreasedf romzero entheobj ecti vefuncti onf decreases
further inval ue. Si mi l ar reasoni ngcnn be used for X2Thereforethesol uti oncanbe
further inproved. Sol uti onhns not converged. The second/ next i terati onwi l l qui re
nssembl i ng another tabl e Tabl e 3.2.
The onl y way for x and/or X2to have a posi ti ve val ueis if y b amea basi c
vari abl e. Si nce thenumber of bnsi c vari abl es is prescri bedfor theprobl em some of
e current bnsi c vari abl es must become nonbasi c. For each iteration e Si mpl ex
met hod a1l ows onl y one pai r of vari abl cs toconduct is basi s-nonbasi s exchange.
Each i terati ontakes pl acethrougha newtable. To affect theexchang it is necessary
Tabl e 3.1 Si mpl ex Tabl e 3.1
XI X
2 XJ X4 X ~ b
0.4 0.6

8.5
3 -1

25
3 6

70
990 - 9

5280
119
Si mpl ex Tabl e 3. 2 ( Tabl e 3.2): The pi vot r owis obtai nedf romori gi n r ow20f
Tabl e 3. 1. The fol l owi ngis ori gi nal r ow
25
The val ue of the first el ement in the newr owwi1l be 1. The r owis modi fi ed by
di vi di ngthroughby 3. The newsecond r owinTabl e 3.2wi l l be
8.3333
InTabl e 3.2 efirst el ement of thefirst r ow( Tabl e3.1) must be reduced toO. Thi s
isal ways achi evedby addi ng/subtracti ngfromtherowbei ngmodi fi edan appropri ate
mul ti pl eofthe pi vot row. The modi fi cati onto efirst r owis
r ow1 - [0.4" (pi vot r ow2)} toyi el d
o 0. 73331 - 0. 1333 0 5. 1667
Thef our r owwi1l be obtai nedby
r ow4 + [(990" (pi vot r ow2) I
Tabl e 3.2is obtai nedafter thesecal cul ati ons.
The negati veval ue of thecoe: ci ent inTabl e 3.2inr ow4 i ndi cates weneed
at least one mor e i terati onl tabl e. The EBV is X2Byexami ni ng posi ti veratios of
eva1ues in bcol umn tothecoeffi ci ent in X2col umn theL BV is id tifiedas
XsThepi vot r owis r ow3. Tabl e 3.3wi1l be constructedby transformi ngtherows of
Tabl e3. 2.
Si mpl ex Tabl e 3. 3(Tabl e 3.3)
pi vot r ow3 = (row3)
newr ow1 =r ow 1- {0. 73333 * (pi votrow3)}
newr ow2 = r ow2 + {0. 3333* (pi vot r ow3)}
newr ow4 =r ow 4 + {1230( pi vot r ow3)}
Tabl e 3.3is obtai nedbased on theabove operati ons.
Si mpl ex Tabl e 3.3 Tabl e3. 3
b
n
u
-
-
-
n
u
n
u
3. 3 NUMERI CAL SOLUTI ON -T HE SI MPLEX METHOD

3
O 0.3333
0. 3333
The thirdr owwi l l be obtai nedby
r ow3 - {3"(pi vot r ow3)}
0.4524
10.4762
6.4285
21437.14
Xs
-0. 047
0.0476
0.1428
175.71
X4
-0. 0285
0. 2857
- 0.1428
154.28
X )
'
n
u
n
U
U
X2
n
u
n
u
-
-
A
U

XI
Ll NEAR PROGRAMMI NG
todetermi ne whi ch is thevari abl e wi l l enter thebasi s (XI or X2) Enteri ngBasi c
Vari abl e (EBV). Al so to be establ i shedis whi ch of evari abl (X3' X4' or xs) wi l l
l eave basi sLeavi ng Basi c Vari abl e( LB V). Interms of thenumeri cal cal cul ati ons
inthenewtabl e theuni t vector under theL BV col umn wilI be transferredtotheEBV
col umn throughel ementary row/ col umn operati ons.
118
Enteri ng 8asi c Vari abl e: The EBV is chosen by its abilitytoprovi de for thel argest
decrl inth obj ecti vefuncti onin ecurrent i terati onl tabl e. lis is detercni nedby
thel argest negati ve coei ci ent inther owassoci atedwi ththeobj ecti vefuncti on. Fr om
Tabl 3.1this val ue is - 990 and co pondsto evari abl c XI EBVis X!. A tiecan
be brokcn arbi trariJ y.
Leavi ng 8asi c Vari abl e: The L BV is al so chosen by its abi l i ty l prove e
o tive. The L BV is onl y determi ned after the EBV has been chosen. is
i mportant because onl y thecol umn under eEBVi sexaI nedtoestabIi sh LBV.
The mi ni muml' atio of the val ues of e ri ght-hand si de (b col umn val ues) to the
corrcspondi ng coeffi ci ents in the EBV col umn provi ded c coeffi ci cnt val uc is
posi ti vc d ci dcs eLBV. Thi s rni ni mumval uc i dcnti fi es ther owwhi ch is aIso the
pi vot row. Inthis r ow thecol umn whi ch contai ns theuni ty val ueof someuni t
vector (or a current basi c vari abl e) is thc col umn of c I eavi ngbasi c vari abl e. lis
is t heLBV.
InTabl e3. 1 eEBV has al ready been i denti fi edas Xl' The rati os under this col umn
are 21. 25 8. 3333 and 23.3333. The least val ue is 8. 3333. l I s corr ponds to e
Scondr owof Tabl e 3.1. The uni t vector under theX4 col umn is in is r ow(row2).
Thcrefore X4 is e L BV d e second r owis the pi vot r owfor Tabl e 3.2. Thi s
i mpIi es se ondr owwill be uti l i zed to chang current XI col umn (EB V
col umn) iilto a uni t vector al is currentl y under X4Thi s mechani cal procedure is
based on thefol l owi ngreasoni ng. leval ueof EBV must be such that it wi l l not cause
constrai nt vi ol ati onwi ththecurrent val ues of eother basi c feasi bl evari abl cs. The
appl i cati onof theGauss- J ordan process shoul d shift thecol umn under theL BV to e
EBV l eavi ngthecol umn of theother uni t vectors (that arepart of thebasi s) unal tred.
Tabl e 3.2is thenconstructedusi ngthc pi vot r ow2.
Si mpl ex Tabl e 3.2 Tabl e3.2
b
n
u
-
-
n
u
n
5. 1667
8.3333
45
13530
Xs
n
u
n
U
E
n
u
X4
-0. 1333
0.3333
330
.t)
-
n
u
n
u
n
u
Xl
0. 7333
- 0.3333
7
- 1230
XI
121
next
A( : 6) / A( : 1)
ans
o 0 0 - 17/ 1980
o 0 0 - 51198
o 0 0 - 7/ 99
0 0 0 16/ 3
' somet hi ng not r i ght - The above di vi si on shoul d have' ;
' been an el emen by el ement one' i
3.3 NUMERI CAL SOLUTI ON-THE SI MPLEXMETHOD
he cr eat e t o r ow pi vo t he i dent i f i es ' The r ow
t abl e' ;
A( : 6) . / A( : l )
ans =
21. 2500
8. 3333
23. 3333
- 5. 3333
' Second r ow i s t he pi vot r ow and x4
' Const r uct i ng Tabl e 2' ;
' not e t he scal i ng f acor f or t he mat r i x' i
A
A
1. 0e+003 *
0. 0004 0. 0006
0. 0030 - 0. 0010
0. 0030 0. 0060
- 0. 9900 - 0. 9000
' The el ement at A( 21)
A( 2: ) = A( 2 : ) / A( 21)
A
1. 0e+003
0. 0004
0. 0010
0. 0030
- 0. 9900
A( : 6) . / A(: l)
ans
85/ 4
25/ 3
70/ 3
- 16/ 3
f or mat shor t
i s LBV' ;
0. 0085
0. 0250
0. 0700
5. 2800
o
o
0. 0010
o
0. 0010 0
o 0. 0010
o 0
o 0
must be 1' ;
0. 0085
0. 00 3
0. 0700
5. 2800
0. 0010
0
o
0. 0003
o
o
0. 0010
0
0
0
*
0. 0006
- 0. 0003
0. 0060
- 0. 9000
1 l
l
-
4
u
r
H
1
U
4
y
Ll NEARPROGRAMMI NG
Tabl e 3.3 does not have y negati ve val ue e obj ecti ve functi on row. A
sol uti onhas been achi eved. The basi c vari abl es have theunit vector intheir col umn.
The nonbasi c vari abl es areset to zero. Fr omTabl e 3.3 the op numval ues canb
observed :
x ; = 10.4762 xi = 6.4285. X; = 0.4524I = -21437. 14
Note theval ueofj " +21437. 14=Of romwhi ch th sol uti onfortheobj ecti vefuncti on
(f.) canbe obtai ned.
120
Sol utl on Usl ng MATLAB
The tables intheprevi ous computati oncanbe generatedqui teeffortlessly usi ng a
spre sheet . lis woul d be accompl i shed by setting up .e mathemati cal
mani pul ati oninone cell and copyi ngand p gto eother cells. Si ncethis book
al so sistsinMATLAB progr nmi ngtheillus ati onherewill al l ow opportuni tyto
practi ce the use of MATLAB for matri x mani pul ati ons. The code presented here is
pturedusi ngMATLAB' S di ary command. The command requi res thename of a
filewhi ch records all communi cati on wi l hMATLAB as wel l i ts responseduri ng
i nteracti ve sessi on (see hel p di ary). The text file is commented liberally (for your
understandi ng) and usi ngthesemi col onat eend of thestr. gs prevents ei recho.
h is filethecomments arequo ds ng. The code represents detailsof anactual
sessi on. These statements are typed a er e MATLAB prompt the Command
wi ndow. Thisisthereforeani nteracti vesessi on. Val ues to eright of ans = istheecho
of result of e ex uti on of e MATLAB command. Note: comments need not be
typed.
3.3.3
Exampl e 3_l. txt
f or mat compact
f or mat r at i onal
' Set up t he coef f i ci ent mat r i x and b as mat r i x A' ;
- 1 0 1 0 25; 3 6 0 0 1 70; - 990 1 0 0 8. 5; 3
5280]
0. 6
o 0
A=[ O. 4
- 900 0
A
17/ 2
25
70
5280
canoni cal f or m' ;
o 0
1 0
o 1
o 0
not e t he
col umn' ;
2/ 5 3/ 5 1
3 1 0
3 6 0
990 - 900 0
' Thi s i s Tabl e 1 -
' EBV i s x1 - f i r st
' To f i nd L BV di vi de l ast col umn by coef f . i n EBV' i
' col umn' ;
' Take t he mi ni mumof t he posi t i ve val ues' ;
122 LI NEARPROGRAMMI NG
' The el ement at A( l l ) must be a 0 ' ;
A( l : ) = A( l : ) 0. 4*A( 2: )
A
1. 0e+003
0. 0007 0. 0010 - 0. 0001 o 0. 0052

0. 0010 0. 0003 o 0. 0003
0. 0083
0. 0030 0. 0060
o 0. 0010 0. 0700
0. 9900 - 0. 9000 o o
5. 2800
' El ement at A( 31) must be a zer o' ;
A( 3: ) = A( 3: ) - A( 31) *A( 2: )
A
1. Oe+003 *
0. 0007 0. 0010 0. 0001
0. 0052
0. 0010 - 0. 0003
0. 0003 o 0. 0083
o 0. 0070
- 0. 0010 0. 0010 0. 0450

- 0. 9900 - 0. 9000 o o o 5. 2800
' El ement at A( 41) must be 0' ;
A( 4: ) A( 4: ) - A( 41) *A( 2: )
A
1. 0e+004 *
0. 0001 0. 0001 0. 0000 o 0. 0005

0. 0001 - 0. 0000
0. 0000 O 0. 0008
o 0. 0007 o - 0. 0001 0. 0001 0. 0045
0. 1230
0. 0330 o 1. 3530
' Tabl e 2 compl et e - and canoni cal f or mi s pr esent ' ;
' Sol ut i on i s not conver ged because of A( 42) ' ;
EBV i s x2' ;
' Cal cul at i on of LBV' ;
A( : 6) . / A(: 2)
ans
7. 0455
25. 0000
6. 4286
- 11. 0000
' Pi vot r ow i s t hi r d r ow and LBV i s x5'
ans
Pi vot r ow i s t hi r d r ow and LBV i s x5
' Oops f or got t he semi col on' ;
' Cal cu1at i on of LBV' ;
' Const r uct i on of Tabl e 3. 3 - no echo of cal cul at i ons' ;
' A( 32) must have val ue 1' ;
A( 3: ) = A( 3: ) / A( 32) ;
' A( 12) must have a val ue of 0' ;
A( l : ) A( l : ) - A( 12) *A( 3: ) ;
3.3 NUMERI CAL SOLUTl ON -THE SI MPLEXMETHOD 123
' A( 22) must have a val ue of 0' ;
A( 2: ) = A( 2: ) - A( 22) *A( 3: ) ;
' A( 42) must have a va1ue of 0' ;
A( 4: ) A( 4: ) - A( 42) *A( 3: ) ;
A
A '"
1. 0e+004 *
o 0
0. 0001 0
o 0. 0001
o 0
f or mat r at i onal
A
n
u
n
U
F
O
n
u
n
U
2
d
n
u
n
u
n
u
n
H
E

n
u
n
u
n
U
-
-
-
-
n
u
n
u
n
u
4
n
u
n
u
n
u
p
n
u
n
u
n
u
4
n
u
n
u
n
u
n
u

n
u
n
u
n
u
n
u

n
u
n
u
n
U
n
u
n
u
n
u
E
J

n
u
n
u
n
U
n
u
n
u
n
u
n
u

-
-
-
-
n
u
n
u
n
u
n
u

u
n
u
n
u
n
u
n
u
n
u
n
u
A
R
d
r
f

o
'
'
'
'
'
'
a
u
-
a
n
u
-
-
4
n
u
n
u
n
u
n
u
n
U
4
n
u
n
u
4
n
u
n
u
- 11/ 105 19/ 42
1/ 21 220/ 21
1/ 7 45/ 7
1230/ 7 150060/ 7
' No f ur t her i t er at i ons necessar y' ;
di ar y of f
The last f ewcommands sugg sl Ihc possi bi l i ty of devel opi nga for l oopfor therow
mani pul ati ons. It is suggestedas a probl emat theend of echapter.
3. 3. 4 Sol utl on Usi ng MATLA8' S Opt i mi za onTool box
Thi s secti on is useful for thosc that have access 10the Opti mi zati onTool box . f rom
MA'1. AB(Ver 5.2).
Start MATI.. AB. IntheCommand wi ndowtypehel plp. Thi s will provi de i nformati on
on theuseof thel i near programmi ng routi ne10sol vetheprobl em. Here. thestandard
useis empl oyed. It i nvol ves speci fyi ngva1ues for thecosl coeffi ci ents (vector). the
coeffi ci entl constrai nl matri x ( A- ma x). and the ri ght-hand si de vector (bvector).
be fol l owi ngsequence of steps illus ates euse of theprogram.
Usi ng theLi near Programmi ng Routi ne
>> f 990; 900] ;
>>A [ 0. 4 0. 6 3 - 1; 3 6)
>>b [ 8. 5 25 70) '
>> x '" 1p( f Ab)
The sol uti onas posl edintheCommand wi ndow:
x
10. 4762
6. 4286
>> sol ' " f ' *x
8 0 1
1. 6157e+004
To this sol uti on must be added constant - 5280 whi ch was omi tted inprobl em
defi ni ti onfor MATLAB. There ar many di fferent ways touse l i near programmi ng
functi oninMATLAB.
3. 4 ADDI TI ONAL EXAMPL ES
h is secti on addi ti onal exampl es e p sented. These exampl es illustrate e
extensi on/modi fi cati on of the Si mpl ex met hod to handl e greater or equal
constrai nts negati veval ues for thedesi gnvari abl es equal i l y constrai nts and soon.
In alI e cases the probl emis transformed appropri atel y and e same Si mpl ex
mel hod is thenappl i ed.
3. 4. 1 Exampl e 3. 2-Transport at i on Pr obl em
The Fresh MiI k cperati ve supp1 i es mi Ik in gal l onj ugs f romits t wo warehouses
l ocatedinBuffal o ( New York) and WilI i amsport (Pennsyl vani a). 1t h acapaci ty of
2000 gal l ons per day at Buffal o and 1600 gaI l ons per day at Wi11i amsport. It del i vers
800 gal l ons/ dayRochesr ( NewYork). Syracuse ( NewYork) r ui res 1440
gal l onsl day and eremai nder (1360 ga1l ons) aretruckedtoNew Yor k Ci ty. lecost
to shi p l he mi l k to each of l he desti nati ons is di fferent and is gi ven inTabl e 3.4.
Estab1 i sh eshi ppi ngs tegyfor mi ni mumst.
Pr obl emFormul at/ on: Let x be e number of gaI l ons shi ppedf romBuffal o to
Rochester. Let y be enumber of gal l ons shi pped mBuffal o toSyracuse.
The warehousi ng cons int at Buf faI o is 2o gal l ons/day. Therefore
x + y $ 2000
Amount shi pped o mWi l l i amsport toR hes =8oo- x
A mount shi pped o mWi l l i amsport toSyracuse = 1440 - y
Amount shi ppedf romWi l l i amsport toNew Yor k Ci ty is
1600 - (800 - x) (1 O- y) o
Tabl e 3.4 Shi ppi ngCost ( Cen perGal l on)
Buffa10
WiIl i nsport
Rochester
4.2
4.7
Syracuse
4.5
4.6
(3.21)
New York Ci ty
6.0
5.1
x y ~ 6 4 O (3.22)
The si deconstrai nts on x and y areth respecti vewarehouse I i mi ts. The shi ppi ngcosl
IS
Cost =4. 2 x+ 4.5* y + 6.0* (2000 - x - y) +
4. 7( 8oo- x) +4. ( 1 0 - y) + 5.1 (x+ y - 640)
Cost = 1. 4' " x- y + 19120
Assemb1i ng theprobl em(Exampl e 3.2)
M b ze f(xy): 1. 4x- y+19120
Subj ect 81( X y): x + y :52000
82(X y): x + y 640
0$x:58; 0:5y:51440
(3.23)
(3.24)
(3.25)
(3.26)
(3.27)
Fi gure 3.7iII ustrates thegraphi cal sol uti ontoExampl e 3.2. The linerepresenti ngthe
constant obj ecti vefuncti onval ues l ermdoes not i ncl udetheconstant term(19120).
The sol uti onis at thei ntersecl i onof constr int gl and theupper 1 i mi t on eval ueof
xas eobj ecti veis todecreasef as much as possi bl e. Fr omthefi gurethesol uti onis
x=800; y=1200f = 168ooor$168. oo (3.28)
Two- Phase S/ mpl ex Met hod: The mai n di fference bel ween Exampl es 3.1 and
3.2is e cons ai nt intheIatter. Si nce thestandardL P probl emonl y requi res the
desi gnvari abl es be semi posi ti ve 0) eri ght-handconstrai nts on x and y have to
be accommodat ed through addi ti onaI i nequaI ity constrai nts. Appl yi ng the regul ar
Si mpl ex met hod wi thout addi ti onaI processi ngwoul d cr tea probl eminrecogni zi ng
e c oni 1 form. Exampl e 3.2 expressed in e standard f ormat of l i near
pr ogr ammmg IS
Mi ni mi ze ft. xy): 1. 4x- y + 19120
Subj ect to: gl (X y): x + y + SI = 20
g2(X y): x + y - S2 = 640
g3(X y): x + S) = 800
84(x y): Y + S4 = 1440
(3.24)
(3.29)
(3.30)
(3.31)
(3.32)
25
g
o
=
H
h
h
2
2
500
E
D
-
-
-
h

-
-
h

-5
-
-1000
800
-1500
0 200 300 400 500 600
x:g 10 110mB f ol oRh 101
100 700
Fl gure3.7 Graphl cal 80lullon Exampl a 3.2.
x 0; y 0; sl' s2' s3' s4 0 (3.33)
Heresl S3' s4arethesl ackvari abl es and S2is surpl . (si mi l ar toa sl ack- used for
constrai nts) vari abl e. In Exampl e 3. 1 when the f1rst Si mpl ex tabl e was set up
(Si mpl ex Tabl e 3.1) the sl ack vari abl es were hel pful in i denti fyi ng the canoni cal
form. Here inExampl e 3.2 it do not wor kout t way because thecoeffl ci ent of S2
is -1. Mul ti pl yi ngEquati on(3.30) by - 1 mi grates thenegati vesi gntotheri ght-hand
side. whi ch is al so disaJ l owed as f : as e recogni ti on of e canoni cal f ormis
concerned. Thi s means addi ti onal probl em-speci fi c preprocessi ngmust takepl aceto
i denti fy theinitial canoni cal form. Si nce theSi mpl ex me odis used tohandl e l arge
probl ems wi thseveral vari abl es it is mor e conveni ent toappl y theSi mpl ex procedu
ina consi stent way. Thi s is accompl i shed by appl yi ngthesame Si mpl ex techni que in
a two-part sequence.
le f1rst p is recogtzedas Phase 1. Here a newki nd of vari abl e. an artificial
vari abl e is defi ned for each surpl us vari abl e in the same equati on. Al so a new
obj ecti vefuncti oncal l edan artificial cost functioll or artificial obj ecti vefuncti on
is i ntroduced. The artiflcial obj ecti veis defi ned asumof aJ l theartificial vari abl es
intheprobl em. InPhase 1 eartiflcial obj ecti ve is ducedtozero usi ngthestandard
Si mpl cx procedure. When this is accompl i shed Phase 1 is compl ete. I f eartifici aJ
i40
obj ecti vedepends onl y on theartifici aJ vari abl es. dif its vaJ ue is zero is i mpl i es
that artifici aJ vari abl es are lI onbasi c vari abl es. This al so suggests that these
vari abl es werc basi c vari abl es at thestart of theprocedurc. When Phase 1 iscompl eted
enboththeartifici aJ obj cti vefuncti onand theartiflcial vari abl es aredi scarded m
tabl eand Phase 1/ begi ns.
Phase II is the standard Si mpl ex techni que appl i ed to the tabl e f romthe end of
Phase 1 negl ecti ngall artificial el ements. The tabl e shoul d be incanoni cal form.
Addi ti onal tabl es areobtai nedas necessary until thesol uti onis reached.
I nExampl e 3.2set up earl i er therewi l l be one artifici aJ vari abl ea 1 and an artifici aJ
cost functi onA" Intheexampl e onl y Equati on (3.30) wi l l be affectedas
g2(Xy): x + y- s2 + a
l
=640 (3. 34)
The artifici aJ cost functi onwi l l be
Af =a
l
(3.35)
Tabl e 3.5 rep sents f1fSt tabl e in Phase I. The f1rst four rows rep sent E
constrai nts. The f1fthr ow p sents eori gi nal obj efuncti on. The last r owis e
artifici aJ obj ecti ve functi on. This is r owuSed to iv i terati ons in Phase 1.
Tabl e 3.5 is not in canoni caJ form. but repl aci ng the si xth r owby the resul t of
subtracti ng esecond r owf romthesix r owwi l l provi de a canoni cal form. Tabl e
3.6illustrates theresul t of such a r owmani pul ati on.
Sl mpl exMe odPhBse 1: InTabl e 3.6therearet wo choi ces (x y) avai l abl efor
eEBV(ente Igbasi c vari abl e) as both of t hemhave a coeffi ci ent of - 1 in elast
row. Whi l e the choi ce can be arbi trary x is a good choi ce because it has a l arger
negati ve coeffl ci ent in the ori gi naJ obj ecti ve functi on. le L BV (l eavi ng basi c
vari abl e) is i denti fi edthrough themi ni mumposi ti vevaJ ue of theratioof theval ues in
the b col umn to e val ues under xcol umn. l esc rati os are2000/ 164011 and
80011. The sel ecti on i denti fi es esecond r owas pi vot r owand a 1 eLBV.
Usi ng the second r ow e pi vot r ow the uni t vector [0 1 0 0 0 O]T ht o be
Tabl e 3.5 Exampl e 3.2: Initial Tabl e Phase 1
x y SI S2 01 S3 S4 b
o o o
2000

640
l

o 800

1440
1. 4
o

f - 19120
o

A
f
Tabl e3.6 Exampl e 3.2: Sl mpl ex Tabl e1 Phnse (Canonl I Form)
x $1 $2 b al $) $4 y
O
-1. 4
1
l
u
l
l
-
-
-
s
h
u
n
u
n
u
h
u
n
u

o f - 19120
o A
f
- 640
n
u
h
u
n
u
'
2000
640
800
1440
A
U
'
a
n
u
u
n
u
n
u
h
u
n
u
-
a
n
u
n
u
h
u

lsfe edf romtheal col umn tothex col umn. Constructi onof Tabl e 3.7st ts wi th
thesecondrowinTabl e 3.6 whi ch is al sothesecondrowof Tabl e 3.7becauseofthe
coe icient of 1 inthesecondrowunder thex col umn (noscal i ngis necessary). This
is the pi vot row. To obtai n a 0 in the first rowin thex col umn the pi vot rowis
subtrctedf rom rst rowinTabl e 3.6. The newfirst rowis
-1 0 0 1360
To obtai na 0 in thex col umn inthethirdrow. thepi vot rowis subtractedfromthe
thirdrowof Tabl e 3.6. The fourthrowhas a 0 inpl aceand hencc iscopi ed' OmTabl e
3.6. Thc fifthrowisobtai ncdby addi nga 1. 4 mul ti pl eof thepi vot rowtothefi t1 row
inTabl e 3.6. The last rowis theaddi ti onof thepi vot rowtothelast rowf romTabl e
3.6. Tabl e 3.7is compi l edas fol l ows:
Tabl e3.7 Exampl e 3.2: Sl mpl e Tabl e2Ph 1(Canonl col Form)
x $2 b $) $1 al $4 y
n
u
'
A
U
n
u
n
u
n
u
u
n
u
'
a
n
u
n
u
n
u
o f - 18224
o A
f
n
u
n
u
h
u
'
a

1360
640
160
1440
a
a
0
1
1
0
0
'
a
n
u
n
u
n
u
n
u
n
u

O
-1. 4
'
a
'
E
'
a
n
u
-
a

-
-
Fr omTabl e 3.7. eval ueof Af is O. and al is not a basi c vari abl eThi s identifies
theend of Phase 1. The a 1 col umn and last rowaredi scardedand Phase 11is started
wi thTabl e 3.8.
b
V
L
F
i
j
i
E

-
E

Tabl e3.8 Exumpl e 3.2: mpl exTabl e1 Phase s (Canoni I Form)
x
$4
b y $2 $) $
u
'
a
U
n
u
n
U
n
u
n
u
n
u
'
a

1360
0
160
1440
0
0. 4
-
E
A
U
A
u
n
u
n
u

n
u
n
'
a
n
u
n
u

o
-1. 4 o f - 18224
Si mpl ex Met hod Phase 11: The col umn wi th the gr test negati ve coeffi ci ent
(1. 4) iss2 lis is eEBV. The LBV is s3 le irdrowis thereforethepi vot row.
The unit vector [00 1 0 O]Tmust be ansferredf romtheS3 col umn to S2 col umn
throughrowmani pul ati ons usi ngthepi vot row. sexerci seresults inTabl e 3.9.
The negati vecoeffi ci ent under they col umn suggests we arenot fmi shedyet. For
thenext iteration EBV isy. L BV is SI ' The first rowis epi vot row. The unit vector
[1 0 0 0 0 ]Tmust beconstructedunder they col umn usi ngrowmani pul ati onusi ng
thepi vot row. This results inTabl e 3.10.
Thereareno negati vecoeffi ci ents inthelast row. The sol uti onhas beenrellched.
Fromi nspecti onof Tabl e 3.10 thesol uti onis
x = 800 Y = 1200. S2 = 1360. S4 = 240f =16800 (3.36)
In this ex npl ethe same Si mpl ex method was repeatedl y appl i ed al though the
probl emwas sol vedusi ng ophases. The first phase was a preprocessi ngph e
move the surpl us vari abl es away f rom initial set of basi c variables. lis was
achi eved by i ntroduci ng addi ti onnl vari abl es for each su lus vari abl e and an
addi ti onal cost functi onthat drove eiterations of thefrrtphase.
To summari ze for and = constrai nts artificial vari abl es and artificial cost
functi on e in du d into the probl emfor conveni ence. le Si mpl ex method is
appl i edintwo phases. lefirst phaseis termi natedwhen theartificial vari abl es clln
beel i mi natedf rom probl em.
Tabl e3.9 Exampl e 3.2: Sl mpl ex Tabl e2 Phas n( Canonl cal Form)
x $) $4 b y $
$2
A
U
A
U
n
u
0
1. 4
o 12
o 8
o 160
1440
o f - 180
u
n
u
'
a
n
u
n
u
'
a
n
u
s
a
E
E
a
-
-
'
E
n
u
n
u
n
u
n
u

3 A
3
130 UNEAR PROGRAMMI NG
Tabl e 3.10 Exampl e 3.2: Sl mpl ex Tabl e 3 (Fioal) Phasc D
x y SI S2 S) S4 b

12

1
800

o
1360

240

0.4
f - 16800
3. 4. 2 Exampl e 3 - Equal l t yConst i nts and Unrestri cted Varl abl es
Exampl e 3.3 wil1 inc1ude an equal i ty constrai nt and iJl ustrates the techni que for
handl i ngvari abl es areunres i ctedinsi gn(canhave negati veval ues). Vari abl es
like profi t temperature and net i ncome can be negati ve qui te ofl 1. The ual i ty
cons int is handl ed by in duci ng ificial vari abl eand appl yi ng et wo- phase
Si mpl ex t l i qu illus ted in Exampl e 3.2. When this correspondi cial
vari abl eis zero( ndof Phase 1) constrai nt has been met. The negati veval uefor
a vari abl eis si mul atedthrough ediffl rencebetween t wo posi ti vevari abl es. Once
agai n thesechang spreservetheSi mpl ex met hod inits ori gi nal form. The fol l owi ng
is an i magi nary probl emi l l ustrateaddi ti onal features sosome L i berty is takeninthe
formul ati on.
The Probl em: Today a full-tirnestudent on campus is al ways dri ventomaxi mi ze
grades. A strongi nfl uencefor a favorabl eout come is eamount of i nves t made
inhi tti ng books versus eti me spent pl ayi ngpi nbal l . Ina typi cal day at least one
hour is defi ni tel y extended to the pursui t of l earni ng or pressi ng the si des of the
coi n-operated machi ne. Not more than fi ve hours is avai l abl for such di spersi on.
Over eye sa fati guethreshol dof four units. basedon a combi nati on of t wo
activities has been establ i shed as a nor mfor acceptabl e perf orm ce. Thi s
combi nati on is thesumof thehours spent at thepi nbal l machi ne and twi c thehours
spent on homewor k an acknowl edgment that hi tti ng ebooks is stressful. A negati ve
hour of pi nbal l pl ayi ng wi1l go to i ncrease the ti me spent studyi ng whi ch will
contri bute posi ti vel y to the grades. The overal l grade is determi ned as a l i near
combi nati on of twi ce the ti me spent academi cal l y and subtracti ng e ti me spent
pl ayi ng. The probl em erefore is to di stri bute the ti me to maxi mi ze the grades
obtai ned.
Pr obl emFormul ati on: There aret wo ori gi nal desi gnvari abl es intheprobl em. XI
is thenumber of hours spent studyi ng and X2 is theti me spent enj oyi ngthegame of
pi nbal l when th sameti me can be advantageousl y spent mai ntai ni nghi ghgrades. 'he
obj tivefuncti oncan be expressedas
3.4 ADOI TI ONAl EXAMPLES 131
Maxi mi ze j{x
l
X
2
): 2x
1
The constrai nts can be recogni zedf romthestatement of eprobl emdi rectl yas
g.:
g2: 2x. + x
2
= 4
g3: x
1
1
x
1
0 x
2
is unrestri ctedinsi gn
Fi gure 3.8 usi ngthedr awLi ne. mm-fi l e isa graphi cal descri pti onof theprobl emand
thesol uti on. 1esol uti onis atxl = 3 and X2 = - 2 in fi gurewhere conveni entl y e
obj ecti vefuncti onis al sopassi ngthrough.
St andar d Format : As thestandardf ormat expe onl ynonnegati ve vari abl es. the
vari abl ex2 is repl acedby a pai r of vari abl es intheformul ati on:
6
1
0
0
o
a
Z ; !
4 5 6
Fl gure3.8 Graphl cal solution Exampl e 3.3.
x
2
=x
21
- X
22
(3.37)
The standardfoml at after converti ngto a mi ni nuzati onprobl em i ntroduci ngsl ack
surpl us dartificial vari abl es and i ncl udi ngEquati on(3.37) results in
3. 4 ADDI TI ONAL EXAMPLES 133
Phase " Tabl e 2: Tabl e 3.13 is 1he secondtable under Phase 1 . The vaJ ueof the
artificial functi onis2 soPhase 1 is not yet over. The EBV is X22TheLBVi sal The
secondrowis 1he pi vot row. Thi s rowwiII be usedfor rowmani pul ati ons yi el d the
next table.
Mi n rize f - 2x1 +x21 - X22
(3.38)
Tabl e3.13 Exampl e 3.3: Tabl c 2 Phasc I
Sujectto: gl : X1 +X21- X22 +81 =5

X22 $) U) 8
2 a2 b X) X2)
gl: 2x
1
+X2 1 - X
+a t =4

4
g3: X1 + X21- X22 - 82+ a2 = 1

2 -2 2

l
The artifici aJ functi ontobedri ventozerois
3 - 3

- 2 2 / +2
(3.39)

-2 3 Ar- 2
f = a l ~
Si mpl ex Tabl e 0: Tabl e3.11 p1hes dardfOm18t ina table. It is caII edTabl e
o toaII ow for preprocessi ngwi 1hrespectthe artifici a1variables sothat a canoni ca1fOl I D
nbeob rved. Addi ng esecond d i rdrow dsubtrac g eresul t o m elast
rowand repl aci ngthelast rowwi 1hthis computati onwil1yi el dthenext table.
Phase " Tabl e 3: Tabl e 3.14is thenewtableafter therowoperati ons. The val ueof
artifici aJ funconis 0 as bo1hal and a2 enonbasi c vari abl es wi tha vaJ ueof 0
for is iteration. l I ssig fies eend of Phase 1. Phase n will start wi ththelast row
( artifici a1 ncti on) and thetwo col umns that represent thearti cial vari abl es removed
f romthecurrent table.
Tabl c 3.11 Exampl c 3.3: Tabl e0 Phasc I
T bl c3.14 Exmpl c3.3: Tabl e3 PhaseI10111)
8) a) S2 a2 b
8
2 oz
b
X) X2) X22
X) X2) X22 $) 1
1

5
l o 4

4
2 I

2 2 2 o
-1 1
3
1
o
- 2 I

f
3 4 4 / +8
Ar
l

1 Ar
Phase 1 Tabl e 1: Tabl e 3.12provi des estart of eSi mpl ex method. InPhase 1
th moti vati onis todri vetheartificial functi onto0 whi ch has a val ueof 5. The EBV
iSXl TheL BV is a2' The thirdrowis thepi vot row. Row mani pul ati ons wi ththepi vot
rowl eadtoTabl e 3.13.
Tllblc3.12 Exampl c 3.3: Tabl e1 Phscl
X X2' X22
$ a) $2 a2 b
1
4
o
- 1

f
2

Ar- 5
'
A
q

1
2
1
2
3

-
-
Phase 11 Tabl e 1: Tabl e 3.15represents etabl tostart thei temti ons for Phase 11.
However thereareno negati vecoeffi ci ents in last row(rowcorrespondi ngtothe
obj ecti vefunction). Scanni ngthe lefu her1he canoni cal f ormcanbe observed.
lereforethesol uti onhas beenobtai ned. lesol uti onis
Tabl c 3.15 Exampl c 3.3: Tabl e1 Phasc 11
x X z
$
82
b

4

2 2
o o
3

4 / +8
134 LJNEARPROGRAMMI NG
x
l
= 3; x
21
=0; x
22
= 2; sl =4; s2 =0; / =- 8
The val ueof X2 is
X
2
= ( X
21
- x
22
) = ( 0- 2) =- 2
whi chwas identifiedearlier graphi cal l y.
leabove exampl e illustratedtheSi mpl ex method for handl i ngnegati vevari abl es
andequal i tyconstraints. Not appl i cati onof theSi mpl ex techni queitself di dnot
change. An interesting feature of eprobJ emwas that Phase II was i mmedi atly
obtai nedaftel' Phase 1 wi thout any further iteration whi ch suggests thesearchfor
the opti mumis taki ngpl ace in Phas 1even though e focus is on the artificial
variables.
3.4.3 Exampl e 3. 4 -A Four-Varl abl e Pr obl em
Exampl e 3.4presents a probl emwi thfour variables. The procedureisidentical tothe
one inExampl e 3.1 except at ereis no graphi cal sol uti ontotheprobl em.
The Pr obl em: The RI T student-run mi croel ectroni c fabrication facility is taki ng
orders for four i ndi genousl y devel oped ASI C chi ps at can be us in (1) touch
sensors (2) LCD (3) pressure sensors and (4) controllers. There are several
constrai nts on theproducti onbasedon space equi pment availability student hours
and the fact at epri mary mi ssi on of e facility is student training. First e
handl i ng ti me cons int outsi de of processi ng for all chi ps is 600 hours. Touch
sensors requi re4 ho sL CD 9 hours press 'esensors 7 hours and con ollers 10
hours. Second theti meavai l abl eon thel i thographi c machi nes is about 420 hours.
Touch sensors and LCD r uire 1 hour pressuresensors 3 hours and controllers 8
hours. Packagi ng consi derati ons pl ace the maxi mumat 800 vol ume units. Touch
sensors requi re30 voJ urneunits LCD 40 vol ume units press ' c sensors 20 units and
con oller 10 units because of eir compact size. All e cons aints above are
indi tedper week of operati onof thefacility. enet revenue is$6 $10 $9 d$20
for thetouchsensor LCD p ssuresensor and control l er respectively. lefacility
is interestedinmaxi mi zi ng revenueper week andwoul d i ketodetermi netheright
mi xof four devi ces.
Pr obl emFormul at i on: The formul ati on is strai ghtforward based on the
statements above. Let x ( rep sent thenumber of touchsensor chi ps per w kX2 E
number of LCD x3 thenumber of pressuresensors and X4 enumber of con ollers.
obj ecti vefuncti onis
Maxi mi ze: f. 6x. + l Ox
2
+ 9x
3
+ 20x
4
The handl i ngti meconstrai nt canbeexpressed
gl: 4x
I
+ 9x
2
+ 7x
3
+ IOx
4
S 600
The avai l abi1i tyof thel i thographi c machi ne canbe devel opedas
g2: x
l
+ x
2
+ 3x
3
+ 8x
4
s; 420
Jepackagi ngconstrai nt is
g3: 30x
I
+ 40. +20x
3
+ IOx
4
S 800
Al l desi gnvari abl es ar p tedtobeg ater zero. As formul atedabove e
probl emsuggests a degreeof i ncompl eteness. nerearefour desi gnvari abl es and onl y
threeconstraints. lenumber of vari abl es inthebasis canonl y be e. Hence at
least one of e vari abl es must have a val ue of zero. Sevral useful addi ti onal
constraints canstiUbe i ncl udedtodefi nea validopti mi zati onprobl emwi tha nonzero
solution. lis is nowa model i ng issue. For probl ems wi tha l i mi tednumber of desi gn
vari abl es payi ng attention 10 the probl emdevel opment al l ows anti ci pati on of the
sol uti on as wel l the opportuni ty to oubl eshoot deci si ons f rompractical
consi derati ons. It is left to estudent toexpl o this probl emfurther.
St andar d For mat : The obj ecti v functi on requi res a mi ni mumformul ati on.
Henc
Mi ni mi ze f. - 6x
1
1 O ~ - 9x
3
- 20x
4
The symbol / has been retai ned for conveni ence even though the dII cti on of
opti mi zati onhas changed. lecons ints areset upusi ngslackvariables SI S2ands
3
g.: 4x
I
+ 9x
2
+ 7x
3
+ 10x
4
+ S. = 600
g2: x. + + 3x
3
+ 8x
4
+ s2 = 420
g3: 30x. + 40x
2
+ 20x
3
+ l Ox
4
+ S3 = 800
Al l vari abl es are~ O. leformul ati onabove suggests thestandardSi mpl ex method.
S 'P1ex Tabl e 1: Tabl e3. 16i s ei ni tablefor Exampl e 3.4. The canoni caI form
is observabl efromthetableand theinitial basi c feasiblesol uti oncanbe determi ned.
Tabl e3.16 Exampl c 3.4: Si mpl ex Tabl e1
XI X2 XJ X
4 81 J"2 SJ
b
4 9 7 10

6
3 8

420
30 40 20 10

800
-6 -10 -9 -20

f
Tabl e 3.17 Exampl e 3.4: Si mpl ex Tabl e2
x X2 X3 X4 s 82 8J b
2.75 7.75 3.25
l -1. 25
75
0.125 0.125 0.375
0.125
52.5
28.75 38.75 16.25

1. 25 275
-3.5 -7.5 -1. 5

2.5
f + 1050
SI S2. and $3arethebasi c variables. The EBV is X4 and eL BV is $2whi ch is E
mi n l umof {60. 52. 5 80}. The pi vot r owis the second rowused for the row
mani pul ati ons tol eadtoTabl e 3.17.
Si mpl ex Tabl e 2: Usi ng thepi vot r owi denti fi edin elast table eunil vector [0
o 1 O]T under the $2 col umn needs to be transferred to e X4 col umn through
el ementary r owoperati ons. Tabl e 3.17shows ecanoni cal f ormafter compl eti onof
all theoperati ons. The basi c vari abl es areX4. $" dS
3TheEBV is X2 and eL BV
is S3 lepi vot r owis the irdrow. The obj ecti ve of the r owmani pul ati ons is to
transfer theunit vector [00 1 O]Tf romthe$3col umn to exzcol umn.
Si mpl ex Tabl e 3: Tabl e 3. 18denotes thereducedtablewi thecanoni caJ f ormafter
therequi redr owoperati ons arecompl eted. The basi s vari abl eX2. x4and SI There
m nonegati vecoeffi ci ents inthelast r owsuggesti ngthesol uti onhas been obtai ned.
Fr om etable sol uti onis
X
J
= 0 x
2
= 7. 096 x3 = 0 x
4
= 51. 61/ =1103. 22
The sol uti onabove is not sati sfactorybecause theactual deci si onwiIl i nvol vei nteger
val ues for thedesi gnvari abl es. At this ti me i nteger programmi ng is not an opti on. The
adj ustedsol uti onis
X
1
= Ox
2
= 7x
3
= OX4 = 52/ =1110
Whi l e is choi cesatisfies econstrai nts g3. g2however needs a Ii ttleelasticity(limit
is 423) tobe satisfied. Not at sJ is a basi c vari abl esoconstrai nt gJ shoul dnot be a
probl em.
Tabl e 3.18 Exampl e 3.4: Si mpl cx Tnbl e 3
x X2 XJ X4
$
82
$) b
-3

-0.2 20
0.032258 0 0.322581 o 0.1290322 -0. 00323 51. 6129
0.741935 1 0.419355

- 0.032258 0. 025806 7. 096774
2. 064516 0 1. 645161

2. 25806451 0. 193548 f + 1103. 22
Usi ng MATLAB
Thi s is a standardLP probl em. Thi s secti onuses MATLAB inani nteracti vesessi onto
soI ve e probl em. le fol l owi ng is a di aryfile for the MATLAB sessi on for is
ex npl e. Note the wami ng f romMATLAB. as weI l e subsequent use of the LP
programusi ng bounds on e desi gn vari abl es: le bol d italicized statements are
commands usedtoobtai nthesol uti on. The MATLAB prompt is edi tedout .
hel plp
LP Li near pr ogr ammi ng.
X=LP( f Ab) sol ves t he l i near pr ogr ammi ng pr obl em:
mi n f ' x subj ect t o: Ax <= b
X=LP( f AbVL BVUB) def i nes a set of l ower and upper
bounds on t he desi gn var i abl es X s o t hat t he sol ut i on
i s al ways i n t he r ange VLB <= X <= VUB.
X=LP( f AbVLBVUBXO) set s t he i ni t i al st ar t i ng poi nt
t o XO.
X=LP( f AbVLBVUBXON) i ndi cat es t hat t he f i r st N
const r ai nt s def i ned by A and b ar e equal i t y const r ai nt s.
X=LP( f AbVLBVUBXONDI SPLAY) cont r ol s t he l evel of
war ni ng messages di spl ayed. War ni ng messages can be
t ur ned of f wi t h DI SPLAY - 1.
[xLAMBDAJ = LP( f Ab) r et ur ns t he se of Lagr angi an
mul t i pl i er s
LAMBDA at t he sol ut i on.
[XLAMBDAHOW] LP( f Ab) al so r et ur ns a st r i ng how
t hat i ndi cat es er r or condi t i ons at t he f i nal i t er at i on.
LP pr oduces war ni ng messages when t he sol ut i on i s ei t her
unbounded or i nf easi bl e.
c = [- 6 - 10 - 9 - 20] ;
A = [4 9 7 10; 1 1 3 8; 30 40 20 10] ;
b = [ 600 420 800] ;
x = [ c A b]
x = l p( c A b)
War ni ng: The sol ut i on i s unbounded and at i nf i ni t y;
t he const r ai nt s ar e not r est r i ci ve enough.
x
1. 0e+015 *
- 0. 0000
3. 7067
8. 8391
2. 8513
vl b = [ 0 0
vub = [ 100
x = l p( c A
x
0. 0000
7. 0968
o
51. 6129
0] ;
100 100] ;
b vl b vub)
Thi s is thc sol uti onobtai nedby worki ng throughtheSi mpl ex method.
3. 5 ADDI TI ONAL TOPI CS I NLl NEAR PROGRAMMI NG
This secti on l ooks bri efl y at addi ti onaJ i deas associ ated wi theL P probl em. First
ereis a di scussi onof thedual probl emassoci atedwi than L P probl em. Dual i ty is
i mpo anti n di scussi onof sensi ti vi tyanal ysi s- var ioninopti mi zati onsol uti on
due to vari ati on in tfte origil 1parameters of tlte probl em. Fol l owi ng is is a
di scussi onof several pi eces of i nform onthat can be ex ctedf rom efinaJ tabl e
of theSi mpl ex method.
3.5.1 Prl mal and Dual Pr obl em
Associ ated wi every L P probl em whi ch wi l l be referred t oa pri mal probl em
there is a correspondi ng di 1probl em. I n m y ways e duaJ pl ' Obl emis a
sposi ti onof thepri maJ probJ em. l I sis meant toi mpJ y that if thepri mal probJ em
is a mi ni mi zati onprobJ em eduaJ probl emwi l l be a maxi mi zati onone. lf thepri maJ
probl emhas 11 vari abl es the dual probl emwi l l have n constrai nts. lf the pri mal
probl emhas m constrai nts theduaJ probl emwi l l have m vari abl es. For thedi scussi on
of dual i ty e standardpri mCl l probl emis general l y defi ned as a maxi mi zati on
probl emwi th in ualities (though e standard probl emin the L P programmi ng
di scussi on was a mi ni mi zati on probl em[s E quati ons (3.10)-(3.12)] d e
ns Ints were ualities). The s rdpri maJ probl eminthis secti onis defi nedas [3]
Maxi mi ze z: CI XI + CI X2 + . . . + cx (3.40)
Subj ect 10: al l xl + al2x2 + . . . + al .xn b (3.41)
a21xI + + . . . + az"xn b2
a" IXI + anaX2 + +an"xn
Oj =12. . . . . n (3.42)
: ;
3.5 ADDI TI ONAL TOPI CSINLl NEAR PROGRAMMI NG 139

Thi s is al soreferred10 anormal maxi mumprobl em.
ledual of theabove probl emis defi nedas
Mi ni mi ze w: bl YI + blY2+ . . . + b"J ' rn (3.43)
Subj ect 10: al l YI + a21Y2+. . . + arnl Ym cJ (3.44)
al 2Y1 + a22Y2 + . . . + am2Ym c2
al J' 1 ... a2"Y2+ . . . + atuJ'" C"
i =1 2. . . m (3.45)
Rel ati ons (3.43) (3.45) descri bea normal mi ni mumprobl em. There is al soan i nverse
rel ati onshi p between defi ni ti on of the t wo probl ems above as far as
i denti fi cati on of the pri mal and dual probl em. If latter is consi dered a pri mal
probl em enthef ormer is ecorrespondi ng dual probl em.
le fol l owi ng observati ons c be made wi spect 10 the pai r of probl ems
establ i shedabove:
1. The number of dual vari abl es is same enumber of pri mal constrai nts.
2. The number of dual constrai nts is thc same as thenumber of pri mal vari abl es.
3. The coeffi ci ent matri x A of the pri mal probl emis transposed to provi de E
coeffi ci ent matri x of thedual probl em.
4. Thei n ualities ' Creversedindi recti on.
5. The maxi mi zati on probJ emof the pri maJ probl embecomes a mi ni mi zati on
probJ eminth dual probl em.
6. The cost coeffi ci ents of thc pri mal probl embecome theri ght-handsi devaJ ues
of theduaJ pl ' Obl em. The ri ght-handsi devaJ ues of thepri mal become thecost
coeffi ci ents of thedual probl em.
7. The pri mal and duaJ vari abl es both sa fythenonnegati vi ty condi ti on.
Exampl e 3. 5 le fol l owi ng exampl e adapted and modi f i ed f romRef erence 3 is
used toiII ustratethepri mal /dual versi ons of the same probl em. The probl emis sol ved
usi ngtechni ques devel oped inthis chapter.
e Probl em: The l ocal col l ege' s School of A meri can Craf tsman has deci ded 10
parti ci pateseri ousl yina 10caJ chari tyevent by manuf acturi ng speci al commemorat i ve
desks tabl es and chai rs. Each 1ype of fumi ture requi s l umber as weIl as t wo types
of ski l l edl abor: f mi shi ng dc' Pentry. Tabl e 3. 19document s theresources needed.
The 10caJ l umber cooperati vehas donated 48 board feet of l umbr. Facul ty and staff
have vol unt d 20 fi ni shi nghours and 8 carpentry hours. The desk wi l l be sol dfor
140 l l NEARPROGAAMM1NG
Tnbl c 3.19 Exampl e 3.5: Rources
Resources Desk Tabl Chai r
Lumber(bo dfeet} 8 6
Fi ni shi nghours (hours) 4 2 1. 5
Carpentry(hours) 2 1. 5 0.5
$60 etabl efor $30 and chai r for $20. The school does not expect to seJ1mor e
vetables. The school woul d liketomaxi mi z revnuecoJ1ected.
8tandard For ma Pri mal Probl em: Def i ni ng Xl as the number of desks
produced X2 as the number of tabl es manuf actured and X3 as the number of chai rs
manuf actured thepri mal probl emcan be expressedinstandardf ormat
Maxi mi ze z: 6 0 x + 30X2 + 20X3
Su ect to: g : 8x + 6x
2
+ X3 :::; 48
g2: 4 x + 2x
2
+ 1. 5X3 :::; 20
g3: 2x + 1. 5x2 + 0. 5X3 ::; 8
1!4: X2: : ; 5
Xl>X2 x 3
(3. 46)
(3.47a)
(3.47b)
(3. 47c)
(3.47d)
The sol uti ontothepri maJ probl emis avai1abl einTabl es 3. 20-3. 22. Tabl e 3. 20 is th
initi aJ table. The EBV is x and theL BV is $3Thepi vot r owis the i rdrow. The
subsequent r owoperati ons determi ne Tabl e 3.21. I nthis tabl e eEBVi sx3' t heLBV
isszzmd thepi vot r owis theSEcond Eow. Tabl e 322i s theanal tabl eestabl i shi ng E
opti mal val ues for the vari abl es. xt e opti maJ number of desks equal s 2. ~ e
Tabl e 3.20 Exnmpl e 3.5: Pri mal Probl cm Si mpl ex Tabl e 1
x X2 X) $ $2 S) $4 b
B 6

o 48
4 2 1. 5

o 20
2 1. 5 0.5

8

5
- 60 - 30 - 20

f +O
l l i l 1

d d d f
S
l 1
E
3.5 ADD1T10NAl TOP1CSINUNEAA PAOGAAMM1NG 141
Tabl c 3.21 Exampl c 3.5: Prbnal Probl cm Si mpl cx Tabl c 2
X X2 X3
$ $2 $3 $4 b

1
4 o 16
0.5
- 2
4
0.75 0.25

0.5
4

5
o 15 - 5

30
f +240
opti mal number of chai rs tobe made is 8. No tabl es wi l l be made. letotaJ revenue
i s$280. 00.
81 . ndardFormat Dual Probl em: Whi letheduaJ probl emcan b xpressed d
setup m h ica1l yit is cul t toassoci ate edesi gnvari abl es ina di rect m ner
aswasdone thepri maJ probl em. Usi ng y" Y 2 Y3 and Y4 as thedesi gnvari abl es the
probl emcan be formul ated as in Equati ons (3.48) and (3.49) bel owusi ng the
defi ni ti onat ebegi nni ng of this secti on. Some associ ati ons arepossi bl einlight of
transposi ti onof ecost coeffi ci ents and theconstrai nts. For exampl e thevari abl ey
can be associ ated wi th the l umber constrai nt because in the dual probl emits cost
coeffi ci ent is the l umber constrai nt of epri mal probl em. Ina si mi l ar manner Y2 is
associ atedwi ththefi ni shi nghours Y3 wi th carpentryhours and Y4 wi l hthespeci aJ
tabl econstrai nt . obj ecti vefuncti onof thedual probl emis expressed inEquati on
(3.48). Invi ewof e f ormof the expressi onof the obj ecti vefuncti on w E is a
possi bi l i ty that e desi V i abl in the duaJ probl emcan have econorni c
i mpl i cati onas fol1ows:
y uni t pri cefor a board foot of l umber
Y2 uni t pri cefor a fi ni shi nghour
Y3 uni t pri cefor an hour of carp ntry
y4 i scannotbe euni t pri ceas edesk has a pri cedef med i nthepri maJ
pr obl em- maybe a speci aJ pri cefor thedesk resource
Tabl e 3.22 Exampl e 3.5: Pri mal Probl em Si mpl ex Tabl e 3
X X2 X3
$ $2 .1' 3 $4 b
- 2
2 - 8 o 24
o - 2
2 4
8
1. 25

-0.5 1. 5
2
o o o

1 5
5 o o 10 10 o f +280
The dual canbeexpressedas
Mi ni mi zew: 48YI + 20Y2+ 8Y3+ 5Y4 (3.48)
Subj ect to: h
l
: 8YI + 4Y2+ 2Y3+ OY4 60 (3.49a)
h
2
: 6YI + 2Y2+ 1.5Y3+ lY4 30 (3.49b)
h
3
: lYI + 1. 5Y2+ 0.5Y3+ OY4 20 (3.49c)
YI>Y2 Y3 0 (3.50)
l ntroduci ngsurpl us vari abl es (SI>S2 S3) and artificial variabl s(al 2' a3) edual
probl emis expressedas a standardLP probl em:
Mi ni rni zew: 48YI + 20Y2+ 8Y3+ 5Y4 (3.48)
Subj ectto: h
l
: 8Yl + 4Y2+2Y3- SI + al = 60 (3.51a)
h2: 6)'1+ 2Y2+ 1. 5Y3+ lY4- S2+ a2= 30 (3.5Ib)
h
3
: lYl + 1. 5Y2+ 0.5Y3+ OY4- S3+ a3= 20 (3.51c)
[n Phase 1 of thetwo-phase approach eartificial vari abl es areremoved from E
basis usi nganar cial cost functi onA defi nedas
Mi ni mi zeA: a
l
+ a
2
+ a
3
(3.52)
The Phase [ computati ons aredi spl ayedinTabl es 3.23-3.26. Tabl e 3.23is identified
as th si mpl exTabl e 1 and it inc1udes some preproccssi ngby whi ch a" a2 and a3 e
made ebasis variables. As a remi nder the last row presents earti ficial st
functi on. Fromi nspecti onof Tabl e 3.23 YI is theEBV. The LBV is a2and pi vot
rowisthesecondrow. Fol l owi ngthestandardrowoperati ons Tabl e 3.24is obtai ned.
[nthis table theEBV isY2( elargest negati vecoeffi ci ent).l eLBV i sa3( E ird
Tabl e3.23 Example3.5: Dual Probl em Phose1 Tabl e1
Y1 Y2 Y3 Y4 51 52 S)
l
Q2 a) b
8 4 2

60
6 2 1.5

30
1. 5 0.5

20
48 20 8 5
o

w
-15 7.5 4

o A - 110
3.5 AOOI TI ONAL TOPI CSINLl NEARPROGRAMMI NG 143
Tabl e3.24 Exampl e 3.5: Dual Probl em Pbase1 Tabl e2
Y1 Y2 Y) Y4 51 52 S) a1 a2 a) b
1. 3333 0 -1. 3331 1. 3333
-1. 3333
20
0.3333 0.25 0.1666 o - 0.1666 o
0.1666
5
o 1.1666 0.25 -0.1667
0.1666 -1 o - 0.1666 15
4 4 -3
o -8 o w- 2
o -2.5 - 0.25 1. 5 1. 5
2.5
A- 35
rowcontai ns therni ni mumof ([20/ 1. 333] [510.3333] [15/ 1. 6667]}). lepi vot rowis
row3. Tabl e3.25rep sentsth next table. l eEBVi ss
2
Noteinthisc therearetwo
candi dates for EBV (S2S3)' S2is chosenarbitrarily. leLBV is al' The pi vot rowis E
trow. After the[ uiredrowmani pul ati ons Tabl e3.26isgenerated. Scanni ngthelast
row th eareonl ypositivecoefficients. Phase1 isnowcompl ete. The artificial variables
and tificial cost functi oncanberemoved fromtheLP probl em.
El i rni nati ng eartificial vari abl es and theartificial cost functi onf romtheprobl em
Phase II is startedwi thTabl e 3.27. Not that Tabl e 3.27contai ns no i nformati onthat
is not al ready avai l abl e in Tabl e 3.26. It onl y provi des an unc1uttered table for
conti nui ng th appl i cati on of the Si mpl ex l echni que and therefore is not really
necessary. InTabl e 3.27 theEBV isS3TheLBV isYl after a toss between Yl and sl'
The pi vot rowis thesecondrow. Thi s rowis usedtoobtai nTabl e 3.28. Inthis table
note that the val ue of the basi c vari abl e SI is O. Thi s is cal l ed a degenerate basi c
vari abl e. The EBV for thenext tableisY3TheLBVi ss
3
Thepi vot rowis es ond
row(note: the pi vot rowcan be erowwi th the degenerate basi c sol uti on if the
col umn of theEBV has a posi ti vecoeffi ci ent in at row). Tabl e 3.29is efinal table
and contai ns theopti mal solution. The final sol uti onof thedual probl emis
Y; =0=l O y; = 10 =0s; =5 w=280
To summari ze thesol uti onof thepri mal probl emis
x; = 2 xi = 0 x; = 8 s;p= 24 p=5z*=280
f hesubscri pt p" is added toassoci atethesl ackvari abl ewi th epri mal probl em.
The sol uti onof thedual probl emis
Y; =0=10 Y ; = 10 Y4= 0 si = 5 W=280
Some For mal Observat i ons
These observati ons arejustifiedmorc formal l y inmany books on linear programmi ng
some of whi ch have been inc1uded inthereferences at theend of this chapter. Here
theformal results areel aboral edusi ngthesol uti on 10theexampl e di scussedinthis
-
Tabl e 3. 25 Exampl e 3.5: Dual Probl em Phase 1 Tabl e 3

y
Y2 Y3 Y4
$
$2 S3 a a
2 a3 b
o - 0. 2857 -1. 1429

1.1429 1.1429
-1. 1429 -l . 429
2. 8571
0. 1786 0. 2143
- 0. 2143 02857
0. 2143
- 0. 2857
0. 7143
0. 2143 - 0. 1429
0. 1429 -0. 8571
- 0. 1429
0. 8571
12. 857
o - 4.8571 2. 4286
7.4286
3.4286
7. 4286 - 3. 4286
w - 291. 43

0. 2857
1.1429
-1. 1429 -1. 1429
2. 1429 2.1429
A - 2. 857
Tabl e 3.26 Exampl e 3.5: Dual Probl em Phase 1 Tabl e 4
y
Y2 Y3 Y4
$
$2 S3
a
a3 b
o -0. 25 - 0.875 0. 875 2. 4999
0. 125 -6E-O - 0. 1875 6E- 06 0.5 0. 1875 - 6E - 0.5 1. 25
0.25 4E- 05 0. 125 - 4E- 05 - 0. 125 4E- 05 12.5

-3. 0001 4. 9998 6. 4998 0. 0002 - 3. 9998 - 6. 4998 -0. 02 3. 9998 w- 3I O

o

A

Tab1e 3.27 Exomp1e 3.5: Duo1 Prob1em Phasc D Tab1c 1
YI Y2 Y3 Y4 SI S2 S3 b

-0. 25 -0. 875 2.5
0.125
0.1875
0.5 1. 25
0.25
0.125
12.5
O
-3 5 6.5
4
w- 31O
secti on. No proofs 'egi ven. Al of theobservati ons may not be rel evant toExampl e
3.5di scussedabove but areincI uded herefor compl eteness.
1. The pri mal and dual probl ems have the sam val ue for theopti mal obj ecti ve
functi on 7." = w" = 280. This is true however onl y i ftheprobl ems have opti mal
sol uti ons.
2. (f x is any feasi bl esol uti ontothepri mal probl cm. and y is y feasi bl esol ul i on
tothc dual probl cm. w( y) ~ 7. (x). Thi s featureprovi des an esti matc of the
bounds of thedua1 opti mal val ue if a fcasi b1e pri mal sol uti on is known. Thi s
rcsul t al sohol ds for thereversecasewhen a feasi bl dual is known.
3. I f e pri ma1 probl emis unbounded. the dual probl emis i nfeasi bl e. The
unbounded probl emi mpl i es theobj ecti vefuncti onva1ue can be pushed to
infinite limits. lis happens if the feasi bl e domai n is not c1osed. Of coursc
practi cal consi derati ons wi l l l i mi t thc obj ecti vefuncti onval uetocorrcspondi ng
Ii mi ts on the desi gn vari abl es. The i nverse rel ati onshi phol ds too. If the dua1
probl emis unbounded thepri mal is i nfeasi bl e.
4. If the ith pri mal constrai nt is an equa1ity constrai nt the ith dual vari abl e is
unresl ri ctedinsi gn. The rcversehol ds too. If thepri mal vari abl eis unrcstri cted
insi gn. enthc dual constrai nt is an equal i ty.
5. Obtai ni ng pri mal sol uti onf romdual sol uti on:
(i) If theithdual constrai nt is a strict i nequal i ty then ei pri mal vari abl eis
nonbasi c (for opt i mumsol uti ons onl y). Fr omthedual sol uti on thesecond
Tobl e 3.28 Exampl e 3.5: Dual Probl em Phose 11 Tnbl e 2
YI Y2 Y.l Y4 SI S2 5J b
- 2
-0.5 -0.5

2
0.25
0.375
2.5
2 0.5
-0. 25

15
8
- 2 5 5

I V- 300
:11
~
3.5 ADDI TI ONAl TOPI CS INLl NEAR PROGRAMMI NG 147
Tab1e 3.29 Examp1e 3.5: Dua1Prob1em Phose 11 Tob1e 3
2
8
2
24
YI Y2 Y.l Y4 .fl S2 SJ b

-1. 25 2 5

-1. 5
4 10
o 0.5 o - 2 10

5 2
8 .f - 280
constrai nt (3. 51b) is sati sfi edas an i nequal i ty. Theref o thesecond pri mal
vari abl ex 2 is nonbasi c d is is evi dent inTabl e 3.22.
(ii) If the ith dua1 vari abl e is basi c then the ith pri mal constrai nt is a strict
equal i ty. Fr omTabl e 3. 29 Y2 and Y3 are basi c vari abl es therefore (3.47b)
d(3.47c) must be equal i ti es. Thi s is i ndecdtruc.
Thesc rel ati ons can al sobc cstabl i shcdby val ucs of sl ack vari abl es.
6. Rccoveri ng pri mal sol uti on f romfinal dual table: When lhe pri mal and dual
probl erns are in the standard f orm the val ueof thc ith pri mnl vari abl eequal s
thereduced coeffi ci ent (that is thecoeffi ci ent inthelast r owof thefinal tabl c)
of the sl ackl surpl us vari abl e associ ated wi th the ith dual constrai nt. InTabl e
3. 29 the val ues of the reduced cost coeffi ci ent correspondi ng to the surpl us
vari abl es SI S2' and S3 are 2 0 and 8 respecti vel y. Th e are preci sey the
opti mal val ues of thepri mal vari abl es.
7. lf e ithdual vari abl e is nonbasi c the val ue of its reduced cost coeffi ci ent is
the val ue of sl ackl surpl us vari abl eof thecorrespondi ngpri ma1constrai nt .
InTabl e 3. 29 YI and Y4arenonbasi c vari abl es wi reducedcost coeffi ci ents of
24 and 5 respti vel y. These arc theval ues of SI (Sl p) and S4 (S4p) inTabl e 3.22.
8. Obtai ni ng duaI sol uti on f rompri mal sol uti on: When the pri mal and dual
probl erns areinthestandardf orm theval ue of theithdual vari abl eequal s the
reduccd coeffi ci ent ( isthecoeffi ci ent inthelast r owof the final table) of
the sl ackl surpl us vari abl e associ ated wi th e ith pri mal constrai nt. ln Tabl e
3. 22 the val ues of the reduced cost coeffi ci ent correspondi ng to the surpl us
vari abl es SI S2' S3 and S4 010 0 and 0 respecti vel y. These arepreci sel y
theopti mal val ues of thedual vari abl es inTabl e 3.29.
9. If theithpri mal vari abl eis nonbasi c theval ueof its reduced cost coeffi ci nt is
theval ueof thesl ackl surpl us vari abl eof thecorrespondi ngdua1constrai nt . In
Tabl e 3. 22 X2 is a nonbasi c vari abl ewi threduccd cost coeffi ci ents of 5. Thi s is
the val ue of S2 inTabl e 3.29. Si nce XI and X3 arebasi c wi ththercduced cost
coeffi ci ent of O ecor spondi ngsl ackl surpl us SI and S3 vari abl es will b zero
as observed inT le3.29.
The above list referred to the pri mal and dual probl ems instandard form. A si mi l ar
list can bc obtai nedfor nonstandardform. There will be appropri atemodi fi cati ons for
negati veval ues of vari abl es wel l as equal i tyconstraints. The listedreferences can
be consul tedfor further i nformal i on.
3. 5. 2 Sensl tl vi ty Anal ysl s
The sol uti on10theL P probl emis dependenl on theval ues for ecoeffi ci ents c b
and A (alsotermedparameters) i nvol vedintheprobl em. Inmany practi cal si tuati ons
these par nel ers are onl y known approxi matel y and may change f romits current
val ue for any number of reasons parti cul arl y afler a sol ul i onhas beenestabl i shed.
I nsteadof recomputi ng a newsol uti onit is possi bl etoobtai na freshone basedon the
exi sti ngone and its final Si mpl ex table. Thi s i ssueis si gni fi cant inpractical probl ems
wi ththousands of vari abl es and constrai nts. The adaptati onof a n wsol uti onwi thout
re-sol vi ng the probl emis cal l ed sensitivi. anal ysi s. In L P probl ems sensitivity
anal ysi s refers todetermi ni ngtherangeof parameters for whi ch theopti mal sol uti on
still has thesame vari abl es inth basi seven thoughval ues al Ihesol ul i onmay change.
Exampl e 3.1is usedfor is di scussi on. Fi gures are edfor i1I ustrati onrather than
a formal proof. Fi gures are not useful for more thantwo variables. General l y mosl
compul er codes that sol ve Ii near programmi ng probl ems also performsensitivily
anal ysi s. Revi si ti ng Exampl e 3.1 it w necessary 10 identify the number of
component pl ncemcnt machi nes of l ypeA and B. The obj ecti ve is to maxi mi ze e
number of boar tobe manufactured. Constrai nt g I represents theacqui si ti ondol l ars
avai l abl e. Cons 'aint g2represents thefloor spaceconstrai nt. Constrai nl g3represents
the number of operators avai l abl e. The probl emstatement and the results are
reproducedbel ow(note X3' X4and Xs arethcsl ackvari abl es)
The sol uti onis
Mi ni mi ze j(X): - 99Ox
I
- 91X2- 5250
subj ecI to: gl (X): O.4xl - + 0. 6X2+ X3= 8.5
(3.5)
(3.6)
g2(X): 3xI - X2+ X4= 25 (3.7)
g3(X): 3xI + 6x
2
+ X5 = 70 (3.8)
x
1
O x
2
o x
3
0 x
4
0 X
s
0 (3.9)
x ; = 10.4762 X; = 6. 4285x ; = 0.4524] = - 21437.14
Changi ng Cost Coeffi cl ent Val ues (c): First consi der the e ect of changi ng
thecost coeffi ci enl s of thedcsi gnvariablcs. If CIOthecoeffi ci cnt of desi gnvari abl eXI
is changed f rom- 990 will theset of basi s vari abl es rcmai n thesame?
20
5
30
g s
1
-15
-20
-30
0
35 AODl TI ONAL TOPI CSINLl NEARPROGRAMMI NG 149
Sensltlvlty10Cost Coefflclent
25
?
. - -
- - - - -
- 16150
-
.
-- 16150
-16150
c1= 200- optl mumchanged10A
5 10 15
Machl nes of Type A
guro3.9 Sensitivityanalysis changi ngcosl c icient .
Changi ng thc cost coeffi ci ent changes the sl ope of the line representi ng e
obj ecti ve functi on at may l eadtoa change in sol uti on. For exampl e Equati on
(3.5) can be wri tten
x
2
= {990/ 900) X I ([5250+ j]/ 900)
If CI were to be made - 1900 en the sl ope of e line will be (1900/ 900). The
sensitivity due to CI can be experi enced by runni ng Sensl tl vl ty_cost. m.
1
The
change wi l h respect 10 the ori gi nal obj ecti ve functi on is shown in ani mati on. The
ori gi nal sol uti onismarked on thefi gureby a circle. The ori gi nal sol ul i onisunchanged
when t hema i tudeof thesl opeis rai sed for i nstanceCI = -1900. On theother hand
if CI were 10be made - 200 Ihedottedlinei ndi cates thenewcost lineand inthis case
thesol uti onwi1l change topoi nt A. This is shown inFi gure 3.9 is E sult of
runni ngthem- fiI e. The floor constrai nt is no l onger bi ndi ng. It cunbeestabl i shedby
si mpl e cacul ati ons infinitely many sol uti ons are possi bl e if CI has a val ue of
- 450. InIhis casc thecost functi onis parallel tothebi ndi ngcons int 83' Fromthis
di scussi onit is apparent 10keep thel ocati onof theori gi nal sol uti onunchanged
CI must be greal er than -450. Such anul ysi s is possi bl e for all Ihe other cosl
coeffi ci enl s i nvol ved in is probl em. In parti cul ar the anal ysi s shoul d indicate for
lFiles 10bcdownl ondedfromtl 1e websilcnrcindicnledbyboldface ftype
150 L1NEARPROGRAMMI NG
what range of cost coeffi ci ents the sol uti on woul d still remai n where it is. This is
referredtoas determi ni ngsensitivitytocost coefficients.
Change Inthe Resour ce Ll ml ts (b Vector): When theval uSon thcri ght-hand
side changcs the constrai nt I i nesl pl an/ boun ries are moved paml l cl to themsel ves.
lis changes thefeasibleregion. Th reforetheopti mal vaI ues may aIso ch gedue to
thechanges in efl sibl :gion. Once agai nthechanges canbe illus tedby runni ng
S ens vi s. m. Fi gure3.10iII us tes thechange intheri ght-handsidevaI ueof
efrrst nstr nt. b.. mits vaI ueof 8.5to6.5. This chang caus tobecome an
activecons aint . Inthis case theopti maI sol uti onhas moved fromA toB. Note that in
theoriginaI probl emg 1 was not active. In enewformul ati onit is. However insensitivity
anal ysi s eprobl cmis todi scover thcrangeof b. so thesol uti onstill has esame
vari abl es in ebasis. lf b
l
we toremai nabove 8. 0475 thenthesol uti onwoul d still be
at A dgl will not be anactiveconsai nt
Change Inthe Coeffl cl ent Matrl x A: A change in the coeffi ci ent matri x is
si mi l ar in effect to that due to the change in the cost coeffi ci ent whi l e directly
20
15
10
5
o -5
E
10
-15
-30
0
SensltlvltytoresourceIImlts
b1: 6. 5- opti mumchanged- acti veconstrai nt
5 ro 15
Machl nes 01Type A
F1gure3.10 Sensillvityanalysis changl ngrighthandside.
REFERENCES 151
20r
Sensitivitytoconstrai nt coel fi ci ents (a11)
l i l ~ O
-
- .
- ....
. . .
": - - - -

5
@
~

-15
-20
my
a1' :0.6- optl mumchanged
s 10 15
Machi nes of Type A
Fl gure3.11 Sensillvilyanalysis changi ngcosl coefflclenl .
i mpacti ng probl cmby changi ngthefeasi bl eregi on. These changes alsodepend on
whether the vari abl e in the col umn is a bnsi c vari nbl c 01' a nonbasi c variublc.
Sensl tl vl ty_coef f . millustrates the result of changi ng the coeffi ci ent all' An
i ncrease inthe vaI ue moves the opti maI sol uti on to B whi l e a decrease l eaves the
sol uti onunchanged. Inbothsi tuati ons XI and x2 arestill part of ebasis. Inthefirst
case g. becomes a bi ndi ng constrai nt. Thi s is illustrated in Fi gure 3.11. Thc
coeffi ci ent is in the flft col umn whi ch corresponds to a basi c variable. Changi ng
coeffi ci ent val ues in the first col umn in other constrai nts wi1 yi el d si mi l ar
i nformati on. The reader is encouragedtotryother changes inthecoeffi ci ent usi ng c
m-fi l es avai l abl einthecode secti onof this chapter.
REFERENCES
1. Dantzi g G. B. Li near Programmi ng and Extensi o Pri nceton University Press
Pri nceton. NJ . 1963.
2. Luenberger D. G. Li near and Nonl i near Programmi lJ gAddi son-Wesl ey. Readi ng.
MA. 1984.
3. Wi nston W. L. l ntroducti on to Mathemati cal Programmi ng. Appl i cati ons and
AI gori thms Duxbury P ssBcl mont CA 1995.
4. Arora J. S. l ntroducti onOpti mal Desi gl l . McGr aw- Hi I I NewYork. 1989.
5. Wi l1i ams G.. Li near AI gebra wi thAppl i cati ons. Wm. C. Brown Publishers Dubuque.
I A 1991.
6. Nob1c. B. . dDani el J. W. Appl i edLi near AI gebra. enti -Hal l Eng1ewood Cliffs.
NJ . 1977.
PROBL EMS
(For all two-vari abl e probl ems provi de a graphi cal defi ni ti onl sol uti on of thc
probl em. )
3.1 Sol ve thefol l owi ngI i near programmi ng probl em:
Mi n ft.. xl X:z}: XI +X2
Sub: 3xI - X2 3
XI
XI +X2S 4
XI
3. 2 Sol vc thefol l owi ngl i near programmi ng probl em:
Max f(x )o xz): XI +
Sub: 3xI 3
XI +2x
2
S 5
XI +X2S 4
XI 2! 0; X2 2! 0
3.3 Sol veth f ol l owi ngl i ncarprogrammi ngprobl cm:
Mi n f(x " X2): XI - X2
Sub: 3xI - X2 3
XI +2x2S 5
XI +X 2 S 4
XI 2! 0; X22! O
3.4 Sol ve thefol l owi ngI i near programmi ng probl em:
M f(x " X2): XI + X2
Sub: 3xI - X2 2! 3
XI +2x
2
S 5
XI +X2S 4
XI 2! 0; X22! 0
3.5 Sol ve efol l owi ngI i near programmi ng probl cm:
' A':_
1' 1.. u ..
Sub: 3xI - X2 S 3
XI
XI
XI 2! 0; X2u c dinsi gn
PROBLEMS 153
3.6 The l ocal bookstore must determi ne how many of each of thefour newbooks
on photoni cs it must order tosati sfythencwi nterest generatedinthedi sci pl i ne.
Book 1 costs $75 wiII provi de a profi t of $13 and qui res 2 i nches of shel f
spacc. Book 2 costs $85 wi1l provi de a profi t of $10 and requi res 3 i nches of
shel f space. Book 3 co $65wi l l provi de a profi t of $8 and requi r 1 i nchof
shel f space. Book 4 costs $100 wi l l provi de a profi t of $15 and requi res 4 i nches
of shel f space. Fi nd thenumber of each typethat must be orderedtom ize
profi t . Total shel f space is 100 i nches. Total amount avai l abl e for ordcri ng is
$5000. It has been deci ded toorder at 1east a to of 0of Book 2 and Book 4.
3.7 The l ocal communi t y col l ege is p1anni ng to gr ow bi otechno10gy offeri ng
through newfcderal and state grants. An ambi ti ous programis bci ngp1anncd
for rccrui ti ngat least 2stude f rominand out of state. They aretorecrui t
at least 40 out-of-statc students. They wi11attempt to ruit at least 30 students
who are in the top20%of their graduati ng hi ghschool c1ass. Current fi gu S
i ndi cate about 8 % of theappl i cants f rominstate and 6 % of theappl i c ts
f romout of statebel ong to ispooL They al sopl antorecrui t at least 40 students
who have APcourses inbi o10gy. The datasuggest that 10%and 15%ofi n-state
and out-of-stal e appl i cants respecti ve1y bcl ong to is pool . They anti ci pal c
that theaddi ti onal cost per student is $800 for cach i n-statc sl udent and $1200
for each out-of-statc student Fi nd thcir actual enrol l ment necded tomi ni mi ze
cost and their actual cost.
3.8 Fi gure3. 12rep sents an opti mi zati onprobl emtodel ermi ne themax . i mumtOla1
number of smaI1er rectang1es at wil fit wi thi n thc l arger one. The
di mensi ons are i ndi cated on e fig " C. ne number of the 1arger r ctang1e
shoul d be at 1east 5 mor c than the sma1er oncs. Thc rectangl es cannot be
rol ated. No fracti onal rcl angl es areal1owed.
(Opti onal : conf i rmaIl sol uti ons usi ngtheOpti mi zal i onTool box . f romMATI . AB.)
100
t
1.5
2. 5 70
Rguro 3.12 Probl m3. 8.
4
NONLI NEAR
PROGRAMMI NG
Opti mi zati on probl ems whose mathemati cal model s are characteri zed by nonl i near
equati ons are cal l edNonl i near Programmi ng LP) probl ems. In Chaptr 3 it w
noted at these probl ems al so fell i nto the category of mathemati cal programmi ng
probl ems. Engi neeri ng desi gn probl ems are mostl y nonl i near. In Chap r 2 several
probl ems were exami ned graphi cal l y nnd it was evi dentthat curvatureand thegradi ent
of the functi ons i nvol ved had a si gni fi cant i nfl uence on the sol uti on. In subsequent
chapters we wi l l conti nue to center e di scussi on of opti mal i ty condi ti ons and
numeri cal tcchni ques around two-vari abl e probl ems because e i deas can al so be
expressedgraphi cal l y. Extensi on to mor e thant wo vari abl es is qui testrai ghtforward
and is most si mpl e when thep sentati onis made throughtheuse of vector a1gebra.
MATI . AB wi l l be uti l i zedfor all graphi c needs.
Tradi ti onal l y there is a bottom-up presentati on of materi al for nonl i near
opti mi zati on. Unconstrai nedprobl ems are di scussed first fol l owd by constrai ned
probl ems. For constrai ned probl ems e equal i ty constrai nedprobl emis di scussed
first. A s ilar progressi on is observed wi th gard to the number of vari abl es. A
si ngl e-vari abl e probl emis i ntroduced f ol l owed by t wo vari abl es whi ch is en
extended to a genernl probl emi nvol vi ng" vari nbl es. Thi s order al l ows i ncremental
i ntroducti onof newconcepts but pri mari l y al l ows thecreati ve use of exi sti ngrul es
toestabl i shsol uti ons totheextended probl erns.
An anal yti cal foundati onisessenti al tounderstandand establ i shthecondi ti ons at
the opti mal sol uti on will have to satisfy. lis is not for the sake of mathemati cal
curi osi ty but is essenti nl component of the numeri cal techni que: notabl y the
stoppi ng criterin. The necessary mnthemnti cal defi ni ti ons and i l l ustrati ons are
i ntroducedinthis chapter. Ref erences ar nvai l nbl efor refreshi ngthecal cul us and the
154
numeri cal techni ques essenti al 10thedevel opment of NLP [12). The books fami l i ar
to the render shoul d do admi rabl y. This chnpter al so i ntroduces the symbol i c
computioll (computer al gebra) resource avai l nbl e in MATLAB namel y Symbol i c
Mat h Tool box [3).
4.1 PROBL EM DEF I NmON
InNL P it is not essenti al at all thefuncti ons i nvol vedbe nonl i ncar. It is suffi ci ent if
j ust onc of t hemis nonl i near. Ther aremany exampl es inengi neeri nginwhi ch onl y
eobj ecti ve functi onis nonl i near whi e theconstrai nts arelinear. lf inthis case the
obj ecti vefuncti onis a quadrati c functi on theseprobl ems aretermed lillear quadral i c
probl ems (LQP). Opti mi zati on probl ems for most p rely on experi ence to
i denti fythemathemati cal model compri si ng of thedesi gnvari abl es obj ecti ve and the
constrai nl s. A knowl edge of engi neeri ng or the appropri ate di sci pl i ne is al so
essenti al tocstabl i sha matl l emati cal model . Pri mari l y this i nvol ves determi ni ngthe
functi onal rel ati onshi ps among thc desi gn vari abl es. Thc remai ni ng task then is 10
establ i shthesol uti on.
How does one establ i shthesol uti ontothenonl i near opti mi zati onprobl em?
In mathemati cs (nfter nll at this stagc Lhere is a mathemati cal model for the
probl em) thesolutioll is obtai nedby sati sfyi ngthenecessaand SI{ cient condi ti ons
rel ated to the cl ass of The necessary condi ti ons ar those rel ati ons that a
C di datefor theopt i mumsol uti onmllst sati.. I f i t does enand this is i mportant
it may be an opti mal sol uti on. To qual i fy a desi gnvector XP (X represents thedesi gn
vector) as an opt i mum it must snti sfy addi ti onnl rel ati ons cnl l ed the sujJi cient
condi ti ons. Therefore opt i mumsol uti onmust sati sfybo necessaryand suffi ci ent
condi ti ons. Thi s chapter establ i shes these condi ti ons for the opti mi zati onprobl em.
Exampl e 4.1 is establ i shednext and is used inseveral ways inthe remai nder of E
chapter 10 devcl op the condi ti ons menti oned above. Once 1he condi l i ons are
avai l abl e thenumeri cal techni ques inopti mi zati onwiUi ncorporatet hemtoestabl i sh
thesol uti on.
4. 1. 1 Pr obl emFor mul at l on - Exampl e 4. 1
The probl emis stri ctedto t wo vari abl es todrawgraphi cal support for some of E
di scussi ons. There aret wo constrai nts whi ch duri ngthedevel opment of this chapter
may swi tchbetwccn equal i ty and i nequal i tyconstrai nts toil ustrate l atedfeatures.
Probl em: Fi nd thc rectangl eof thc l argest ar (in posi ti vequadrant) atcanbe
anscri bedw ina gi venel l i pseand sati sfya prescri bedl i near constrai nt.
Fr om c probl emspeci fi cati on the el l i pse will provi de an i ncqual i ty constrai nt
whi l e e l i near rel ati on among thc vari abl cs will provi de cqual i ty constrai nt for
tbis cxampl e.
156 NONLl NEAR PROGRAMMI NG
3
2.5
x
z
0.5
/
/

/
/

/
f

J
f
f

/
/

0.5 1. 5 2 2.5
X
1
Fl gure 4.1 Consl rai nl s of Exampl e 4.1.
Mat hemat l cal Model : Fi gure 4.1 capturcs the es of thep l cm. Code filcs
Ex4L1. m
l
and creat e_el l l pse. mare filcs n cssary for MATI . AB to create the
fi gure. 'herearet wo vari abl es X I and X2 l erearestandardmathemati cal expressi ons
for eel l i pse and thestrai ght l i neis no probl cmafter Chapter 3.
St andar d Format : The standard f ormat of the NL P is reproduced here for
convemence:
Mi ni rni ze f(xtt X2' X
n
) (4. 1)
Subj ect hk(x. h . x) = 0 k = 12 . L (4.2)
(XI X2. . Xn) $; 0 j = 1 2 . . m (4.3)
xl $; Xj $; xl' ; =12... n (4.4)
Invector notati on
IFi I 10bedownl oadedfromIhewebsilc cindicllledbyboldfnce riflype.
3

4.1 PROBL EMDEFI NI TI ON 157
Mi ni rni ze f {X) ln (4.5)
(4.6) Subj ect to: [h(X)] = 0
[g(X)]m$; 0 (4.7)
X
10w
$; X $; Xup (4.8)
For the speci fi c probl embei ng di scussed and ref eni ng to Fi gure 4.1 the desi gn
vari abl es are the cool' di nate val ues xl d X2 al al l owthc computati on of the
rectl I ar area. The opti rni zati onprobl emis
Mi ni rni ze -XI X2 (4.9)
(4.10) Subj ectto: hl (x. xz}: 20xI + 15X2 - 30 = 0
81( X. X2): (xT/ 4) + (4.11)
O$; xl $; 3; O$; x2$; 3 (4.12)
The si deconstrai nts in(4.12) can al so be postul atedas one-si dedand can be wri tten
as
X
1

(4.13)
4. 1.2 Dl scussi on of Const rai nt s
Usi ng the rel ati ons (4.9) (4.12) several addi ti onal c1asscs of probl ems can be
descri bedby i ncl udi ngonl y a subset of thc re1ati ons. They areexami ned bel ow.
Unconst ral ned Probl em: Thcre are no functi onal constrai nts al though the si de
constrai nts arc necessary tokeep thesol uti onfinite. For this cxampl e
Mi ni rni ze (4.9) j{X
I
-x
1

In this probl em if desi gn vari abl cs are unbounded at c upper l i mi t then e
sol uti onwoul d be at thel argest posi ti veval u of XI and X2A two-si dedlimit for the
dcsi gn vari abl es is usual l y a good i dea. The desi gner does have the ponsi bi l i ty of
defi ni ngan acceptabl edesi gnspace.
Equal i ty Const ral ned Pr obl em1: The functi onal constrai nts inthisprobl em
onl y equal i ti es. Wi t h reference10Exampl e 4.1 c fol l owi ngprobl emcanbe scl up
(after changi ng c i nequal i tytoan cqual i ty):
158 NONL1NEAAPAOGAAMMI NG
Mi ni mi ze ft.. x. X2): -x. x2
Subj ect to: 11)(x.. X2): 20 x. + 15X2- 30 = 0
h
2
(x. X2): + ( 4) 1=0
::;;3;
I ntui ti vel y such a probl emmay not b opti mi zed si nce the t wo constrai nts by
themsel ves shoul d establ i sh the val ues for the I wo desi gn vari abl es. The argumenl s
used inL P for accepl abl eprobl emdefi ni l i onareal soval i dh . There is al ways the
possi bi l i tyof mul ti pl esol uti ons- whi ch is a strongfeal ureof nonl i n ar probl ems. In
such an event theset of vari abl es that yi el d el owsl val uefor theobj ecti vewi l l be
theopti mal sol uti on. NOl e sucha sol uti onis obtai nedby scanni ng theacceptabl e
sol uti ons rather anthrough appl i cati onof any ri gorous condi ti ons.
Equal l ty Const ral ned Pr obl em2: I ftheprobl emwe 10incI ude onl y one of l he
constrai nts for exampl e
Mi ni mi ze ft.. xl X2): -XI X2
Subj ecl l o: h
2
(x"X2): +
0::;; XI ::;; 3; 0::;; X2::;; 3
Thi s is a val i dopti mi zati onprobl em. A si mi l ar probl emcan be defi nedwi l hthefirst
constrai nt by itself.
I nequal l ty Const rai ned Probl em: Inthis casetheconstrai nts arc all i nequal i ti es.
A vari ati on on Exarnpl e 4.1 woul d be (the equal i ty cons aint is transf ormed to an
equal i ty constrai nt)
Mi ni mi ze ftx.. x2l: -XI Xz
Subj cct to: g. (x" X2): 20 XI + 15X2- 30::;; 0
g2(X"XZ): (xt/4) + ( 4) 1::;;0
O::;;XI ::;;3; 0::;;x2::;;3
Li ke its counte t in l i near programmi ng is is a val i d opl i mi zati on probl em.
Equal l y val i d woul d be a probl emthat incI uded j ust one of l he constrai nts or any
number of i n ual i ty constrai nts.
It is essenl i al to understand both the nature and the number of constrai nts as
wel l as how they affect the probl em. I n general equal i ty constrai nts are easy 10
handl emathematI Cal l ydi ffi cul ttosati sfy numeri cal l y. andmorerestri cti veonthe
search for the sol uti on. I nequal i ty constrai nts are di ffi cul t to resol ve
mathemati cal l y and are mor e fl exi bl e wi th respect to the search for the opti mal
sol uti onas they defi ne a l arger feasi bl eregi on. A wel l -posed probl em that
4.2 MATHEMATI CALCONCEPTS 159
someconstrai ntsareacti ve (equal i ty) at theopti mal sol uti onas otherwi se it woul d
be an unconstrai ned sol uti on.
4.2 MATHEMATI CAL CONCEPTS
Li ke L P some mathemati cal defi ni ti ons are n sary before the n sary and
suffi ci ent condi ti ons for theNL P can be establ i shed. Defi ni ti ons areneeded for both
e anal yti cal di scussi on as wel l as numeri cal techni ques. MATLAB provi des a
Symbol i c Mat h Tool box whi ch permi ts symbol i c computati on i ntegral ed in c
numeri cal envi ronment of MATLAB. This al l ows I he user 10 expl o probl ems in
cal cul us l i near al gebra sol uti ons of systemof equati ons and other areas. In fact
usi ngsymbol i c computati on students can easi l yrecover epr qui si tei nformati on
for thecourse. A short hands- on exerci setosymbol i c computati on is provi ded.
1
'he
pri mary defi ni ti ons we need arederi vati ves partial deri vati ves matri ces deri vati ves
of matri ces and sol uti ons of nonl i near equati ons.
4.2.1 Symbol l c Comput at l on Usl ng MATLA
One of thebesl ways to gel f ami1iar wi thsymbol i c computati on is totakethequi ck
onl i ne i ntroducti on 10 the Symbol i c Mat h Tool box avai l abl e in the MATLAB Demos
di al ogbox [4]. The fol l owi ngsequence l ocates thelutorial:
>> demos
Symbol i c Mat h - > I nt r oduct i on
The computal i onal engi ne execul i ng symbol i c operati ons in MAT1. AB is thc
kemel of Mapl e markel ed and suppo d by Waterl oo Mapl e Inc. lf E ader is
al ready fami l i ar wi th Mapl e en MAT1. AB provi des a hook through whi ch Mapl e
commands can be ex uted in MATLAB. The symbol i c computati on in MATLAB is
perf ormed usi nga symbol i c obj ect or sym. Thi s is another datatypeI i ke thenumber
and stri ngdatatypes used in rlier exerci ses. The Symbol i c Mat h Tool box uses sym
obj ects to present symbol i c vari abl es expressi ons and matri ces.
Intheexerci sethat fol l ows a functi onof one vari abl e and t wo functi ons of I wo
vari abl es (constrai nts f romExampl e 4.1) at1 used for iII ustral i on. Drawi ng on the
au or' s c1assroomex nce is prel i mi nary di scussi on is interms of vari abl es X
and y for i mproved comprehensi on. I nlater secti ons subscri pts on x areused todefi ne
mul ti pl evari abl es so that etransi ti ontothegeneral probl emcan be facilitatedusi ng
vector descri pti on. The functi ons inthcsc exeai ses are
f t x) =12+( x- I ) 2( x- 2) ( x- 3)
g. (xy): 20x+ 15y - 30
g2(Xy): x2/4+ l - 1
ITOOrclcasc of 100 newSwdent Edi l l ol l includcs Ihc symbolic loolbOl ulongwilhIhe basic MATLAD
sortw t
161 4.2 MATHEMATI CAl CONCEPTS
-
g

'
E
=
S
E
. f
S
5

1
n
l
d
a

9 col umn vect or based on gl g2 9 '" [ gl ; g2)
9
[ 20*x+15*y- 30]
1/ 4*x+y- 1]
9 can be t he const r ai nt vect or i n opt i mi zat i on
pr obl ems
t he par t i al der i vat i ves of 9 wi t h r espect t o desi gn
var i abl es i s cal l ed t he J acobi an mat r i x
t he pr oper t i es of t hi s mat r i x ar e i mpor t ant f or
numer i cal t echni ques
xy .. [x y] ; r owvect or of var i abl es
J '" j acobi an( gxy) cal cul at i ng t he J acobi an
15]
1]
a pl ot of f f or - 2 pi
df di f f ( f ) ;
h01d 00
ezp10t ( df [041) pl ot t i ng f unct i on and der i vat i ve
combi ne wi t h MATL AB gr aphi cs - dr aw a l i ne
1i oe ( [ 0 4] [0 0) ' C010r ' r' )
g
9
[ 20*x+15*y 30]
1/ 4*x+y- 1]
( def aul t ) 2 pi
<= 4
J
20
[ 1/ 4
ezp10t ( f ) x
pl ot bet ween 0 <= x ezp10t ( f [04) )
"J' ljl
NONLl NEAR PROGRAMMI NG
le fol l owi ngMATL AB sessi on was capred as a di ary file and edi ted in a text
editor. The fo e MATL AB prompt does not appe . le bol dface words are
commands at ereader will typeat thecommand line.
160
def i ni ng x as a si ngl e symbol i c obj ect x " sym( ' x' )
x
x
def i ni t i on of mul t i pl e obj ect s
var i abl es i n t he wor kspace
Byt es Cl ass
126 symobj ect
126 symobj ect
128 symobj ect
128 symobj e' ct
126 symobj ect
126 s n obj ect
g
of
g1 g2
t ypes
Si ze
1X1
1x1
1X1
1x1
1X1
1X1
syms y f
whos %
Name
f
g
gl
92
x
y
Gr and t ot al i s 14 el ement s usi ng 760 byt es
const r uct i ng f (x1) *( x- 1) *( x2) *( x3) f.. 12 +
f
12+( x- 1) " 2*( x2) *( x- 3)
di f f ( }
aos
2*( x- 1) *( x- 2) *( x- 3) +( x- 1) ^2*( x- 3) +( x- 1) "2*( x- 2)
f i r st der i vat i ve
Y 2. 5
Addi ti onal symbo1 i c computati ons will be i ntroduced throughcode as appropri ate.
Note that e result of both numeri c and symbol i c computati ons can be easily
combi ned a10ng wi thgraphi cs toprovi dea powerful computi ng envi ronment .
t o eval uat e 9 at x 1
subs( g{xy}{12. 5})
ans
27. 5000
1. 7500
3
5
Z O ~
3
2
not e t he chai n r ul e f or der i vat i ves
% not e t he i ndependent var i abl e i s assumed t o be x
di f f ( x2)
ans
2*( x- 2) *( x 3) +4*( x- 1) *( x- 3) +4*( x- 1) *( x 2) +2*( x- 1) "' 2
t he second der i vat i ve wr t x
t he t hi r d der i vat i ve wr t x }

X
2

'
n
H
E

a
L
=
(
x

f
g
*

f
n
4

1
a
2

d

def i ne 9 n
u
q
d

y

-
w
n
u
w

E
d
-
-
A
T
Y

*

X
5

*
4
0
+

2
x

=
*

E
n
u

t
-
1
4
a
g
g

def i ne g2
gl g2 can onl y have par t i al der i vat i ves
i ndependent var i abl es have t o be i dent i f i ed
}

x

t
-
g

'
t
=
8
-
a
E
M
n
u

i
n
2

d
a

par t i al der i vat i ve
g2 '" 0. 2S*x + Y 1;
162 NONLl NEAAPAOGAAMMI NG

4. 2. 2 Basi c Mathemati cal Concept s
The basi c mathemati cal e1ements in the di scussi on of NLP are deri vati ves parti
deri vati ves vectors matri ces J acobi an and Hessi an. We havc used the Symbol i c
Mat h Tool box in theprevi ous sccti ontocal cul atesome of thesequanti ti es wi thout
dcfi ni ng them. These topi cs will have been extensi vel y covered in the foundati on
courses on mathemati cs inmost disciplines. A brief revi ewis offeredin is secti on
usi ngMATLAB. lis opportuni ty will alsobe utilizedtoi ncreasefamiliarity wi the
Symbol i c Math Tool box and i ncorporat it wi th eMATLAB commands that were
usedin earlier chapter.
Funct l on o' One Vari abl e: ft.. x) identifies a functi onof one variable:
f (x) = 12+ ( x- l i ( x- 2) ( x- 3)
is usedas a specific exampl e of sucha functi on. The deri val i veof efuncti onat e
l ocati onx is wri ttenas
- 1 kx + ax) -.f(x)
dx- i l m Ax =l l fI 1-L
axo ax ax->oax
(4.14)
x is the poi nt about whi ch deri vati ve is compud. r is thc di stame tu a
nei ghbori ngpoi nt whos l ocati onthereforewill be x + ax. The val ueof thederi vati ve
is obtai nedas a limit of thc ratioof edi fferen intheval ueof thefuncti onat e
two poi nts ( edis separati ngthe two poi nts ( ) this sep tion is
reducedzero. The computati onof thcderi vati vefor thespecificex npl eis usual l y
obtai n dusi ngtheproduct rule. Resul ts fromtheexe iseon symbol i c computati on
provi dethc resuJt
= 3) + (x- 1) ) + 1) ^ 2)
Tabl e 4.1 illustrates elimitingprocess for thc deri vati veof thefuncti onchosen as
ecxampJ e. The deri vati ve is bei ngeval uated at poi nt 3. As thedi spJ acements are
reduced val ueof theratioapproachcs val ueof 4 whi ch is theaClual val ueof
thederivative. Whenax is l arge(wi tha val ueof 1) cratiois 18. Thi s issignificantly
different f romtheval ueof 4. Wi th ax of 0.1 theval ueis around4. 851. Wi th fur er
reducti ontoO. 1 theratiohas a val ueof 4. 008. Fromthesecomputal i ons it Canbc
expected as ax approaches 0 theratiowill reachtheexact val ueof 4.0.
Numeri cal Deri vati ve Compu 'ion: Many numcri cal techni qucs in NLP (1 ui
ecomputati onof derivatives. Most softwarei mpl ementi ngthesetechni ques do not
usesymbol i c computati on. Automati on and seof use qui re at esederi vati ves
becomputed numeri cal l y. leresults inTabl e 4.1justify enumeri cal computati on
of a deri vati vethrougha techni quecal l edthefi rsl forwardd rence[5]. Usi ng a very
smal l pe urbati onax lh deri vati veat a poi nl is numeri cal l y cal cul atedas theratio
4.2 MATHEMATI CAL CONCEPTS 163
Tabl e4.1 Cal cul ati onor Deri vaUve
x dx x+dx M
dfl dx Derivative
3 4 18 18 4
3 0.1 3.1 0.4851 4.851 4
3 0.01 3.01 0. 0408 4. 0805 4
3 0.001 3.001 O. 4 4.008 4
of thechange inthefuncti onval ue 4f 10 echange in di spl acemenl ax. For E
exampJ e thederi val i veat x = 3 wi thax = 0. 001 is obtai nedas
El I _f.: (3+ 0. 001) 1(32- .11 nnSl
dxl .rc3 0. 001
(4.15)
The deri vati vc for lhesi ngJe-vari abl efuncti onal any val uc x is aJsocaJl edlhesl ope
or thegradi etl t of lhc functi on al poi nt . If a lineis drawn tangenl 10 thefuncti on
al theval uex thetangent of E glethat is Jinemakes wi thex-axi s will have
thes neval uc as thederivative. If is angl eis 9 en
=tan9
Fi gure4. 2illustrates etangency propertyof thederi vati ve.
MATLAB Code: Fi gure4. 2was creal edby thecode tangent. m. Inthesame di rectory
e code derl vati ve. mprovi des the illustration of tangency through a fi re
ani mati oninMATLAB. In efigurewi ndowboththefuncti on(drawnas thecurve) as
wel l asthelinerepresenti ng4f and ax aredrawn. As thenei ghbori ngpoi nts arccl oser
it is evi de omthe figure that thecurve can be approxi matedby a straight line.
When ax = 0. 001 ederi vati veis tangent tothecurveas eyarei ndi sti ngui shabl e.
Hher Deri vati ves: The deri vati ve of the dcri vati ve is the called e second
deri val i ve. Formal l y it is dcfi ned
-!!..( -l i " L!;rn 1. . - 4yal x
tJi l - dxldx) --10 ax : ax (4.16)
Si mi l arl ythel hi rdderi val i veis defi nedas
24
22
201- "
18
12
10
a
o 0.5
tangent - slope- derlvativeat x=3
lo12+(xt)(x-l)(x- 21(x-3)
1.5 2
x
2. 5
Fl gure4.2 lII uslrallon01thel angent .
=)
3 3.5 4
(4.17)
Thi s can go on provi ded efuncti onhas suffi ci ent hi gher-order dependence on E
i ndependent vari abl ex.
Functi on of Two VBri Bbl es: The l wo-vari abl efuncti on
j{x :x214+y2- 1 (4.18)
is chosen toiIl ustratetherequi redmathcmati cal concepts. The first i mportant fcature
of t wo or mor e vari abl es is thederi vati ves defi nedfor a si ngl evari abl e(ordi nary
deri vati ves) do notappl y. eequi val ent concept hereis theparti al derivative. Partial
deri vati ves aredcf med f orcach i ndependcnt vari abl e. lepartial deri vati veisdenoted
by the symbol d. These dcri vati ves are obtai ned in the same way as the ordi nary
deri vati veexcept thc other vari abl es arehel dat a constant val ue. I nthecx npl ewhen
compul i ng ep tial deri vati veof x thc val ue of y is not al l owed tochange.
4. 2 MATHEMATI CALCONCEPTS 165
dfl + ax y) - f(x y) (4.19)
dxl ^ /:u
1(.<. ) 1 )
Thc above rel ati onexpresses thepartial deri vati veof thef uncti onf wi th respect to E
vari abl ex. Graphi cal l y this suggcsts that thc t wo poi nts are di spl aced hori zontal l y
(assumi ng thex-axi s is bori zonta1). Intheabove expressi on(andpri or ones too) the
subscri pt after theverti cal l i neestabl i sbes tbepoi nt a utwhi ch epartial deri vati vewi th
pect tox is bei ngeval uatcd. (xy) se any/al l points. A si mi l ar expressi onc be
wri ttenfor theparal deri vati veof f wi th spect toy. For this exampl e:
For thepoi ntx = 2 y = 1:

- : "=2y
dx - 2' dy
df df
f (2I) = 1; =1 =2
dx- " dy
Unti l this poi nt wc have made use of symbol s reprcsenti ng changes in val ues
(functi ons vari abl es) wi thout a forrnal defi ni ti on. In is book
s( ) : presents fi ni tel si gni fi cant changes inthequanti ty ( )
d( ) o( ) : represents di ercnti al / i nfi ni tesi mal changes in( )
Changes in functi ons occur due to changes in the vari abl es. Fr omcal cul us [1] thc
di rential change inj{xy) ( due to thedi fferenti al change in vari abl es x (dx)
and y (dy) is expressedas
L" dj . df
df =-:J dx + -:J dv
"J dx _. . dy- J
(4. 20)
For convcni ence and si mpl i ci ty the subscri pt representi ng e poi nt whcre the
exprcssi on is eval uated is not i ndi cated. The defi ni ti on of the partial deri vati ve is
apparent intheabove expressi onas hol di ngy at a constant val uei mpl i es at =0.
Another i nterpretati on for the partial deri vati ve can be observed f romthc above
defi ni ti on: cl tangeof tltefuncti onper unit ci tangeintltevariables.
Gradi ent 01theFuncti on: Inthe functi on of a si ngl e vari abl e the deri val i vc was
associ ated wi the sl op. In t wo or mor e vari abl es the sl ope is equi val ent 10 e
gradi ent . The gradi ent is a vector and at any poi nt rep sents thedirectioninwhi ch
thefuncti onwill i ncreasemost rapi d Exarni ni ngtheconventi onal obj ecti veof NL P
mi ni mi zati on of obj ecti vemcti ons the gradi ent has a natural part to pl ay in the
devel opment of methods tosol vetheprobl em. The gradi ent is composed of thepartial
dcri vati ves organi zedas a vccl or. Vectors in is book arecol umn vectors unl ess othcr
noted. The gradi ent has a standardmathemati cal symbol . lt is dcfi nedas
166 NONLl NEARPROGRAMMI NG
Vf =
(4.21)
At this stage it is appropri ate to consol i date is i nformati on usi ng graphi cs. The
graphi caI dscri pti on of e ex npl e defi ned in Equati on (4.18) has to be three
di mensi onal as we need one axi s for x ano er for y and thethirdforj{xy). Chapter
2 i ntroducedthree-di mensi onal pl otti ngusi ngMATLAB. A mor e useful descri pti onof
theprobl mis touse contour plots. Contour pl ots aredrawn for speci fi c val ues of the
functi on. IntheiII ustrati ontheval ues for thecontours of f are 0 12 d3. I nFi gure
4.3 t wo ki nds of contour pl ots are shown. In the tophal f a three-di mensi onal (3D)
contour pl ot is shown whi l e on l owerhal f samei nformati onis presen a
two-di mensi onal (2D) contour pl ot (this is consi de dth standardcontour plot). Thc
2D contour pl ot is mor e useful for graphi caI support of some of thei deas inNLP. The
3D contour wi l l be di spens dwi thin eremai nder of thc book.
MATLAB Code: Fl g4_3. m. The annotati oninFi gure 4.3(i ncl udi ngthetangent linc
and thegradi ent ow) wasdone roughthepl ot itingcommands avai l abl eon e
menu bsr in the fi gure wi ndow( MATLADversi on 5.2 and later). A m 01" pOl"ti onuf
cpl ots a generatedthrough thestatcments inthem- fiI e i ndi catedabove. The codc
mi xes numeri c and symbol i c computati on. It al l ows mul ti pl e pl ots and targets
graphi c commands tospeci fi c pl ots.
Di scussi on o( Fi gure 4. 3: In Fi gure 4.3 poi nt P is on contour f = O. Poi nt Q is on
thecont ourf =2. Poi nt S is on thecontourf =1. Poi nt R has esame y val uc as poi nt
P d esame x vaJ uc as poi nt Q. For eval ues inthefi gurethecontour val ue is 0. 75
(this val ue shoul d be di spl ayedinMATLAB wi ndowwhcn thecod Fl g4_3. mis run).
lineP Q is a measure 4f
linePR represents changes indf whcn d. y is 0
l i neRQ presents changes indf when d.xis 0
Mathemati cal l y f can onl y be esti matedby addi ng thechangcs al ongthe lines PR
andRQsi nce cdefi ni ti onof thepartial deri vati ves onl y pcrmi ts cal cul ati ngchangcs
al ong thecoordi nate di recti ons.
At poi nt S thedottedline is thetangent. The gradi ent at thc samc poi nt is normal
(pcrpcndi cul ar) to c tI.ngent d is dir ted toward i ncreasi ng the vaJ ue of e
functi on(i ndi cated by earrow). By defi ni ti on if dfrepresents a di fferenti aJ move
aJ ong thegradi ent at any poi nt en(dxP measuredal ongthegradi ent vector)
df =+ dy
(4.22)
3
2
0
3
4.2 MATHEMATI CALCONCEPTS 167
3Dconl our for f ( 1=x2/ 4+I - l
.
2
2.5
3

x
2Dconl our
3
2.5
gllldlenl 01S
- longenl 815
2 3
x
Fl gur 4.3 Gradi enl andl angen! IIn a! a point .
Fr om fi guretheval ue of f wiIl change al ong gradi ent di recti on. If dfrepresents a
di fferenti aJ move aJ ong gent line en(dX aremeasured aJ ong gent line)
df =
e J y
(4.23)
Equati on (4.23) shoul d bc zero because movi ng tangcnti al l y (a smal l amount) the
vaJ u offi s not changed.
The J acobi an[J] defi nes a useful way toorgani ze thegradi ents of sevcraJ
functi ons. Usi ng threc vari abl es and t wo func!i onsj{xyz) and g( xyz) thedefi ni ti on
of thc J acobi anis
I eJf
[JJ =1 1

(4.24)
InEquati on(4.24) thegradi ents of the functi onapp in esame row. The first r ow
is egradi ent of f whi l e thesecond r owis thegradi ent of g. lf thet wo functi ons are
coUected i ntoa col umn vector. thedi fferenti al changes [df dg]Tin efuncti ons due
to e di fferenti al change in thc vari abl es [dx dy dz] can be expressedas a matri x
mul ti pucati onusi ngtheJ acobi an
d
r
i
l
l
-
-
L

]

4 [

=

l
E
E
-
J
4
r
B
E
E
-
E
E
E
L

(4.25)
whi ch is si mi l ar toEquati on(4.20).
Hessi an: The Hessi an matri x [H] is thesame as thematri x of second deri vati ves of
a functi onof several vari abl es. For ft... xy)
I da
[ =1
I iJ L[ iJ L[ I
I dydx
(4.26)
The Hessi matri x is s ymm ic. For exampl edefi nedby Equ i on(4.18).
n
v
&
h
f
A
U

Funct l on of n Varl abl es: For ft... X). where ] =[XI X2. . . . x.rr. egradi ent is
The Hessi an is
d'i d
~ dx
1
iJ x
2
iJ x1ax
n
~ a'i
=1dX2ax1 ~
~
a
iJ xndx d..c
(4.27)
(4.28)
4. 2 MATHEMATI CAl CONCEPTS 169
Equati ons (4.27) and (4.28) will app r qui te often in succeedi ng chapters. A f ew
mi nutes of fami l i ari zati onwill provi de sustai nedcomprehnsi onlater.
4. 2. 3 Tayl or' s Theor ern/ Serl es
Sl ngl e Varl abl e: The Tayl or seri es is a useful mechani smtoapproxi mate eval ue
of thefuncti onft... x) at poi nt (x
p
+ ) if efuncti onis compl etel y known at poi nt
x
p
l e exp si onis (for finitc n)
~ ( 2+
+ ) ) + I( )
~
The seri es is wi del y used inmost di sci pl i nes toestabl i shconti nuous model s. It is E
mai nstay of many numeri cal techni ques incI udi ng those in opti mi zati on. Equati on
(4.29) is usual l y truncatedt he first t wo or terms wi eunderstandi ng
approxi mati on wi l l suffer some error whose order depends on term t is bei ng
truncated:
+ I { AY .1.1. fll
[ ( Xp+At) f ( X J & J h ) +2 1 d l J ) 2+O( )3
(4.30)
lf thefirst termis brought to eleft eequati on. di scardi ngtheerror term canbe
wri ttenas
M = f(x
p
+) f(x :X/) = ))= ()+
dxL 21 dX- 1
- A b - x h
InEquati on (4.31) thefirst termon eri ght is cal l edthefi rst-order/li near vari ati on
whi l e thesecond termis esecond-order/quadrati c vari ati on.
Fi gure 4.4demonstrates theapproxi mati ons usi ngTayl or' s seri es of vari ous orders
at poi nt 2.5wi threspect to ori gi nal functi on(red). The pri nci pal i deais todeal wi
theapproxi mati ng curve whi ch has known prope es i nsteadof theori gi nal curve.
lC constant Tayl or seri es is woeful l y i nadequate. le l i near expansi on is onl y
margi nal l y better. The quadrati c expansi on approxi mates theright si deof thefuncti on
comp to theleft si de whi ch is insi gni fi cant eor. le fifthorder expansi on is
defi ni tel y acceptabl e in the range shown. To see how the Tayl or seri es is used
consi der thequadrati c curve about c poi nt x = 2.5. At this poi nt val ue of the
functi onft... 2.5) = 11.4375 theval ueof th first deri vati vef' (2.5) = -0. 75 dthcval uc
of thesecond dcri vati veisf"(2.5) = 4. Usi ng Equati on (4.31)
(4.31)
ft... 2.5+ =1 5+(- 0 ) +
(4.32)
For di fferent val ues of At. bo posi ti vcand ncgati ve theval ue ofj(x) canbe obtai ned.
leplot of Equati on(4.32) shoul dbe thesame as onel abel edquadrati c inFi gure4.4.
171
(4.34)
4. 3 GRAPHI CAL SOLUTI ONS
f ( xp+dx+ ) =f ( xp' ) + +

(4.35)
For nv abl eswi th thecurrent poi nt and d.Xthedi spl acement vector
f p + Il X) = f ( X
p
) + +j mTm
GRAPHI CAL SOLUTI ONS
legraphi ca1sol uti ons arepresentedfor threeki nds of probl ems: uncons ai ned equality
constrai ned andinequalitycons i nedbasedon thefuncti ons inExampl e 4.1.
4.3.1
o 3; 0 3 (4.37)
The sid constrai nts srve to l i mi t the desi gn space. Consi deri ng the obj ecti ve
functi on in (4.36) it is cl ear that the mi ni mumval ue of f wi1l be ral i zed if the
vari abl es are at the maxi mum(Xi = 3=3). The val ue of the functi on is -9.
Fi gure 4.5 il1ustrates theprobl em. The sol uti on is at the boundary of the desi gn
space. Fi g4_5. mis theMATLAB code at wi l l produce Fi gure 4.5. The maxi mum
val ueof thefuncti onis 0 (basedon thesi deconstrai nts). InFi gure 4.5thetangent
and the gradi ent to the obj ecti ve functi on at the sol uti on are drawn in usi ng
MATLAB pl otedi t commands. l D is parti cul ar exampl e the si de constrai nts are
necessary todetermi ne thesol uti on. If thedesi gnspace were i ncreasedthesol uti on
woul d correspondi ngl y change.
Thexampl echosentoillustrate eunconstrai nedprobl emdefi nedby (4.36) and
(4.37) is not usual l y empl oyed to devel op the opti mal i ty condi ti ons for general
unconstrai ned probl ems. Two requi rements are expected to be satistied by such
probl ems: (1) thesol uti onmust be intheinterior of thedesi gnspaceand(2) theremust
b a uni que sol uti on or e probl emis uni modal . Neverthel ess in practical
appl i cati ons thesecondi ti ons may not exi st. Most softwarearedevel opedtoappl y the
opti mal i ty condi ti ons and they rarely verify eserequi rements are bei ngmet. It is
usua1l y thedesi gner' s responsi bi l i tytoensuretheserequi rements aremet. To devel op
eopti mal i tycondi ti ons anaI temate unconstrai nedprobl emis presented:
(4.36) JtXI x2): - XI X2 Mi ni mi ze
(4.38) JtXI x2): ( XI - 1)2+ ( X2 - 1)2- XI X2 Mi ni mi ze
1t 1
I
fifth-order orlg!nal
NONLl NEAR PROGRAMMI NG
Approxlmat!onus!rtgTaylor serles
170
22
-
0
a
Unconst ral ned Pr obl em
4.3
20
18
i
E
12
4
MATLAB Code: Fi gure4.4iscompl etel y createdusi ngthecode Fl g4_ 4. m. 1t uses the
MATLAB-provi ded symbol i c tayl oc" functi on to generate the vari ous expansi ons
about poi nt 2.5.
3.5 3 2.5
Taylor series approximalion.
2
x
1.5
Fi gure4.4
0.5 D
Two or Mor e Vari abl es: The series onl yexpanded to qu 'aticterms. The
truncati onerror is i gnored. 'he two-vari abl efuncti onexpansi onis shown indetai1and
alsoorgani zedinterms of vectors and matri ces. The first -order expansi onisexpressed
interms of thegradi ent . The second-order expansi onwill be expressedinterms of tbe
Hessi anma ix
IMPI+
f(x'J + Y/J + Il y) = Jtx'J' y) + 1 +
Q 1 (.I"p}
IttpY/
e)2f I
( w- Y+
1 r 2 q
21 (4.33)
I f di spl acements are organi zed as a col umn vector [ ]Tthe expansi on in
(4.33) canbe expressedina condensed manner as
4
3.5
3
2.5
.N 2
1.5
0.5
0.5 1.5 2
x
'
2.5
Fl gure4.5 Unconsl rai nedsolution.
o $XI $ 3; O$ X2 $ 3
3 3.5 4
(4.37)
Fi gure 4. 6di spl ays theprobl emand its sol uti on. Fl g4_. 6. mprovi des thecode.that wiII
general e most of Fi gure 4.6. It is cJ ear at the sol uti on appears to be ot
xi = 2 and xi = 2. The opti mal val ue of e obj ecti ve functi on is -2. Inthecontour
pl ot eopt i mumis a poi nt maki ng it difficuJ t todrawthegradi ent. I f is is thecase
eninthreedi mcnsi ons at theopt i mum cgradi ent wilIi e ina pl ane tangent tothe
functi on surface. Movi ng in is pl ane shoul dnot change the val ueof theobj ecti vc
functi on. This observati onis uscd todevel op then essarycondi ti ons later.
4. 3. 2 Equal l ty Const r ai ned Pr obl em
For a two-vari abl c probl emwe can onl y utilize one constrai nt for a meani ngf ul
opti mi zati onprobl em:
Mi nI mi ze j{XI x2): 'XI X2
Subj ect hl(XI x2): X!/ 4 + - 1
(4.36)
(4.39)
4. 3 GRAPHI CAL SOL UnONS 173
x
t
gure4. 6 Unconstrai nedprobl em: interior 1lI llon.
OS X
1
$3; OSx
2
S 3
(4.37)
Fi u" e 4.7 ( rough FI g47. m) i1I ustrates the probl em. The dashed l i ne is e
ns ai nt . Si nce it is an equaJ ityconstrai nt thesol uti onmust bea poi nt on thedashed
line. The contour of f = 1appears toj ust gr e constrai nt and therefore is the
mi ni mumpossi bl evaJ ue of thefuncti onwi thout vi ol ati ngtheconstrai nt . InFi gure4.7
egradi ent of theobj ecti vefuncti onas weIl as egradi ent of econstrai nts at the
sol uti onari l I ustrated. It appears at at is poi nt thesegradi enl s areparal l el cvcn
though ey aredi rectedopposi te to cach other. By defi ni ti on thegradi ent is inthe
di recti onof thc most rapi di ncreaseof thefuncti onat thesel ectedpoi nt . Thi s is not a
coi nci denc. This fact is used toestabl i shthenecessary condi ti ons for theprobl em.
4. 3. 3 I nequal l ty Const I ned Pr obl em
The number of inqual i tyconstrai nts is not dependent on thenumber of vari abl es. For
i1I us nbo cons int functi ons of Exampl e 4.1 ar formul ated as i nequal i ty
cons amts.
Mi ni mi ze j{x) X2): - XI X2 (4.36)
4
3.5
3
2
1. 5
0.5
0.5 1.5 2
X
1
2.5
Fl gure4. 7 Equalitynstralnedprobl em.
Subj ectto: gl (XI x2): 20xI + 15x2- 30S0
g2( Xl>X2): xI / 4 + I SO
OSXI S3; OSX2S3
3 3.5 4
(4.40)
(4.41)
(4.37)
Fi gure4. 8 ilI ustrates thegraphi cal sol uti on. Thesol uti on i sat thei ntersecti onofthe
dotted lines. The code is a vai l abl einFl g4_8. m. legradi ents aredrawn usi ngthe
pl otedi t functi ons on themenubar inthewi ndow. Al so font styl eand font sizehave
becn adj ustedusi ngthepl otedi t commands. The opti mal sol uti onmust lieon or to E
left of the dashed line. Si mi l arl y it must I i e bel owor on the dashed curve.
Si mul taneousl y it shoul d a1so decrease the obj ecti ve functi on val ue as much as
possi bl e.
4. 3. 4 Equal i ty and l nequal l ty Const ral nt s
Exampl e 4.1is devel oped wi thbothtypes on constrai nts present . Il is producedhere:
4
3.5
3
2.5
1.5
0.5
0.5
4. 4 ANAl YTI CAl CONDI TI ONS 175
1.5 2
x
t
2.5
Fl gure4. 8 Inequalityconstral nedprobl em.
Mi ni rni ze j{Xx2): -xl x2
Subj ect 10: h
l
(XI x2): 20 XI + 15X2 - 30 = 0
gl (XI x: (xI / 4) + ~ I S 0
o S XI S 3; 0 S X2 S 3
3
(4.9)
(4.10)
(4.11)
(4.12)
4
The graphi cal sol uti onof Secti on4.3.3suggests that thei nequal i tyconstrai nt will be
acti veat thesol uti on. I f is thecase thenFi gure 4. 8wi l l once agai nrepresent e
graphi cal sol uti ontothis secti on so. This wi l l be vcri fi edinthcncxt scclI on.
4. 4 ANAL YTI CAL CONDI TI ONS
Analytical condi ti ons rcfer to n essaryand suffi ci ent condi ti ons twill permi t
the recogni ti on of the sol uti on to the opti mal desi gn probl cm. Thc condi ti ons
devel oped hcre empower the numeri cal techni ques to fol l ow later. They are
i ntroduced in same sequence as in e previ ous secti on. l nstead of fonnal
mathemati cal details the condi ti ons are establ i shed less fonnal l y f romthe
geometri cal descri pti on of probl emandl or rough i ni ti ve reasoni ng. Fonnal
dcvel opment of thc anal yti cal condi ti ons can be f ound in Refercnces 6-8. For
establ i shi ng the condi ti ons it is expected that the sol uti on is in the i nteri or of e
feasi bl regi onand thereis onl y one mi ni mum.
4. 4. 1 Unconst r ai ned Pr obl em
The probl emused for i l l ustrati onis
Mi ni mi ze J(x.. X2): (XI - 1)2 + (X2- 1)2- XI X2 (4.38)
(4.37) OSx
I
S3; OSX2S 3
Fi gurc 4. 6 provi ded a contour pl ot of the probl em. Fi gure 4.9 provi des a
three-di mensi onal pl ot of thesame probl em. It is a mesh pl ot. The commands tocreate
Fi gure 4. 9areavai l abl einFl g4_9. m. The fi gureneeds tobe rotatedi ntemcti vcl y to
appear as illus ted. A tangent pl aneis dmwn mi ni mumfor emphasi s. Fi gure
4. 9wil1be us dtoi denti fy theprope es of the functi onJ(xl xz) at themi ni mum.
The mi ni mumis i denti ti ed by a supe cript asteri sk ( Xor lxi X2J
T
). Studyi ng
Fi gure 4. 9 the functi onhas a mi ni mumval ueof - 2 whi l e xj = 2 and x2= 2. If e
val ues of X1 andl or X2 we tochange even by a sl i ght amount f rom opti mal val ue
inany di recti on theval ueof the functi onwil1certai nl yi ncreasesi nceX' is thel owest
poi nt of the concave surface rep senti ng e functi on f Reprcsenti ng E
di spl acemcnt f romtheopt i mumval ues of thevari abl es as dX and thc changc inthe
functi on val uc f romthe opt i mumas llf f romdi rect observati on it is cl ear that the
opti mal sol uti onmust be a poi nt at satisfies
4f>0 for al1dX
Fl rst- Order Condl tl ons: The same i dca can bc appl i ed in the limit that is for
i nfi ni tesi mal di spl acement dx
1
and dx
2
about X . The functi on itself can be
approxi mated by a pl anetangent tothefuncti onat thesol uti on( shown inFi re4. 9)
(consi dcr is afi rst-ordr Tayl or seri es expansi on). Movi ng to any poi nt in the
pl ane f romtheopt i mum(see Fi gure 4.9) will not change theval uc of the functi on
E foredf = O. Movi ng away f romopt i mumi mpl i es dx
1
and dx
2
are not zero.
Rewri ti ng Equati on(4.20) intcnns of X1 and X2
df = 1+ =0
df = ][:J=O
(4.42)
4.4 ANAl YTI CAL CONDI T10NS 1 n
10
4
B
6
F
H
X
2
- 2
4
X
2
F1gure4.9 Three-di mensi onal plot 01Fi gure4.6.

Si nce this shoul dhol dfor al1poi nts in epl ane dx
1
1:- 0 and dx
2
1:- O. Therefore
df - f l - f l
dX
1
- v dX2 - v
or thegradi ent of f at theopt i mummust be zero. at is
V f x ~ x ; ) = 0
(4.43)
Equati on (4.43) expresses the necessary condi ti on or firstorder condi ti ons (FOC)
for unconstrai nedopti mi zati on. It is termed thefrrstorder condi ti onbecause Equati on
(4.43) uscs egradi ent or thefirst deri vati ve. Equati on (4.43) is usedto i denti fy thc
possi bl esol uti ons totheopti mi zati onprobl em. I f functi onwere toflipover so at
the same functi on wc to maxi mi ze the val ue of e functi on the sol uti on for the
vari abl s wi l l be at the same val ueof the desi gn vari abl c. It is cl ear that Equati on
(4.43) appl i est he maxi mumprobl emalso. Equati on (4.43) by itself will not
determi ne themi ni mumval ue of thefuncti on. Addi ti onal consi derati ons a necessary
toensure at thesol uti onestabl i shedby fi rst-order condi ti ons is opti mal in is
casc a mi ni mum. For a general unconstrai nedprobl cm the necessary condi ti ons of
Equati on(4.43) can be statedas
Vf (X') =0
(4.44)
whi ch is pri mari l y a vector expressi onof the rel ati on in Equati on (4.43). Equati on
(4.44) is usedtoestabl i shtheval ueof thedesi gnvari abl es X' bothanal yti cal l y and
numeri cal l y.
Second- Dr der Condi ti ons: The second-order condi ti ons ( SOC) are usual1y
regardedas suffi ci ent condi ti ons. It can be i nfe 'ed at thesecondi ti ons wi l l i nvol ve
second deri vati ves of th functi on. The SOC is often obtai ned through theTayl or
expansi on of thefuncti ontosccond order. If X is sol uti onand d.Xrepresents e
changeof vari abl cs f rom opti mal val uewhi ch will yi el da change d i f then
+6X) ) = ( X + t a h r
(4.45)
Thi s is sirnilar to Equati on (4.35) except eexpansi on is about c sol uti on. 6f
must be grcater thanzero. Empl oyi ng thenecessary condi ti ons (4.44) thefirst term
on theri ght-handsi deof Equati on 4.45 is zero. Thi s I cavcs efol l owi ngi nequal i ty
a f =j a T H
(4.46)
wher eH( X) is theHessi an matri x (thema ixof second deri vati ves) of thefuncti on
f at thc possi bl opt i mumval ueX For erel ati ons inEquati on (4.46) to hol d the
matri x H( X must be posi ti ve definite. There are three ways to stabl i sh the H is
posi ti vedefinite.
(i) For all possi bl e.6X d.X
T
H( X' ). 6X> O.
(ii) The ei genval ues of H( X' ) areall posi ti vc.
(i u) The determi nants of ] l ower orders (submatri ces) of H(X' ) that i ncl udeth
mai n di agonal areaIl posi ti ve.
Of t he reeonl y (ii) and (iii) can be practi cal l yappl i ed whi ch is illustratedbel ow.
Exampl e
Mi ni rni ze j(XIx2): (XI - 1)2+ (X2- 1)2- XI X2 (4.38)
(4.37)
F DC
=2( x
l
- 2) - x
2
=O
(4.47a)
4.4 ANAL YTI CAL CONDI TI ONS 179
=2 - 1) - x
1
=
(4.47b)
EQuati ons (4.47a) and (4.47b) represent a pai r ofl i nearequati ons whi ch can be sol ved
as- X1=2I 2=2. The val ue of the t-uncti onf i s- 2. So far onl y the necessary
condi ti ons have been satisfied. The val ues above can al so refer to a poi nt where the
ncti onis a maxi mum or where thereis a saddl epoi nt .
SDC: For is probl em[seeEquati on (4.26)]
b
l
r

-
H

Is it posi ti vedefi ni te?
(i) Not possi bl e10test all d.X
(ii) To ei genvaues of H
= - 3)-- ))
- 1 2- l
'he ei genval ues ar = = 3 and thematri x is posi ti vedefinite.
(ui) To cal cuate deterrni nants of all order al incJ ude the mai n di agonal and
incJ ude theel ement in efrrst r owand first col umn
121>0
n
u
q
J

-
-
2
'a
The matri x is posi ti vedefinite.

Sec_ 4_ 4- 1. mprovi des econfi rmati onof thenumeri cal val ues for this exampl e
usi ngSymbol i c Tool box and basi c MATLAB commands.
4. 4. 2 Equal i ty Const r ai ned Pr obl em
Thc probl em:
Mi ni rni ze f(XI x2): -XI X2
(4.36)

(4.39)

(4.37)
Fi gure 4.7i l l ustrated e graphi caJ sol uti on. It w noti ced at at thesol uti on the
gradi ent of obj ecti vefuncti on dthc gradi ent of theconstrai nt we paral l el and
opposi tel y di rected. Exami ni ng other feasi bl e poi nts in Fi st I m4. 7( on the dashed
curve) it can be ascert nedthat thespeci al geometri cal rel ati onshi pis onl y possi bl c
at thesol uti on. At thc sol uti ona proporti onal rel ati onshi pcxi sts between thegradi ents
at the sol uti on. Usi ng thc constant of proporti onal i ty AI (a posi ti ve val ue) thc
rel ati onshi pbetween thegradi cnts can be expressedas
= I Vhl or Vf + =0
(4.48)
Equati on (4.48) is usual l y obtai ned in a more formal way usi ng thc met hod of
Lagrange mul ti pl i ers.
Met hod of Lagr ange: Inlhis met hod theprobl emis transformedby i ntroduci ng
an augmented functi on cal l edtheLagrangi an. as theobj ecti vefuncti onsubj ect tothe
same ual i ty cons nts. The Lagrangi an is defi ned as the sumof the ori gi nal
obj ecti vc functi onand a l i near combi nati on of theconstrai nts. lecocffi ci cnts of lhis
l i ncar combi nati on known Lagrange mul t liers. Wi t h rcfcrence 10 thc
exampl c in is sccti on
Mi ni mi ze F(x2AI ) =f(xl xu + AI h
l
(XI x2) (4.49)
be compl ete probl emis devel oped as
Mi ni mi ze F(XI x2AI) = - XI X2 +1(./ 4+4- 1) (4.50)
Subj ecl to: h
l
(XI x2): 4 / 4 +4 - 1
(4.39)
O::;;XI ::;;3; 0::;;x2::;;3
(4.37)
Is esol uti ontothe l sformedprobl cm[Equal i ons (4.50) (4.39) (4.37)] thesame
as thesol uti ontotheori gi nal probl em[Equati ons (436)(4.39)(437)]?I f thedesi gn
is f i si bl e en most defi ni tel y yes. For feasi bl e desi gns hl (x" X2) = 0 d
o tive functi ons in Equati ons (4.36) and (4.50) are the same. If desi gn is nol
feasi bl e thcnby defi ni ti onthereis no sol uti onanyway.
be F OC areobtai nedby consi deri ngF( x" X21) as unconstrai nedmcti ol l in
evari abl cs XI X2l' This provi des reerel ati ons tosol vefor xj xi
dF i)f ah
=
+ . - ! . =o
ax

i)x
l
' .'. ax
aF af . " ahl
= + =0
aA2 ax
2
" '. aX
2
aF
=h. =O
(4.51)
4.4 ANAL YTI CAL CONOI TI ONS 181
Equati ons (4.51) express eFOCor n S y condi ti ons for qual i tyconstrai ned
probl emint wo vari abl es. Thc last equati oninthc above set is theCOns int equati on.
is ensures the sol uti on is feasi bl e. The first two equati ons can be assembl ed in
vcctor f ormtoyi cl dthesame i nformati onexpressedby Equati on (4.48) whi ch was
ob nedgraphi cal l y. leI eft-handexp ssi ons in efirst l wo equati ons abovc are
thegradi ent of theLagrangi an functi on:
VF = +I Vh. =0
Appl yi ng Equati ons (4.51) tothc exampl e of is secti on:
a F I X1
? =- x. + =0
oX
1
L
aF
? =- x. + 2 x. =o
".:__ - -""2 ' "'""ol
A
2
VA2
h 1 4 4 - l =o
(4.52)
Equati ons (4.52) present equati ons in vari abl es whi ch shoul ddetermi ne
the val ues for xj. xi j. Note Equati ons (4.52) onl y defi ne the necessary
condi ti ons whi ch means thesol uti oncoul d be a maxi mumalso.
Thcreis one probl emwi threspect totheset of Equati ons (4.52). Th equati ons are
a nonl i ncar set . Most prerequi si te courses on numcri cal methods do not attempt to
sol v anonl i near systemof equati ons. Usual l y. l hcyonl y handl e a si ngl eone through
Newt on- Raphson or the bi sccti on method. In fact NL P or desi gn opti mi zati on is
pri mari l y about tcchni ques for sol vi ng a systemof nonl i near uati onsalbeit of
spi fi c forms. Thismeans Equati ons (4.52) cannot be sol vcduntil we advancc rther
in is course. Fortunatel y havi ng a tool Ii ke MA ABobvi ates this di ffi cul ty and
provi des a s ngj usti fi cati onof usi nga numeri cal/ symbol i c tool for supporti ng
devel opment of thecourse.
2
MA ABCode: In MATLAB. there are t wo ways of sol vi ng Equati ons (4.52) usi ng
symbol i c support functi ons or usi ngthc numeri cal support functi ons. Both procedurcs
havc l i mi tati ons when appl i cdtohi ghl y nonl i near functi ons. The symbol i c functi onis
sol veand numeri cal functi oni sfsol ve. The numeri cal tcchni queis an iterativeone
and requi res you tochoose an initial gucss tostart theprocedure. Qui te oftensevcral
di fferent guesses may bc qui red to fi nd c sol uti on. Sol uti ons of systcms of
equati ons aref undamental totherest of ebook. The fol l owi ngcode is i ncl uded
a hands-on exerci se. It is avaiI abl e as Sec4_ 4_2. m. It requi res eqns4_ 4_2. mto
execute ol vecommand. Thc co of efilcs arelistcdbcl ow
111C nrcolher 10018besldcsMATLADsuch MUl hcnd Id ccl.
182 NONl l NEARPROGRAMMI NG
S 4_4_2. m
Necessar y/ Suf f i ci ent condi t i ons f or
Equal i t y const r ai ned pr obl em
Opt i mi zat i on wi t h MATLAB Sect i on 4. 4. 2

Mi ni mi ze f ( X1x2) - x1
- - - - - - -
symbol i c pr ocedur e
- - " - - '
def i ne symbol i c var i abl es
f or mat compact .
syms x1 x2 l am1 h1 F
def i ne F
F - x1*x2 + l am1*( X1*X1/ 4 + x2*x2 1) ;
h1 x1*x1/ 4 +x2*x2 - 1;
he gr adi ent of F
gr ad1 di f f ( FX1) ;
gr ad2 = di f f ( FX2) ;
opt i mal val ues
sa i sf act i on of necessar y condi t i ons
[ l ams1 xs1 xs2] sOl ve( gr ad1gr ad2h1' X1x2l am1' ) ;
t he sol ut i on i s r et ur ned as a vect or of
t he t hr ee unknowns i n case of mul t i pl e sol ut i ons
l ams1 i s t he sol ut i on vect or f or l am1 et c.
; mOR mT : t h e r esul t s ar e sor t ed habet i l y
f pr i nt f i s used t o pr i nt a st r i ng i n t he
command wi ndow
di sp i s used t o pr i nt val ues of mat r i x
f - xs1. *xs2;
f pr i nt f ( ' The sol ut i on ( x1* x2 l am* f * ) n' )
di sp( doubl e( [ xs1 xs2 l ams1 f ] ) )
- - - - - - - - - -
% Numer i cal pr ocedur e
- - - -
sol ut i on t o non- l i near syst emusi ng f sol ve
see hel p f sol ve
t he unknowns have t o be def i ned as a vect or

t he f unct i ons have t o be set up i n an m- f i l e
def i ne i ni t i al val ues
xi ni t =[ l 1 0. 5] ' ; i ni t i al guess f or x1 x2 1am1
t he equat i ons t o be sol ved ar e avai l abl e i n
eqns4_4_2. m
xf i na1 f so1ve( ' eqns4_4_2' xi ni t ) ;
f pr i nt f ( ' The numer i cal sol ut i on (x1* x2* l am1*) : n' )
di sp( xf i na1) ;
eqns4_ 4_2. m
f unct i on r et eqns4_4_2( x)
x i s a vect or
x( l ) x1 x( 2) x2 x( 3) l am1
r et =[ ( - x( 2) + 0. 5*x( 1) *x( 3) )
( - x( l ) + 2*x( 2) x( 3) )
( 0. 25*x( 1) *x( 1) + x( 2) *x( 2) - 1) ] ;
Out put InMATLABCommand Wl ndow
The sol ut i on ( x1* x2* l am1* f *) :
1. 4142 0. 7071 1. 0000 - 1. 0000
- 1. 4142 - 0. 7071 1. 0000 - 1. 0000
- 1. 4142 0. 7071 - 1. 0000 1. 0000
1. 4142 - 0. 7071 - 1. 0000 1. 0000
Opt i mi zat i on t er mi nat ed successf ul 1y:
Rel at i ve f unct i on va1ue changi ng by 1ess t han
OPTI ONS. Tol F un
The numer i ca1 sol ut i on ( x1* x2* l am*) :
1. 4141
0. 7071
0. 9997
The symbol i c computati ongenerates four sol uti ons. Onl y thefirst one is val i dfor this
probl em. Thi s is deci dedby esi deconstrai nts expressedby Equati on(4.37). Thi s is
i I I ustrati onof thea priori manner by whi ch thesi deconstrai nts affect theprobl em.
On theother hand. enumeri cal techni ques provi deonl y one sol uti on10theprobl em.
l I s is a functi onof the initial guess. General l y. numeri cal t hni ques will del i ver
sol uti ons cl osest tolhepoi nt they startedfrom. The sol uti onis
x; =1. 4141; x; =0. 7071; =1. 0
'he soh ons for XI dX2C be veri fi edf romthegraphi cal sol uti oninFi gure4.7.
The sol uti ons canal sobe veri fi edthroughthehand-hel dcal cul ator. The val ueand si gn
for are usual l yi mmateri al for establ i shi ngtheopti mum. For a wel l -posedprobl em
it shoul dbe posi ti vesi ncc inc asi ngtheconstrai nt val uc is useful onl y if wc aretryi ng
toi denti fy a l ower val uefor emi ni mum. This forces h and f tomovc inopposi te
184 NONUNEAR PROGRAMMI NG
di recti ons (gradi ents). I ncreasi ngconstrai nt val uecan be associ atedwi thenl argi ngthe
feasi bl edomai n whi ch may yi el da better desi gn.
Lagr ange Mul t l l ers: The Lagrange mul ti pl i er met hod is anl egant formul ati on
toobtai nthesol uti ontoa constrai nedprobl em. Inovervi ew it seems strange at we
have toi ntroducean addi ti onal unknown (1) tosol vetheconstrai nedprobl em. Thi s
vi ol ates e conventi onal rul e for NL P the f ewer the vari abl es the better the
chances of obtai ni ng e sol uti on. As i ndi cated in the di scussi on earl i er E
Lagrangi an al l ows the ansformati onof a constrai nedprobl emi ntoan unconstrai ned
probl em. The Lagrange mul ti pl i r al so has a physi cal si gni fi cance. At thesol uti onit
expresses e ratio of change in the obj ecti ve functi on to the change in
constrai nt val ue. To illustrate is consi der:
F =f +I hl
dF=df +I dl1
d F . d F . dF
:: aA.
dX
1
_. . . dX2- ' 2 d - ..
At the sol uti on theF OC deems that dF = 0 (whi ch can al so be seen in the above
detai l edexpansi on). Hence
L
1 - dh 611
The above dependence do s not affect theestabl i shment of theopti mal desi gn. It
does hav an i mportant rol e in the di scussi on of desi gn sensi ti vi ty in NL P
probl ems.
Gener al Equal l ty Constral necl Pr obl em: Rememberi ng n - 1>0
Mi ni mi ze f ( X) J'I (4.5)
(4.6) Subj ect [lI (X)] = 0
X
10w
SXSXup (4.8)
The augment ed probl emwi ththeLagrangi an:
Mi ni m F( X' ) =f ( X) +L (X)
k=l
=f ( X) +A' h;
=f ( +l Th (4.53)
4.4 ANALYTI CAl CONDI TI ONS 185
Subj ect to: [h(X)] = 0
(4.6)
X
10w
SXSXup
(4.8)
10Equati on(4.53) threeequi val ent representati ons for theLagrangi an areshown. The
FOCar e
dF h" (4.54)
=+) A." =0; i =1 2
aJEi axi z k h'
Subj ect: (X)]=o
X
10w

(4)
(4.8)
Equati ons (4.54) and (4) provi de the11+ 1 rel ati ons todetermi ne n+ 1 unknowns
x . Equati on(4. 8)i s used after thesol uti oni sobtai ned-i f necessal y. Equati on
(4.54) is al soexpressedas
n
u
-
-
-
h
H V

H
V

'
n
n
v
r
L
+

J

V

E
E
V

(4.55)
Second-Order Condi ti ons: At sol uti on determi ned by F OCthe functi on
shoul d i ncreasefor changes in6X. Changes inX arenot arbitrary. They have to
satisfy el i neari zedequal i tycons aint at thesol uti on. 1t is takenfor gr d atthe
suffi ci ent condi ti ons Me usual l y appl i edin a smal l nei ghborhood of the opti mum.
Al sochanges arecontempl atedonl y wi threSPECt to X and not wi threspect to thE
Lagrangemul ti pl i er- TKLagrangemethodi s o b n cal l edthemethodqf undetermi ned
coefficient. i ndi cati ngit is oot a vari abl e. 1nthe aly c deri va6o nof theFOCf or
:pp emm l Lagrangi a I was c on u n c o ns i B 0 r r n g f m mn t he
previ ouss c0 n( c on strai nedmin1iza t i on ) SOCc beexpec dtosatisfythe
fol l owi ngrel ati ons:
=F(X* +)-F '(X.) ) =VF( '(X' ) d T [ v z 2F(X' ( X' ) )] O
VhTAX=O
Intheabove [ V2F( Xl i s theHessi an of theLagrangi anwi threspect tothedeSI gn
vari abl es onl y eval uatdat thesol uti on. Al sotheF 0C mqui mt hat VF( X) =0. Wi t h
referencetot wo vari abl es and one constrai nt:
t:.. F = +2)+
M = i + 2 t L)+
(4.57)
t:..x
l
dh/ ox
2
(4.58)
t:..x
2
dh1/ dx
1
Substi tuteEquati on(4.58) inEquati on(4.57) and theSOC requi res that theexpressi on
in br amust be posi ti ve. deri vati ves in Equati ons (4.57) d (4.58)
eval uatedat them i mum.
Appl yi ng thesecond-order condi ti on to theexampl e of is secti on is 1 ft
exerci se for the ader. Compar d to e SOC for the unconstrai ned mi ni mi zati on
probl em Equati ons (4.57) and (4.58) not syappl yespeci al l ythesubsti tuti on
of Equati on(4.58). Fr oma practi cal perspecti v SOCis not i mposed for uality
constrai ned probl ems. It is left to the dcsi gncr to ensure by other means e
sol uti onis a mi ni mumsol uti on.
4. 4. 3 I nequal l ty Const r ai ned Opt l mi zat i on
Thepr obl em
Mi ni mi ze f(XI Xz}: XI X2 (4.36)
Subj ect to: gl (XI x2): 20 XI + 15X2- 30 S 0 (4.40)
S O (4.41)
OSXI S3; OSX2S3 (4.37)
The number of vari abl es (n= 2) dthenumber of i nequal i tyconstrints ( m= 2) do
not depend on each other. In fact. an i nequal i ty consai ned probl emin a si ngl e
vari abl e can be useful l y desi gned and sol ved. Secti on 4.4.2 sol ved the uality
constrai ned probl em. If the above probl emcan be nsf ormed to an equi val ent
cqual i ty constrai ned probl em. en we have f ound the sol uti on. 1e standard
1sformati onreq res a sl ack vari abl ezj for eachi nequal i tycons nt g
j
Unl i keLP
probl ems thesl ack vari abl efor NL P is not restri ctedinsi gn. berefore thesquare of
the newvari abl eis added tothel eft-handsi deof thecorr pondi ngcons ai nt . Thl s
adds a posi ti veval uetotheleft-h dsi detobri ngthecons int up tozero. Of course
a zeroval uewi l l be added if theconstrllint is alrelldy zero.
1 ; 1
Transformatl ont o s n Equal i tyConstrai nedProbl em
Mi ni mi ze j(XI X2): -XI X2
Subj ectl o: gl (XI ' Xz}+ zi: 20xI + - 30 + zi = 0
g2(X. X2)
OSXI S3; OSX2S 3
(4.36)
(4.59)
(4.60)
(4.37)
There arefour vari abl es (XI.X2. ZI. Z2) and twoequal i ty cOllstraints. It is a val i dequa1ity
cons i ned probl em. The Lagrange mul ti pl i er m thod can be appl i ed to this
mau- - Tod ngui sh emul ti p oci atedwi i ity cons
esymbol is used. This is s ct1y for cI ty.
Method of Lsgrsnge: The augmentcd functi onor theLagrangi anis:
Mi ni mi ze

=f (x
l
.X
2
) + s
1
[gI(X
1
x
2
) + zil + s2[C2(XI' ) (4.61)
If the Lagrangi an is consi dered an unconstrai ned obj ecti ve functi on. the FOC
(necessary condi ti ons) are
dF df . / l ogl . / l dg2
= - 4 1 +s
=0
axl a x l a X1 2 a x l
dF of . / l ogl . / l
= + + s2 =0
dX
2
dX2' I"' . dx
2
.
dF _ n
=2sz=0
a Z
1
dF . n
? =2s. z. =0
a Z
2
dF ?
=g. +zf =O
dPI u l
iJ F ?

ds
2
U "
(4.62a)
(4. 62b)
(4.62c)
(4.62d)
(4.62e)
(4.62t)
Equati ons (4.62e) and (4. 62t) are ual i tyconstrai nts. Equati onset (4.62) provi des six
uati onstosol vef orx1xEz; Z5.91. . By si mpl e recombi nati on Equ ons(4. 62c)
to(4. 620 can be col l apsedtot wo equati ons whi l e thesl ack vari abl es ZI and Z2can be
el i mi natedfromtlzeprobl em.
Fi rst mul ti pl y Equati on (4. 62c) by ZI Repl acez t by -81 f romEquati on (4.62e).
Dr op thenegati vesi gnas wel l t he coeffi ci ent 2 toobtai n
18. =0
13282=0
(4.63a)
(4.63b)
Equati on (4. 63b) is obtai ned by carryi ngout si mi l ar mani pul ati ons wi thEquati ons
(4. 62d) and (4. 620. The F OC can be restatedas
dF df . n d8. . n d82
=;J + 132 =0
dx. dx. r1dx. '
dF dr d8I d8
=: : L+ 132=0
dX
2
dX
2
r1dx2
1318. =0
13282=0
(4.62a)
(4. 62b)
(4.63a)
(4. 63b)
These four equati ons have tobe sol vedfor xj xi j s 2 Note tzi Z2areoot bei ng
det ermi ncd- whi ch suggests eycan be di scardedf rom probl cma1togc er.
It woul d be useful topretcndz's ncvcr exi stcdinthefirst pl ace.
Equati ons (4.63) l ayout a dcfi ni tefeaturefor a nontri vi al sol uti on: ci ther sj is zero
(and8/ 0) or 8/ is zero ( ) . Si nce si mul taneous cquati ons bei ngsol ved e
condi ti ons on e mul ti pl i ers and constrai nts must be sati sfi ed si mul taneousl y. For
Equati ons (4.63) is sl as i nto efol l owi ngfour cases. Thc i nformati onon 8 in
brackets is tocmphasi ze an accompanyi ng consequence.
Case a: s. = 0 [8. < 0]; 132= 0 [82< OJ
Case b: =0[ 8. <0] ; * 0[82= 0]
Casec: 13
1
* 0 [8. =0]; 13
2
=0 [ 82<0)
C d: I 0[8. =0); * 0[82= 0]
(4.64)
In Equati on (4.64) if sj * 0 (or correspondi ng 8/ = 0) then the correspondi ng
consaI nt is an equal i ty. Intheprcvi ous sccti ona si mpl e reasoni ngwas usedshow
c si gnof e m tiplier must be posi ti ve( >0) for a wel l -formul atedprobl em.
Whi lethesi gnof themul ti pl i er was i gno d for eequal i tyconstrai nedprobl em it
is i ncl udedas part of theF OC for i nequal i tyconstrai nedprobl em. Bef ore restati ng
l
4.4 ANAL YT' CAL CONOtnONS 189
Equati on(4.64) theLagrangi anis reformul atedwi thout thesl ack vari abl es (weare
goi ngtopretendZ never exi sted)
Mi ni mi ze F(x. x
2
.s.13
2
) = .f{x. x
2
) + s.[g.(x..x2)] + s
2
[ 8T . X
2
)] (4. 65)
whi ch is thesame formul ati oninEquati on (4.53). The sl ack vari abl ewas i ntroduced
to provi de transformati on to an equ ity cons int . It is al so evi dent e
cons cti onof Lagrangi anfuncti onis i nsensi ti veto typeof constrai nt Si nce
emul ti pl i ers tiedtothei nequal i tyconstrai nt arerequi redtobc posi ti ve whi l e those
correspondi ngtotheequal i tyconstrai nts arenot this book wi l l conti nuetodi sti ngui sh
between mul ti pl i ers. This will servetoenforcc clarity of presentati on. The FOC
for theprobl em:
e)F dr d8 I d8
= L 4 1 4 4 H=o
dx dx ' I"" 'OXt . I""Zdx.
dF df . n dg. . n dg
2
= + + 132 =0
dXz dx
2
' 1""' d.Xz . I""Z d.Xz
Case a: =0[gl <0]; 2=0 [g2 < 0]
Case b: 13 = 0 [8. < 0]; >0 [g2 = 0]
Case c: s. > 0 [g. = 0]; s2= 0 [g2 < 0]
Case d: s. > 0 [8. = 0]; >0 [82= 0]
(4.62a)
(4. 62b)
(4.64)
Equati on set (4.62) and any si ngl ecaseinEquati on (4.64) provi des four equati ons to
sol v for thc four unknowns of the probl em. Al l four scts must be sol vedfor the
sol uti on. The best desi gnis deci dcdby scanni ngtheseveral sol uti ons.
Thc si gn of the mul ti pl i er in the sol uti on is not a sujJi ci ent condi ti on for the
ual i tyconstrai nedprobl emThc val ue is un portant for opti mi zati onbut may be
rel evant for scnsi ti vi tyanal ysi s. General l y a posi ti veval ueof mul ti pl i er i ndi catcs
the sol uti on is not a l ocal maxi mum. Formal l y veri fyi ng a mi ni mumsol uti on
req s consi derati on of the sccond dcri vati ve of thc Lagrangi an. In practi cal
si tuati ons. if thc probl emis wel l defi ned thc posi ti veval ue of c mul ti pl i er usual l y
suggcsts a mi ni mumsol uti on. This is used cxtensi vcl y in thc book to i denti fy the
opt i mumsol uti on.
Sol utl on of the Exampl e: Thc Lagrangi anfor theprobl emis defi ncdas
F(x.x
2
.s.s
2
) =- x. .Xz+ s(2Ck. + 15Xz- 30)
TheFOCar e
190 NONLI NEARPROGRAMMI NG
dF
Z; =42+2ORI +05h=0
(4.66a)
dF
55=41+15P l +2h=0
(4.66b)
s1
(2Ox
I
+ 15
Xz
- 30) =0
s2
(0 +4- 1) =0
(4.66c)
(4.66d)
The si deconstrai nts
OS X
1
S3; O S
x.z
S 3 (4.37)
Case a:
sl = 0;
s2= 0: The sol uti onis ivial and by i nspecti onof Equati ons (4.66n)
and (4.66b).
X1=0; x
2
=0; 1=0; gJ =30;
g2= (4.67)
The i nequa1itycon ts aresatisfied. The si decon aints are isfied. The val ues
inEquati on(4.67) rep sent a possi bl esol uti on.
The sol uti ons for theother cases ' c obtai nedusi ngMA11. AB lecode is avai l abl e
inSec4_4_3
m. For Cases b and c theappropri atemul ti pl i er is set tozeroand the

resul ti ngthrEeEquati ons inthreeunbEowns aresol ved. For casE d thecompl ete set
of four equati ons infour unknowns is sol ved. cased is al soSOl vednumeri cal l y. le
MATI . AB code will a1soruncode Fl g4_8. mwhi ch contai ns th commands todraw e
figure. The output f romtheCommand wi ndowis patchedbe10w.
The sol ut i on *** Case a ***( x1* x2* f *
91
9
2)
o 0 0 30 1
The sol ut i on
**Case (b) **
( x1* x2
b2* f * 91 92) :
1. 4142 0. 7071 1. 0000 - 1. 0000 8. 8909 0
- 1. 4142 - 0. 7071 1. 0000 - 1. 0000 6 . 8909 0
- 1. 4142 0. 7071 - 1. 0000 1. 0000 - 47. 6777 0
1. 4142 - 0. 7071 - 1. 0000 1. 0000 - 12. 3223
0
The sol ut i on
*
Case ( c) *** ( x1* X2* b1* f *
91 92) :
o . 7 5 0 0 1 . o o oo o . 0 5 0 0 - o . 7 5 0 0 o
o . 14o 6
The sol ut i on
***Case( d}***{xl * x2* bl * b2* f * gl g2) :
1. 8150 0. 4200
0. 0426 1. 4007 0. 7624 0 0
0. 8151 0. 9132
0. 0439 0. 0856 - 0. 7443 0 - 0. 0
Maxi mumnu
of f unct i on eval uat i ons

exceeded;
l ncr ease opt i ons. MaxFu
nEval s
Opt i mi zer i s st uck at a mi ni mumt hat i s not a r oot
Tr y a9ai n wi t h a new
st ar t i n9 guess
The numer i cal sol ut i on (x1
X2
b1* b2*) :
0. 8187 0. 9085 0. 0435
0. 0913
4.4 ANALYTI CALCONDI Tl ONS 191
The sol uti onfor Case a was di scussedearlier and is conrtrmed above.
Case b hns four sol uti ons. The first one is unacceptabl ebecause constrni nt gl is not
satisfied. The second sol uti onis feasi bl ens far as thefuncti ona1constrai nts g and g2
are concemed but they do not satisfy the si de constrai nts. The third and fourth
sol uti ons are unacceptabl e for the same reason. Thus. Case b is not nn opti mnl
sol uti on.
Cnse c is unacceptabl ebecause constrai nt g2 is inviolation.
Cnse d hns t wo sol uti ons thefirst of whi ch is not acceptabl efor several rensons. The
secondsol uti onsntisfies nll of thereq rements
It sntisfies theconstrai nts. 80th constmi nts nctive.

The mul ti pl i ers eposi ti ve(maybe a suffi ci ent condi ti on).
I t satisfies thesi deconstrai nts.
lesol uti onis
x
1
=0. 8151; X2= 0. 9132;
=0. 0439;
=0. 0856;
1 =0. 7443;
gl =
0;
g2 =
0
This is nl most confi rmed by numeri cnl sol uti onwhi ch nppears tohnve a probl em
wi convergen . It is al most at thesol uti on. 1t must be nol edherethat theMATLAD
functi onused tosol ve probl emis part of thestandardpackage. The functi ons in
the Opti mi zati on Tool box will certain1y do better. Neverthel ess. the i ncompl ete
sol uti onis reportedbel owbut will be i gnoreddue to esymbol i c sol uti onavai l abl e
inCase d.
x=0. 8187; x
2
=0. 9085;
s
=O. 35;
=0. 0913
?
?
?
E
n
s
There arenowtwo candi dates for esol uti on: etrivinl sol uti oninCase a and the
sol uti oninCnse d. The sol uti oninCns dis favoredns it hns a l ower obj ecti vefuncti on
va1ue. Al l of the cnses above can be expl ored wi spect to Fi gure 4.8. It is an
excel l ent facilitator for comprehendi ng thecases and sol uti on.
4. 4. 4 General Opti ml zatl on Pr obl em
'he genera1opti mi zati onprobl emisdescri bedintheset of Equati ons (4.1) (4.8). le
speci fi c probl eminthis chapter dfi nedinSecti on4.3is
Mi ni mi ze j(Xl ) : 'XX2
(4.9)
Subj ect to: h1(Xl
x
V: 20 x + 15X2- 30 = 0 (4.10)
gl (XxV: <4/ 4) + ~ I S 0 (4.11 )
OSXl S3; OSX2S3 (4.12)
i
1
1
1
1
1
4
I
1
1
J
j
q
193
(4.3)
4. 4 ANAL YTI CAL CONOI TI ONS
;(XIoX2. . . xJ S O. j = 1.2. . . . m
NONUNEAR PROGRAMMI NG
ZAiil ;;355
192
(4.4) XI j = 1.2. . . . n
The Lagrangi an:
; Mu/ i od: The probl cmsfiy mi ni mi zi ng thc
Mi ni mi ze
F(x
1
XnA. ... /. sI' . . . . srn) = F(XJ
X
2. A. ..sU= - X .X2+1(20x. + 15x2- 30) + s. (0. 25x? +xi - 1)
Mi ni mi ze
Subj ect 10:
(4.70)
Therearen + 1 + m unk.nowns. The sarnenumber of cquati ons arer . ui redtosol ve
eprob1em. ese "eprovi dedby theF OC or eKuhn- Tucker condi ti ons:
f(x . . X
n
) +h ++ +s. g+ +srnCm
"1(XJ
X
2): 20xI + 15x2- 30=0
OSX2S 3
OSXI S3;
(4.71)
i =1. 2. . . . . n
11cquati ons areobtai nedas
dF df . dh" . dhl . n dg. . n dg
= +A. - : ; + +PI - - ++sm-:
m
=0;
dx dx" - ' dx - --'dx - r ax - r m ax
(4.68a)
(4. 68b)
aF
Z; =- X2+2 I +O.5s =0
aF
=- x. + x
2
=o
TheFOC
l uati ons ar obtai neddi rcctl y ruugh ecqu ityconstrai nts
aF
- =2 1+
(4.72)
m cquati ons areapp1 i ed through 2
m
cases. Thi s i mpl i es ereare2
m
possi b1e
sol uti ons. Thesc sol uti ons must i ncl udeEquati ons (4.71) and (4.72). Each cas sets the
mul ti pl i er or correspondi ngi nequal i ty c 0 n stmai n t to ze Ifthe ml t
set to zero. en e correspondi ng cons int must be feasi b1e for an acccptabl e
sol uti on. If e constrai nt is set to zero (acti ve constrai nt). then the correspondi ng
mul ti pl i er must be posi ti vefor a mi ni mum. Wi t h is inmi nd cm equati ons canb
expressedas
k = 1.2.. ...1
. X
n
) =0; (4.68c)
(4.69a)
(4. 69b)
32;: ; ; z z u y f a ui rthesol uuo system
i 23222! 1223;
gl <0
g. =0
s =0;
s>O;
Case a:
Case b:
(4.73)
If con ons in Equati ons (4.73) are not met. e di gn is not acceptab1e. In
i mpl ementi ng Equati on (4.73). for each case a si mu1taneous to of m va1ues and
cqual i ti es must be assi gned. Once theseF OC condi ti ons determi ne a possi bl esol uti on.
thc si deconstrai nl Shavc tobe checked. As cvi denced in c examp1es eari er. this is
not bui l t i nto FOC. lt is on1y conf trmed after a possi b1e sol uti on has been
i denti fi ed. Equati ons (4.71)(4.73) arereferredtoas theKuhnTuckercondi ti ons [see
Exarnpl e 4.3(Secti on4.5.2) for addi ti onal di scussi onof theKuhn-Tucker condi ti ons].
then gj < 0
>0
n

p
u

U
n
u
-
-
=

n
u
-
-
.
J

e
o
d
d

; :
(4.1)
(4.2)
f(X. x2.. . xJ
"k(XI X2... . . xn)=O. k = 1.2... ..1
Su lto:
Mi ni mi ze
194 NONlI NEARPROGRAMMI NG
4. 5 EXAMPL ES
Two exarnpl es arepresentedinthis secti on. The first is an unconstrai nedprobl em
has si gni fi cant us indatareducl i on. Speci fi cal l y it illustrates theprobl cmof curve
fittingor regressi on. The second is thc bearndesi gnprobl emexpl o d inChapters 1
and 2.
4. 5. 1 Exampl e 4. 2
Probl em: A set of y Z data is gi ven. It is neccssary to fi ndthe best strai ght line
roughthedata. Thi s exerci seis termed c vefittingor data ducti onor regressi on.
Tok Pthisexarnpl esi mpl e thedatapoi nts aregeneratedusi ngtheparabol i c equati on
Z = O.5y2. The dataaregeneratedinarbi traryorder and nonunor mi nterval s.
Least Squar ed Error: The obj ecti vefuncti on dri ves a l argenumber of curve
fi tti ngl regressi on methods is the mi ni mi zati on of the least squared error. le
constructi on of the obj ecti ve ires t wo enti ti - the data and type of
mathemal i cal rel ati onshi p betw n the data. In this xarnpl e it is a strai ght line.
General l y it can be any pol ynomi al or any functi on ror at manner. Thi s is usual l y
theway corrcl ati onequati ons aredetcrmi nedinex. peri mcntal fluiddynami cs and heat
transfer. For this ex npl e edataarethecol l ecti onof poi nts (Y; Z/) i = 12 n
Th 1ee. xp ecdstrai ght Ii n c y
Z=X1Y+Xz
where xl is thesl opeand X2 is ei ntercept. Usi ng opti mi zi ngtermi nol ogy thereare
t wo desi gnvari abl e. For a cubi c pol ynomi a I therewilbe four desi gnvari abl es.
Zpj is fittedval uefor ei ndependcnt vari abl eY; eobj ecti vefuncti on whi ch is
themi ni mumof thc squ eof theerror ovcr aI l of thedatapoi nts. can bcxp ssedas
n 11
Mi ni mi ze f (x
1
) = L [ Z/ - zpi = L [Z/ (X +X2)]2 (4. 74)
;=1 1=>1
It is possi bl e tocompl ete formul ati onwi si decons nts aI ough is not
necess y. leF OC ar
= 2[ Zj - ( x
(4.75a)
=; 2I Zi - i
(4.75b)
4.5 EXAMPLES 195
Exp di ngtheexpressi ons inthebrackets and reorgani zi ngas a line ma i xequ on
11
"
11
yf 2 Y; LZ;Y;
LY; L11 I iz;
/=1 I=J ;=1
Not that the matri ces can be set up easi l y and sol ved usi ng MATLAB. The Hessi an
matri x is thesqu ematri x on theleft The SOC requi res at it be posi ti vedefinite.
Code: Sec4_5_1. m: Exarnpl e 4. 2is sol vedusi ngMATLAB dtheresults f rom e
Command wi ndowareshown bel ow. Compar e this code wi th at inChapter 1. The
results aredi spl ayedon a pl ot (not i ncl udedin book). New features in codeare
theuseof randomnumbers usi ngthesumcommand and wri ti ngto theCommand
wi ndow. Fr omthepl ot and tabul ar vaI ues it iscl ear that a l i near fit is not acceptabl e
for is exampl e. The quaI ityof thefit is not undcr di scussi onin is xarnpl eas it is
apparent anonl i near fit shoul dhave bcen chosen. Thi s is lefl as anexerci sc. Thc
fol l owi ngwill appear in eCommand wi ndowwhcn ecode is run.
Resul t s f r omLi near f i t
obj ect i ve f unct i on: 16. 5827
desi gn var i abl es x1 x2:
2. 3458
- 1. 5305
yi Zl. Zp di f f
4. 7427 11. 2465 9. 5951 1. 6514
3. 7230 6. 9304 7. 2031 - 0. 2727
4. 3420 9. 4266 8. 6552 0. 7714
0. 3766 0. 0709 0. 6470 0. 7179
2. 4040 2. 8895 4. 1089 - 1. 2193
4. 1054 8. 4272 8. 1002 0. 3270
4. 1329 8. 5404 8. 1646 0. 3757
3. 3369 5. 5674 6. 2973 0. 7299
2. 7835 3. 8739 4. 9991 - 1. 1253
4. 2032 8. 8334 8. 3296 0. 5039
3. 1131 4. 8456 5. 7723 - 0. 9267
3. 1164 4. 8561 5. 7802 - 0. 9241
3. 2415 5. 2538 6. 0737 0. 8199
0. 1954 0. 0191 - 1. 0720
1. 0911
0. 7284 0. 2653 0. 1781 0. 0871
4. 5082 10. 1619 9. 0450 1. 1168
1. 9070 1. 8183 2. 9430 - 1. 1247
0. 4974 0. 1237 - 0. 3638 0. 4874
196 NONlI NEAA PAOGAAMMI NG
1. 9025
0. 1749
1. 8098 2. 9325 - 1. 1227
0. 0153 - 1. 1203 1. 1356
A

X

-
- r

a

M

F
o

s

e

u

4
F
E
q
4
a
7
2

V
0
4

n
4
5

e
9
5
4

1
0
e
2

4. 5. 2 Exampl e 4. 3
Thi s exmnpl c is thesame as Exampl e 23. t he fl agpol epmMem. I t is qui tedi ffi cul t to
3315
itzi F c z
Mi ni mi ze f(xl ) = 6. 0559 E -
(4.76)
Subj ect to:
gl (X ): 7. 4969E05.x7+ 40000x
1
- s 0
(4.77a)
g2(XI' ):(5 +1. 4994E 05x
r
)(.x7+ X ) 17083E07( Xi S 0 (4.77b)
g3(Xl' x2): 1. 909I E- 03xl +6. 1116E- - 0. 05( x1- xi) S 0
- x
1
+ 0. 001 S 0
(4. 77c)
(4. 77d)
(4.78)
0. 02S x
1
S 1. 0; 0. 02S x
2
S 1. 0
K 'uh n- 7 i 'ucke. r 'ondii N ons : F ou r Lagrang cmL t pli ersa r innIr r
u ty c o treU nts dtheLagra ngi an nis
F( Xs) = 6. 0559 E 05(xf +
s p .4969E 05xf + 40 OX
r
- 9. 7418E06( x1- xi ) ) +
13
2
((50+ 1.4994E 05x
l
)(xf + +
(1. 9091E- 03x. +6. 1l 16E- 0. 05( x1- 4) +
s4( X
2
- x . + 0. 001)
(4.79)
4.5 EXAMPLES 197
where X = [XoX2] and s = [131 132 s4]T. leKuhn-Tucker condi ti ons requi ret wo
uati ons usi ngthegradi ents of IheLagrangi an wi spectl o edesi gnvari abl es
4
2
9
E
p
d
p
f
J
S
E
J
aF _ aF
=0: =0
aX
r
aX2
(4.80)
The acaI expressi onfor egradi ents obtai nedf romEquati on (4.80) is left as
cxerci sefor thestudent. leKuhn-Tucker condi ti ons areappl i edby i denti fyi ngthe
vari ous cases based on theacti veness of vari ous sets of constrai nts.
For this probl emn = 2 and m = 4. Thcre are 2 =2
4
= 16 cases that must be
i nvesti gatcd part of KuhnTucker condi ti ons gi venby Equati on (4.73). Whi l e
some of thesecases aretrivial neverthel ess it is a formi dabl e task. Thi s probl emwas
sol vedgraphi cal l y inChapter 2 and thesame graphi cal sol uti oncan be expl oi ted10
i denti fythep ticular case needs tobe sol vedfor thesol uti on. Vi si ti ngChapter 2
and sc ni ngFi gure 2.9 it c bei denti fi ed at cons ints 81 and 83areacti ve(as
conf i rmed by ezoomed sol uti on inFi gu 2.10). The sol uti on is xT= 0. 68m and
xi = 0. 65m.
lf 81and 8J areacti vecons EnemL ul
By thc sa mereasoni ngthemu ltipliers associ atedw i inacti ve constrai nts 82 d
E4 ' t iS132and 134 must bc set tozero. This i nformati onon theacti veconstrai nts
can be used tosol vefor xj xi is is a sy mof t wo uati ons int wo unknowns.
Thi s d not however compl ete sfacti onof eKuhn- T ker condi ti ons. 131
and sJ must be sol vedand veri fi edthat they are ve. 82and 84must be eval uated
and veri fi edthat ey areless thanzero.
Sec4_5_2. mis thecode segment at is usedtosol ve parti cul ar casedi scusscd
abovc. Thc probl emis sol vcdsymbol i cal l y. Two vcrsi ons of theprobl emaresol vcd:
theori gi nal probl emand a scal edversi on. 1 obj ecti vefuncti on(4.76) is lJl odi fi ed
tohave a coeffi ci ent of uni ty. Thi s shoul dnot ch getheval ueof thedesi gnvari abl es
(why?). You can experi ment wi th codeand veri fy if is is i ndeed e. Note c
val ues of theLagrange mul ti pl i ers duri ng is cxerci seand i nfer therel ati onbetween
the m tipliers and the scal i ng of obj ecti vefuncti on. The fol l owi ngdi scussi on
assumes that thecode has been runand val ues avai l abl e.
Referri ng to e sol uti on of thc casc above the opti mal va)ues for desi gn
vari abl es are xj :::: 0. 6773 m and xi = 0. 6443 m. Thi s is very cl ose to the graphi cal
sol uti on. Bef ore we concl ude thesol uti onisachi eved takenote of themul ti pl i ers:
=5. 2622e - 010 d =-0.1270. If themul ti pl i ers arenegati ve enit is not a
mi ni mum. The val ues of econstrai nts at theopti mal val ues of thedesi gnare82=
- 4. 5553e + 007 and 84 = 0. 0320. Bot h val ues suggest that the cons nts are
i nacti ve. At least esol uti onis feasi bl e-al l cons ints aresatisfied. The val ueof
st is zeroand it is a1soan acti vecon
.
}
consi deredacti ve. 1esol uti on however vi ol at thesi decons aints. 'hegraphi cal
sol uti onis rather expl i ci t ini denti fyi ngthemi ni mumat epoi n establ i shedinthe
first sol uti on. 1e vial caseof themul ti pl i er bei ngzerowhen theconstrai nt is acti ve
is observcd wi threspect to the first t wo constrai nts express stress rel ati ons. ln
general c tcrms inthesei nequal i ti es have l argeorders of magni tude especi al l y if
thereareothrequati ons at express constrai nts on di spl acements asor thelike.
Large val ues exert a l argei nfl uenceon theprobl emsomuch as tocause l ower val ue
constrai nts tohave littleor no signifi ncc for eprobl em. This is usual l y a severe
probl eminnumeri cal i nvesti gati ons. The remedy for is is scal i ngof thefuncti ons.
Scal l ng: Consi dcr e order of magni tude in Equati ons (4.77a) and (4.77c).
Numeri cal cal cul ati ons are i ven by l arger magni tudes. l nequal i ty (4.77c) wi l l be
i gnoredinrel ati ontotheothcr functi ons even though thegraphi cal sol uti oni ndi cates
is acti ve. sis a frequent curren inall ki nds of numeri cal t hni qu. The
standardapproach to mi ni mi ze e i mpact of l argevari ati ons inmagni tudes among
di ffe 1t equati ons is tonormal i ze erel ati ons. Inpracti cethis is al soextended to e
vari abl es. is is referredtoas scal i ngl hevari abl es and scal i ngthefuncl i ons. Many
current software wi11scal etheprobl mwi thout user i nterventi on.
S l i ng Vari abl es: 1eprcsencc of si deconstrai nts inprobl emformul ati onal10ws
a natura1dcfi ni ti onof scal edvari abJes. The user-defi nedupper and l ower bounds are
used toscal eeach varl abl ebetween 0 d 1. Therefore

=- - ' ; x: = scal edithvari abl e
Xi - Xi
X=x;(. -x:) +X:
(4.81a)
(48I b)
Intheori gi nal probl emEquati on (4.81b) is usedtosubsti tutefor theori gi nal vari abl es
after whi ch the probl emc be expressed in terms of scal ed vari abl e. An al temate
formul ati onis to use on1y theupper val ue of thesi deconstrai nt to sca1e thc desi gn
vari abl e:
si =xi / 4e
xj =xj
(4.82a)
(4. 82b)
Whi l e is l i mi ts hi gher scal edva1ue to 1 it does not set el ower scal edval ueto
zero. For e exampl e of is secti on there is no necessi ty for scal i ng the desi gn
vari abl es si ncetheir ordeJ' of magni tudc is one whi ch is exactl y what sca1i ngatl empts
toachi eve.
Scal i ng the Constrai nts: Scal i ngof thefuncti ons in eprobl emis usual l y critical
for a successful sol uti on. Numeri cal techni qucs usedinopti mi zati onareiterative. In
4. 5 EXAMPLES 199
ch i terati onusual l y egr i ent of thefuncti ons at thc current val ueof thedesi gn
vari abl es isi nvol vedinthecal cul ati ons. Thi s gradi ent exp ssed amatri x is ca1l ed
eJ acobi matri x or si mpl y theJ acobi al l . Sophi sti cated scal i ngtechni ques [78]
empl oy thedi agonal en es of this matri x as metri cs toscal etherespecti vefuncti ons.
Theseen ar eeval uatedat thestarti ngval ues of thedesi gnvari abl es. The functi on
canal sobe sca1ed in esame manner as Equati ons (4.81) and (4.82). For thef ormer
anexpectedl ower and upper val ues of theconstrai nts ar necessary. Inthis exerci sc
e constrai nts will be scal edusi ng rel ati ons si mi l ar to the rel ati ons expressed by
ualities (4.82). The scal i ngfactor for each cons nt wi l l be determi nedusi ng E
starti ng val ueor theinitial guess for thevari abl es. A starti ng val ueof 0.6for both
desi gnvari abl es is sel ectedtocomput e eval ues n cessaryfor scal i ngthefuncti ons.
The scal i ngconstants for theeq ons arecal cul ated
gO= 293888.4; g20=102561.12; g30=1 70E- 03; g40=1
1e first three constrai nts are di vi ded through by their scal i ng constants. The last
uati on is unchanged. 1eobj tive functi on has a coeffi ci ent of one. The scal ed
probl emis
Mi ni mi ze 1 -4
(4.83)
8.:
(4. 84a)
If
2
: (0. 88+ 1. 4619x)(+x+) 166.5641 (xi - 0 (4. 84b)
: 1. 0868x. + 0. 3482 - 28.4641 (x1- 0
C4: X2- XI +0. 001
0. 02 XI 1. 0; 0. 025; x2 1. 0
(4.84c)
(4. 84d)
(4.79)
Sec4_5_2_scal ed. m: The code (coul d have becn part of Sec4_5_2. m)
i nvesti gates the scal ed probl emin same way t the ori gi nal probl emwas
i nvesti ga . Pri mari l y is expl oi ts i nformati onf romthegraphi cal soI on. 1t is
expected t the sol uti on for the desi gn vari ab1es will be the samc al though the
mu1tiplier va1ues areexpected tobe di fferent . Fr omthei nformati onin eWorkspace
wi ndowtheopti mal val ues for thedesi gnvari abl es arexi = 0. 6774 and xi = 0. 6445.
AIl of constrai nts arefeasi bl e. The opti ma1val ues of emu1ti pl i ers have changed.
The is no l ooger a zero mul ti pI i er-i t has t umed posi ti ve. There is still one
mu1tiplier wi th a negati ve si gn i mpl yi ng the poi nt has not met Kuhn- Tucker
condi ti oDS. Actual l y it shoul dhave been anti ci patedsi ncechangi ng ofl he sigll of the
mul ti pl i er is not possi bl ewi thscal i ng.
Thereis sti lI thematter of i nconsi stency betw n thegraphi cal sol uti onand the
di ssati sfacti onof theKuhn- Tucker condi ti ons. Expl ori ngthis parti cul ar exampl e has
been very torturous but has provi ded eopportuni ty toexpl ore i mportant but rel ated
quanti ti es Ii kc scal i ng. Another i mportant consi derati on is e scopc of the
Kuhn- Tucker condi ti ons. These condi ti ons app onl yat regl l l ar poi nts.
Regul ar Pol nts: Regul ar poi nts [8] arise when eq.uality constrai nts h(X) are
P sent eprob1em. l eyar a1soextended toacti vein I i tycons nts(pseudo
equa1i tycons nts). Kuhn- Tuckcr condi ti ons arc val i don1y for regu1ar poi nts. le
t wo csscnti a1features of a regul ar poi nt X. are
Thcpoi nt Xis feasi bl e(satisfies alI constrai nts).
. Thc gradi ents of the ua1ity nstr ints wclI as the acti ve in lity
constrai nts at X. must f orma l i ne i ndependent set of vectors.
In the sca1ed probl emof Examp1c 4.3 thc constrai nts g I and 83 are acti ve. The
sol uti on xj = 0. 6774 and xi = 0. 6445 is feasi bl e. The gradi ent of g I is Vg. =
[- 37. 631435.454T. The gradi ent of g2is Vg
1
= [- 34. 3119 30.4786]T. lt appears
that thet wo gradi cnts arc a1most paral1cl toeach oth r. Thi s is al socvi dent inthe
graphi cal sol uti on. The gradi ents thereforc are not Ii nearl y i ndepcndcnt -
Kuhn- Tuckcr condi ti ons cannot bc appl i ed at this poi nt . Thi s does not mean the
poi nt is not a l ocal mi ni mum. Thr is graphi cal evi dence that it is i ndeed so.
KuhnTucker are the on1y avai l abl e f orma1 condi ti ons for recogni ti on of the
opt i mumva1ues. It is uni vcrsa1ly appl i ed wi thout regard to regul ari . Some
addi ti ona1consi dcrati ons shoul dbc kept inmi nd [8].
If equal i ty constrai nts are present and al1 of e i nequa1ity cons 'aints are
i nacti ve en the poi nts sati sfyi ng c Kuhn-Tucker condi ti ons may be
mi ni mum maxi mum or a sadd1epoi nt. Hi ghcr-order condi ti ons areneccssary
toi dcnti fy thc typeof sol uti on.
If the mu1ti pl i ers of the acti ve i nequa1ity constr nts are posi ti ve the poi nt
cannot be a 10cal maxi mum. It may not be a local mi ni mumeither. A poi nt may
bc a maxi mumif mul ti pl i cr of an acti vc in ual i tyconstrai nt is zero.
REFERENCES
1. Stcin S. K. Cal cul us and Anal yti cal Geometry. McGmw- Hi I I NewYork. (1987)
2. Hostetl er G. H. Santi na M. S. and Montnl vo P. 0. Anal yti cal . Nwneri cal . and
Computati onal Methods f or Sci enceand Engi neeri ng. Prenti - Ha1I. Engl ewood CIi
NJ . (1991)
3. Mol cr C. andCosta. P. J. Symbol i c Mat h Tool box- f or e wi thMATUB- User s Gui de
MathWorks Inc.. MA.
PROBL EMS 201
4. MATLAB Demos Symbol i c Tool Box-I ntroducti on. onl i ne resource in MATLAB.
MathWorks Inc. MA.
5. Burden R. L. andFaires J. 0. Numeri cal 'sis. 4th ed.. PWS_KENT Publ i shi ng
Company Boston. (1989)
6. Fox R. L.. Opti mi wti on Methodsf or ElI gi neeri ng Desi gn. Addi son-Wesl ey Readi ng
MA. (1971)
7. V derplaatsG. N. N/l meri cal Opti mi wti on Techl l i ques for ElI gi neeri ng Desi gl l .
McGmw-Hi 11 NewYork. (1984)
8. Arora J. S. l ntroducti ontoOpti maI Desi gn. McG mw- Hi ll NewYork. (1989)
9. Kuhn H. W. andTuckcr A. W. Nonl ul ear Programmi ng. Proceedi ngs 01theSecond
Berkel ey Symposi umon Mathemati cal Statistics al l d Probabi/i. J. Ncyman (cd.)
Uni vcrsi tyof Cal i forni aPrcss 1951.
PROBL EMS
(In many of the probl ems be1ow you are requi red to obtai n the numeri cal
sol uti on.)
4.1 Defi nc two nonl i near functi ons in two vari abl es. Fi nd thcir sol uti on through
contour plots.
4.2 For thc functi ons inProbl em4.1 obtai n the gradi ents of the functi on and UI
J acobi an. Conf i rmt hemusi ngtheSymbol i c Mat h Tool box.
4.3 Def i ne thedesi gnspace (chooses si deconstrai nts) for a two-vari abl eprobl cm.
Dcf i ne t wo non1i near functi ons in t wo vari abl es that do not have a sol uti on
wi thi nthis space. Graphi ca1lyconf i rmth rcsul t .
4.4 Defi ne a desi gn space for a two-vari ab1e probl em. Defi nc t wo nonl i near
functi ons int wo vari abl es haveat least t wo sol uti ons wi thi n espacc.
4.5 Def i ne a nonl i near functi onof t wo vari abl es. Choose a contour val ueand draw
thc contour. I dcnti fy a poi nt on thc contour. Cal cul atetheva1uc of thegradi ent
at that poi nt. Dr awthegrdi ent at thepoi nt usi ngthecomputed val ueCal cul atc
Hessi c above poi nt .
4. 6 Usi ng therel nti oninEquati on(4.23) establ i sh t thegradi ent is normal tothe
tangent [aphysi cal i nterpretati onfor Equati on (4.23)].
4.7 Def i ne a nonl i near functi onof vari abl es. Choose a poi nt in the desi gn
space. Fi nd e gradi ent of functi on at the poi nt . Ca1cul ate the Hessi an
matri x at thesamc poi nt.
4. 8 Exprcss theTayl or seri es expansi on (quadrati c) of thefuncti onftx) = (2- 3x +
x2) sinx about poi nt x = 0.707. Conf i rmyour resul ts throughtheSymbol i c
Mat h Tool box. Pl ot theori gi nal functi onand eapproxi mati on.
4.9 Expand thefuncti onftx.y) = 10(1- x2)2+ (y - 2)2quadrati ca1lyabout the ~ n t
(11). How wil1youdi spl ay thei nformati on? Dr awthecontours of theori gi na1
functi onand theapproxi mati on.
4. 10 Obtai n thesol uti ongraphi cal l yforthecase when Equati on(4.10) is added to
probl emdefi ned by Equati ons (4.36) (4.39) and (4.37). Obtai n e sol uti on
usi ng eSymbol i c Mat h Tool box.
4. 11 Obtai n csol uti ontothc opti mi zati onprobl eminSccti on4.4.4usi ngMATLAB.
4. 12 1nSecti on4.5.1 obtai nthecoeffi ci ents for a qua C fit to eprobl em.
4. 13 Use an exampl e toshow opt i mumval ues for thedesi gnvari abl es do not
cb ge theobj ecti vefuncti onis mul ti pl i edby a const . Prove thesame if a
constant is added to efuncti on.
4. 14 How does scaI i ngof theobj ecti vefuncti onaffcct theval uc of emul ti pl i ers?
Usc ancxampl c to i nfer the sul t. How does scal i ngtheconstrai nt affect the
mul ti pl i s ? Check wi than exampl e.
5
NUMERI CAL TECHNI QUE8 -
THE ONE- DI MENSI ONAL
PROBLEM
'he one-di mensi onal unconstrai ncd opti mi zati on probl emis often i ntroduced d
sol vednumeri cal l y in most courses at deaI wi th numeri cal anal ysi sJ techni ques. It
appears i ndi rectl y notabl y as fi ndi ngthe root of a nonl i near equati on whi ch is e
same as satis i ng eFOC for an unconstrai nedopti mi zati onprobl em. Two of the
pop ar techni ques are eNewton- Raphson and ebi sccti on techni que. 'hey E
i ntroduced in Secti on 5.2 to provi de a compari son wi th the methods used in
opti mi zati on. In engi neeri ng dcsi gn e si ngl e-vari abl e opti mi zati on probl emis
prob l ya vialitybut one-di mensi onal opti mi zati on is a criticaI component of
m ti vari abl e desi and is di scussed in S on 5.3. Meanwhi l e the cI assi cal root
fi ndi ng methods i denti fi ed above do not pl ay any si gni fi cant part in numeri cal
opti mi zati on techni ques as they are consi dered computati onal l y expensi ve and not
robust enough over a l argecI ass of probl ems. An i mportant consi dcrati oninnonIi near
desi gnopti mi zati onisthat duri ngtheinitial i terati ons when you aretypi cal l yfar away
o m e sol uti on accuracy c bc ded for speed. In these i terati ons it is al so
i mportant tomove throughto cncxl itcration i nstcadof fal teri ngat ecurrent one.
These.criteriahave qui dthat thcone. di mensi onal techni ques be si mpl e inconcept
as wel l as easi l y i mpl emcntabl e. Two di fferent techni ques the pol ynomi al
approx l ati onand egol densecti on provi de popul ar support for most opti mi zati on
softw e. Very oftenthet wo techni ques are i mpl emented incombi nati on. They are
di scussedinSecti on5.2.
203
204 NUME ICALTECHNI OUE5 -THE ONE. OI MENSI ONAL PROBLEM
5. 1 PROBL EM DEFl NI TI ON
In order to connect wi ththe di scussi ons later theone-di mensi onal vari abl e will be
i denti fi ednsi nstead of X or XI Inthefol1owi ng chapters is is a1soreferredtons
theonc-di mensi onal stepsi zecomputati on.
Exampl e 5.1 This exampl e is si mi l ar toone inChapter 4:
Mi ni mi ze f() =( _ 1)2(- 2) ( - 3)
Subj ectto: 0 S S 4
Theprobl emdoesnotrepresentanyreal desi gn. It wasconstructedtohavemul ti pl e
l ocal mi ni mums wi thi n the area of i nterest. Expressi on (5.1) reprcsents an
unconstrni nedprobl em. Thesi deconstrai ntsaccompanyal1probl cmsi nthi sbookto
con vey thei dea that therearenotru Iyu nconstrai ned prob l ems. Si deconstr ai ntsal so
defi ne an acceptabl e desi gnregi on for all probl ems. Secti on5.2will expl ore this
prob l emthroughthcvar i ousmet hodsmen ti onedcar I i er .
5. 1. 1 Const r al ned One- Dl mensl onal Pr obl em
The onl y that Exampl e 5. 1 a si ngl e-vari abl eprobl em cana ommodat e is
i nequal i constrai nt . Fr omthc previ ous chaptcr there can be mo onc such
cons int . A si ngl e-vari abl eprobl emcnnnot have ual i tyconstrai nt (Chapter 4).
Ina constrni nd mul ti vari abl eopti mi zati onprobl emit will oftenbe necessary l o.fi nd
estepsi zesuch econs int j ust becomes acti ve. It is possi bl etodef me
probl em.
Exampl e 5. 1a
Mi ni mi ze f(. )=( 1)2(- 2) ( - 3)
Subj ect to: g(): 0. 752- 1. 5 I S0
(5.1)
(5.3)
OS S 4 (5.2)
In actual appl i cati ons it is mor e I i kel y at cxpressi ons (5.1)(5.3) establ i sh t wo
probl ems that provi de di fferent sol uti ons to c stepsi zc. The first is esol uti onto
expressi ons (5. 1) and (5.2) whi ch is the samc as Exnmpl c 5. 1. The second is the
sol uti onto
g(. ): 0. 752- 1. 5 1=0
OS S 4
(5.3)
(5.2)
(5.1)
(5.2)
5.2 SOLUnONTOTHE PROBLEM 205
The second sol uti onisuseful onl y if expressi on(5.3) isan acti veconstrai nt Otherwi se
thesol uti ontoExampl e 5.1wil1prevai 1. lefinal sol uti onchosen ensu sexpressi ons
(5.1) (5.3) aresatisfied. A graphi cal i nvesti gati onin enext secti onshoul dillustrate
this featurec1earl y.
5. 2 SOL UTl ON T O T HE PROBL EM
The sol uti ontoExampl es 5.1 d5. 1ai s exami nedgr phi caIIy. Next thec1nssi cal root
fi ndi ngtec i qucs of Newt on- Raphson and ebi secti onmet hod arepresented. These
M fol l owedby thepol ynomi al approxi mati on and egol den secti onmcthod.
5.2.1 Graphl cal Sol ut l on
The exampl e has al ready made its appearance inChapter 4. Fi gu 5.1 is a pl ot of the
obj ecti vefuncti oninExampl e 5. 1. It al so i ncl udes c sol uti onExampl e 5.1a. If
expressi on (5.3) represents an i ncqual i ty constrai nt the feasi bl e regi on for one-
vari abl e functi ons is qui tedi fferent f romthe two-vrui abl e probl ems encountered in

J
" '
/
3
g()
'
h
{
)
-
constral nedmi ni mum
1
.

H J " ' l g l o b a l
unconstrai nedmi ni mum
.
;.. - -
-2
4
2
a
3.5 0.5 1. 6
gure5.1 Graphi cal solulion: Exampl e 5.1.
206 NUMEAI CAL TECHNI QUES THEONE. OI MENSI ONAL PAOBLEM
::tz;; : ; 321i222211;
1 F Z Z ; i t mt
Exampl e5. 1
=2.6484; f * = 0. 6195
(5.4a)
Exampl e 5.1a
* = 2.5275; f. = - 0. 5815
(5.4b)
5. 2. 2 Newt on- Raphson Techni que
The Ncwton- Raphson techni que alsorcferr dtoasNcwton' s techni que isa gradi ent-
bascd n to fi ndi ng c root of a si ngl c nonl i near eq on. Root fi ndi ng
l li t U 1 2 2 Z Z ; : : r h eequati a ;:;L
( ) =0
wher () is a nonl i near equati oninthevari abl e.
letechni quehas addi ti onal featurcs:
It hasag metri cunde i nni ng.
It ust he Tayl or series expanded I i nearI y.
It is iterative.
It has thepropertyof quadrati c convergence.
Any iterativc tcchni queaddrcssi ngEquati on(5.5) canbesummari zed as
Generi c Al gori tl rm(A5.1)
Step 1. Assum
Step2. Cal cul ate.1
Step3. Update a = +.1
If converged( () =:0) enexit
If not converged(( :;f: 0) en a
go toStep2
I
p
iI
10bedownl oadedf wmthcwebsile indicaledbyboldfnccsnnsserif Iype.
(5.5)
5.2 SOLUT10NTOTHE PAOBLEM 207
The above sequen is fairly standard in iterative techni ques for many cl asses of
probl ems. It al socapl ures theessencefor most of thetechni ques tofol l owinthisbook.
Sl ep 1 i ndi cates thepri mary f i rc of iterativeme ods- t hestartingsolution. This
is an initial guess provi ded by the user 10 slart the iterative process. The iterative
process is conti nued10thesol uti onby cal cul ati ngchanges inthevari abl e(.1.). The
val ueof thevari abl eis updated. Convergence ischecked. If Equati on(5.5) is satisfied
thenthe sol uti on has been found. If convergence is not achi eved the updated
val ue provi des the val ue to proceed wi th next iteration. Of course .1can be
cal cul atedinmany ways and Newton- Raphson isjust one of theways establ i shthe
I change inthevariable.
l j
Cal cul atl on 01 A ( Newt on- Raphson) : Let b e the current val ue of the
variable. It is assumed that Equati on(5.5) is not satisfiedat is val ueof thevari abl e
( e asonfor us toiterate). Let a be a nei ghbori ngval ue. Ideal l y we woul d like10
achi eveconvergence at this val u (even we are aware it rni ght take us several
itcrations 10achi evc converg nce). Usi ng Tayl or' s theoremexpanded tothe linear
terrnonl y
d
() = ct(+) = ( ) +25A=0
A= 1=- ' I cb( o.)
Id
l
(5.6)
(5.7)
For Equati on(5.7) tobc effccti ve egradi ent of thefuncti onshoul dnot be zcro. lt
is a1soi nevi tabl e at changes i nwi l l be l argewhere ct is flat and smal 1where the
sl opeis large. For theNewton- Raphson method tobe effccti vetheiterations shoul d
avoi d gi ons where thesl opeis smal l . Thi s is a scri ous di sadvantageof is mcthod.
Fi gure5.2ilI ustrates thegeometri cal constructi onof tbetechni quc at thepoi nt = 3.
Exampl e 5.1 The Newton- Raphson tcchni queis appl i edtothefirst-order condi ti ons
( FOC) of Exampl e 5. 1.
Mi ni mi zc f ()=( 1)2( 2) ( - 3) (5.1)
Subj ect to: 0 SS4 (5.2)
The FOCi s
{ == l - 2 2)( -3) )+( 1) 3 )+( l) r-2 2
(5.8)
208 NUMERI CAL TECHNI QUES -THE ONE. OI MENSI ONAL PROBLEM
10
8
6
2
- 2
0 0.5 1.5 2 2.5 3

Fl gure5.2 Devel opment 01Newton. Raphsontechni que.
3.5 4
J e Technl que: Two i terati ons areshown here. The compl ete appl i cati onis done
through MA11AB codesecs- 2- 2. m. The code uses symbol i c cal cul ati on.I nr umi no
codethefi gurewi ndowdi spl ays rnati onof theiterati ravel 60;
Al so shown is the change in 6wi t h each iteration. The iterative resul ts are
summari zcd ina tabl e8t ecnd. The ca1cul ati ons requi rc thegradi ent of ():
' ( ) =2 ( - 2) ( - 3) +4*( 1)*( 3) + 4*( 1) *( - 2) +2*( 1)^2
Iterati on 1:
=3; f () =0. 0; () = 4 I t' ( ) =16 6=- 0.25
I t ( +L1.) = 0. 875- not converged at least another i terati onneccssary
Iterati on2:
=2. 75; f(1. )=-0. 5742; (. )=0. 0875; . )=9. 25; = O. 46
( +6) =0. 1040 not converged at least anothcr i terati onnccessary
5.2 SOLUTl ON TOTHE PROBLEM 209
'he code Sec5_2_2. mr ui res fi ve i terati ons to converge the va1ue of I t to
1. 0E 8(consi deredzerohere). T svaI ue of 1. 0E08is termed theconvergence (or
stoppi ng) criterion. Thc acccl erati on of convergence or rate of convergence to thc
sol uti onis formal l y defi ncdinmany di fferent ways. One of thc ways is tomoni tor the
ratio of ( +6 )1 (). Anothcr i st r ack 6. ln appl yi ng and understandi ng
numeri caI techni ques it is i mportant todevel op an i nsti nct and feel for thenumbers
correspondi ngtovari ous functi ons ei terati ons progress. The speedor equal i ty
of convergence can be gauged by e shift in the dcci maI poi nt in the quanti ty of
interest through the iterations. Consi der the vaI ue of I t through the fi ve i terati ons
[4. 00000. 87500.10400. 00230. 0000]. Defi ni tel y echangcs inare nonl i near.
The Newt on- Raphson techni quecan be di recUy appl i edto Equati on(5. 3)-after
aI l themeUl od is desi gnedtooperateon theseki nds of probl ems. It is left as anexerci se
for r der. lere is a very i mportant feature of most itcrative techni ques that
exp!ore m tiple sol uti ons- as is e case of Exampl e 5.1. Rerun the code
Sec5_Z 2. mwi tha starti ngvaI ue of 0.5 for a.. After six i terati onsconverges to
evaI ue of 1.0 whi ch is a l ocaI mi ni mum. Wherewoul d i terati ons l eadif thestart
vaI ue is 1.5 ? These answers are sy to s graphi caI ly in a one- or maybe a
two-vari abl eprobl em. Ina typi caI engi neeri ngdesi gnprobl em si gni fi cant experi ence
and i ntui ti on are requi red to recogni ze Ul esc si tuati ons when cy occur. A heal thy
S pi ci onof thesol uti on must aI ways be intheback of one' s mi nd when expl ori ng
mul ti vari abl e probl ems. In summar y most iterative numeri cal techni ques are
desi gnedtoconverge tosol uti o1Sthat are c/ ose 10 where theystartfrom.
5. 2. 3 Bi sect l on Techni que
This is another popul ar tcchni quetofi nd tof a functi on. It is al socaI l eda bi nary
search or i ntervaI haI vi ng procedure. Unl i ke the Newt on- Raphson met hod this
procedure does not qui retheevaI uati onof egradi ent of efuncti onwhose root
is bei ngsought. One of thereasons themet hod isbei ng i ncl udedis tocompare it wi th
thegol den sccti onmethod di scussedlatcr. The numeri cal techni que in is method is
basedonthe oni ng therl
t(or zero) of a functi onis trappedor bound between

a posi ti veand a negati vevaI ue of thefuncti on. The actuaI sol uti onachi evedby E
method is an i ntcrval inwhi ch ezero or thesol uti oncan be l ocated. Thc finaI i nterval
is typi l I y dctcrmi ned tobe a vcry smalI val ue cal l cdthc tol erance of theordcr of
l OE-08. S ce is is a root fi ndi ng procedure when appl i ed to opti mi zati on it is
appl i cdto gradi ent of ot tive whi ch we i denti fi cdas (.) Equ on(5.5).
Establ i shi ngthemi ni mumof theobj ecti veis thereby transl atcdtol ocati ng erootof
e gradi ent functi on. It is possi bl c to devcl op a bi secti on tcchni que for di rectl y
handl i ngthemi ni mi zati onprobl em.
The met hod r ui res t wo poi nts to start sayn and bThe vaI ue of It at E
poi nts must be opposi te in si gn. 'hesc can be the si dc constrai nts of the
one-di mensi onaI opti mi zati onprobl em. lt is assumed at Icast one sol uti onexi sts
in is initi interval. Duri ng each i terati on is i nterva1is haI ved wi th emi dpoi nt
of thei nterval repl aci ngci ther or bwhi l e kccpi ng thc root still trappcdbctween
210 NUMERI CAL TECHNI QUES -THE ONE-DI MENSI ONAL PROBLEM
amcti on val ues of opposi te si gn. The i t emt i veMmi que isexpressedas thefol l owi ng
aI gori thm(aset of procedures erepeated).
Al gori tI J m!or Bi secti onMetl l od (AS. 2)
Step 1: Choose and tostart . Let
Step2: Set = + ( )12
Step3: If (.) = 0. 0- Converged Sol uti onexi t
se If ( b- n) <l OE04- tol erancemet - exit
El se If (.)* ( )>0. 0; en
El seb
go toStep2
We consi der Exampl e 5.1 whi ch is reproduced here
Mi ni mi ze f{) =( _ 1)2(- 2) ( - 3)
Subj ect to: 0 S 4
ThcFOCi s
(
(5.1)
(5.2)
(5.8)
S 5_ . 3. m: This AB code runs the bi secti on met hod on pl e 5. 1. It
iIl ustrates thetrappi ngof emul l rnumby thet wo val ues on ei ther si deOf themot.
le 17 i terati ons for a tol erance of 1. 0 E04 whi ch termi nated thc program.
Compa hi s wi th5 itcrati sfor theNewt on- Raphson 1 hod. for a bettcr sol uti on.
U l ower number of i ratJ ons c be to quaJ ity of ir ' nnati on edto
upd e d vari abl e changcnc gradi ent-bascd i nformati on uscd in thc
Newton- RaPI EBon tEChmque ls mor e sophi sti cated. though cgradi ent computati on is
an addi ti onaJ work l oad.
T K probl emdescri bedby Equati on ( 53) cmbe handl ed di rectl yby thEbi secti on
od (.) is same (). Ina wa Y. t he
opp o utY to rev1 1 sit pri or knowl cdge. parti arl y wi th resp defi ni ng and
sl ati nganal gori i ntoI UDngcode. There is u su astron gc orre spo nd l
betwecn estcp-by-stcpd cri pti onof e numeri caJ techni que (theal gori thrE
e c od e at wi l l i mpl ement e ps effect a sol uti on (for exampl e.
:22tit-B; EP;521111:ZZ220: 22LtJbre
fol l owl ngt wo methods areused extensi vel y f oronedi mensi onal mi mi zati on. They
handl c themi ni mi zati onprobl emdirectly.
5.2 SOLUTI ONTOTHE PROBLEM 211
5.2.4 Pol ynoml al Appr oxl mat l on
The me od is si mpl c inconcept. I nsteadof mi ni mi zi ng a difficult functi onof one
vari abl e mi ni mi ze a pol ynomi al at approxi mates thefuncti on. Thc opti mal vaI ueof
e vari abl mi ni mi zes the pol ynomi al is thcn consi dcred to approxi matc the
opti mal vaJ uc of evari abl efor t1J e ori gi naI functi on. It is rarefor t1J e dcgree of t1J e
approxi mati ng pol ynomi aJ to excecd threc. A quadrati c approxi mati on is standard
less e irddegree is w ranted. It is c1ear that scri ous errors inapproxi mati onare
expectedif epol ynomi al is tosi mul atethebehavi or of eori gi nal functi onovcr a
l .ge range of vaI ues of vari abl e. Mathemati cal t1J eorems exist justify a
quadrati c representati onof t1J e functi on wi tha prescri beddegree of error wi tIJ ina
smaJ l nei ghborhood of t1J e mi ni mum. What is ensures is at t1J e pol ynomi al
approxi mati on gets better as e mi ni mumis bei ng approach . Exampl e 5.1.
repf ucedfor conveni epce is used for iI1ustrati onof theprocedure.
Exampl e5. 1
Mi ni mi ze f(. ) =( _ 1)2(- 2)( 3) (5.1)
Subj cct to: 0 S S 4 (5.2)
A quadrati c pol ynomi aJ P() is used for the approxi mati on. This pol ynomi al is
expressedas
P() =bo+b +b
2
2
(5.9)
Two el cments n d to understood pri or to t1J e fol1owi ng di scussi on. le first
conccms eeval uati onof t1J e pol ynomi . and t1J e second concerns e i ncl usi onof
exp ssi on (5.2) in e di scussi on. Thc pol ynomi aJ is compl etel y defi ned if the
coeffi ci ents b
o
b. and b
2
arc known. To determi ne m. t1J reedatapoi nts [(1' /.)
( .2' !z). (.313)] arc generatedf romEquati on (5.1). Thi s sets up a I i ncar systemofthree
equati ons in e unknowns by qui ri ng t 1J e vaJ ucs of t1J e functi on and the
pol ynomi aJ must be the same at t1J e t1J rce poi nts. The sol uti on to t1J is systemof
equati ons is val ues of t1J c coeffi ci ents. 1 consi derati on of expressi on (5.2)
depends on t1J e typeof one- di mensi onaJ probl cmbei ngsol ved. If theone-di mcnsi onaI
probl emis a genui ne si ngl e-vari abl edesi gnprobl em enexpressi on(5.2) needs to
be present. If e one- di mensi onaI probl em is a subprobl em m e
muJ ti di mensi onal opti mi zati onprobl em t1J en expressi on(5.2) is not avai l abl e. Inthat
ce a scanni ng proccdure is used todefi n 1 and3
Scannl ng Procedure: lepr is startedf rom l ower I i mi t f or. A vaJ ue
of zero for is vaJ ue bej usti fi edsi nceit refers tovaJ ues current iteration.
A constant i ntcrval for 6. i s al soi denti fi ed. For a wel1-scal edprobl emt1J is vaI ue
is usual1y 1. Starti ngat t1J e l ower I i mi t i ntcrvaI is doubl ed until tlJree poi nts are
.
213
5.2 SOLUTONTOTHE PROBLEM

.

t

r

NUMERCAL TECHNQUES- THE ONE. OMENSONAL PROBLEM
detenni ned such at the mi ni mumis bracketed between them. Wi respect to
Exampl e 5. 1 thescanni ngprocedure generat thefol l owi ngval ues:
212
10
=0; /=6 = 0; j{O) = 6;
B
=1; =0 = 1; j {l ) = 0;
this cannot be t hemi ni mumis n01yet pped = 2; j (2) = 0;
a

2
/2= 18
A process suchas that illustrat d is essenti al sSit is bothi ndi fferent to theprobl em
bei ngsol vedand can be programmed sily. The i mportant requi rement of any such
proc s is to ensu at the mi ni mumlies between e Ii mi ts establ i shed by the
procedure. lis procedure is devel oped as MATI . ABm- fi) e Upper Bound_1Var . m.
The set of I i near equati ons to estabIi sh the coeffi ci ents of the pol ynomi al in
Equati on(5.9) is
6 = b
o
+ b(O) + b
2
(0) (5. 108)
=4; = 4; j(4) = 18;
(5. l Ob) 0::: b
o
+ b(l) + b
2
(l)
4 3.5 3
2

1.5
0.5
(5. l Oc) 18= b
o
+ b(4) + b
2
(16)
pol ynomi al approma1ion.
de devel opment is termed a bl ock structuredapproach- I n this way- l arg code
devel opment is possi bl eusmgsmal l erpi eces ofexi sti ngcode.
Rgure5. 3
(5.11)
Usi ng MATI . AB esoh onto eequati ons is
ho=6. 0; b=- 9; b
2
=3
ICapproxi mate sol uti ontoEquati on(5.1) is 1.5. Fi gu 5.3dcri besthepol ynomi al
approxi mati onprocedure. lt is cl ear at theapproxi mati on l eaves much tobe desi red.
On theother hand if thedatnfor thepol ynomi al coeffi ci ents were around a smal l er
regi onnear =2. 6 then results woul d be mor e i mpressi ve. Note at ra m
usc a hi gher-order pol ynomi al a better set of datais p .
The scanni ngprocedure and epol ynomi al approxi mati on techni ques wi l l fi nd
si gni fi cant use expl ori ngmul ti di mensi onal opti mi zati onprobl ems. The s tegy for
devcl opi ngMATLAB codc f romnow on must i ncl ude epossi bi l i tyof code reuse. le
same code segments can be used inmany di erent numeri cal techni ques. Such generi c
upper Bound1Var . m: Thi s codcsegment i mpl ements e detcnni nati on of e
upper boundoi l hE onof one vari abl e. The i nput tothefuncti onis the ncof t hc
:
c o t Z Z3 ; 4 2 ; ; ; 2 2 d E mu s t h a M Bm- fi)ein esame di rectory
tZTftU3231:;:;Z?5512- twhi ch thc functi on
P(. }=6- 9+32
dP
=- 9+6=0; : = =1. 5
0 r
and thepol ynomi al is
The mi ni mumfor thepol ynomi al is thesol uti onto
(5.12)
cod segment i mpl ements e pol ynomi al
functi on of one vari abl Thi s functi on uscs
upper Bound_1Var ( ' EXam
p
l e5_1' 0110)
Pol yAppr ox_1 Var. m: lis
approxi mati on method for a
Usage:
214 NUMEAI CAL TECHNI QUE5 -THEONE. OI MENSI ONAL PAOBLEM
Upper Bound_1 Var . mto detenni ne the range of e variable. The i nput to e
functi on is the name of the functi on; the order (2 or 3) of the approxi mati on;
l owbound- t he start val uc of the scan passed to Upper Bound_1Var . m;
intvlstep scanni ng interval passed to Upper Bound_1 Var. m; in ials
number of scanni ng steps passed to Upper Bound_1Var . m. The output of e
programis a vector of two values. The first el ement of thevector is thel ocati onof the
mi ni mumof theapproxi mati ngpol ynomi al andthesecondis thefuncti onval ueat
this location.
The functi on ferencedby ecode must be a MA TI . AB mfileinthesame di rectory
( Examp 5_1. m). 1e i nput for Exampl e5_1 is the val ue at whi ch the functi on
needs tobecomputed. and its output is theval ueof thefunction.
Usagc: Val ue Pol yAppr ox_1Var ( ' Exampl e5_1' 20110)(5.13)
The two programs us eswi tch/case and the/eval statements fromMATLAB. The
code alsoilI us ates callingandretumi ngf romother nctions.
5.2.5 Gol den Secti on Met hod
Th methodof gol densecti onis thccrcamof thefami l yof interval duci ngmethods
(for exampl e. ebi secti onmethod). It reduces einterval by thc same fractionwi th
eachiteration. The intervals ederi vedf romthegol densecl i onratio. 1. 61803. Thi s
ratiohas si gni fi cance a sthetics as wel l as mathemati cs [34]. The method issi mpl e
toi mpl ement . It is indifferent to eshapeand conti nui ty properti es of efuncti on
bei ngmi ni mi zed. Most i mportant Lhe number of iterations to achi evea presi bed
tol erancecanbc establ i shedbeforeLhe iterations start.
Al gori thmf or Gol den Secti ol l Met hod (AS. 3)
Step1. Choose l ow
= 0. 38197(fromGol den Rati o)
e = tol erance= (L)Onal /(up_l ow)
N = number of iterations = -2. 078lne
Sp2. 1 =( 1 -) l ow+ ;fl =/ (1)
= ri
O W
+ (l ) U
P
;f2= )
nc poi nts areeq distant frombounds
Step3. If ( i < N)
I f/I >/ 2
10" . ; 12;fi 12
~ I o w + (1-) =/ ( )
i i + 1
GoTo Step3
I f h>/ .
Il ' J
5.2 SOLUTI ONTOTHEPROBLEM 215
up/:2; 1;12A
1= (1- 't)a!ow+ ;/1=/ (. )
; r i + 1
GoTo Step3
s i iiZ!
onl ycal late evalues. The al go mis i mpl emented ina different file.
Exampl eS. l
(5.1)
Mi ni mi ze / ( ) =( _ 1)2(- 2)(- 3)
Gol den Secti onMet hod
St 1. (1.10" = o. o;l ow= 6; u
P
=4; !"' =18
L<Xj!nol = 0.001; E = (0.00114); N = 17
Step2. . = 1. 5279;/. = 0.1937; =2.4721; 12=0. 5401
F1gure54 i1l ustrates th l ayout of i nformati onat this stage.It canbCVEri nd at
. . 2arel ocatedsymm icallywi thmspect toouter bounds dO "p.
Step3. i = 1
/ 1>12
(1.lol V= 1. 5279; l ow= 0.1937; =2.4721;/. = - 0.5401
2 = 3.0557;12= 0. 2486
d
i
j
j
Y
3
5
2
3
3
i =2
Step3. conti nued. . .
12>13
a. up = 3.0557; =0. 2486; 2= 2.4721; =- 0.5401
1= 2.1115;12=0. 1224
i =3
Go To Step3 etc
IneachitcrationLhereisonl y one ofcal lationtoevaa evari abl eand
val ueofthe functi on. Therest of thEComputauonts masSI gnsngextsti ngi nformatIon.
The differcn ou bomdsi sal sochangi ngwi theachkmuon- hExmpl eal
Start: L=4. 0
Iteration1: L = 4.0- 1. 5279= 2.4721
217
vector of four pairs of vari abl eand functi onval ues after the final iteration. Wi th a
smal l tol erance yone of l hesepairs provi des mi ni muml ocati onof thefunction.
It is recommended at the secondpair be accessedfor this pu ose. 1e functi on
referenced by the code must be a MATLAB m-fi l e in the same di ctory
(Exampl e5_1. m). The i npul for Exampl e5_1. mis theval ueal whi chthefuncti on
needs tobe computed andits output is theval ueof thefuncti on.
U .ge:
Val ue Gol dSect i on_1Var (' Exampl eS_1' 0. 0010110) (5.14)
There areno newcommands. Thc organi zi nganddi spl ayi ngof i nformati oninthe
Command wi ndowis worthnoti ng. If this code is part of a larger program the
pri nti ngof i nformati oninthecommand wi ndowneeds tobe swi tchedoff. It c be
medon for debuggi ng.
Compar / son wl th Pol ynom/ al Approxl mat l on: The two popul ar methods for
mi ni ouzi ng a functi on of one vari abl e are pol ynomi al approxi mati on or gol den
section. The al gori l hms for pol ynooual approxi mati onand gol densectionhave
significant differences. The former is a one-shot approach and is accompani ed by
significant error intheesti mati onof themi ni mum whi ch will i mprove as thedatafor
approxi mati onget better. I mpliedinthei mpl ementati onof thepol ynomi al approach
is theconti nui tyof efuncti on. The gol densecti onmethod on theother hand is
iterativetechni que. The number of iterations depends on etol eranc expectedin e
final sult and isknown prior to start of thei terati ons-asignifi nt i mprovement
relative to theNewl on- Raphson method where number of iterations cannot be
predi cteda priori. The i mpl ementati onissi mpl eandtheresults i mpressi veasit isable
t ohomei non oun 1um. It is indifferent tothenatureof thefunction.
There is no reasonwhy thetwo cannot be combi ned. be gol densecti oncanbe
usedtoestablishfour datapoi nts wi tha reasonabl etol erance(i nsteadof a l owvalue)
anda cubi c pol ynooual canbe fit 10identifytheapproxi mate locationof mi ni mum.
1 one-di mensi onal subprobl emin a mul ti vari abl e opti mi zati on probl emis
empl oyed for edetermi nati onof the stepsi zeafter thc searchdi recti onhas been
identified. It is easier 10undcrstandthesel erms by recogni zi ngthegeneri c al go m
for unconstrai n opti ouzati on. In order to focus the di scussi on rather than e
general obj ccti vefuncti onrepresentati on a specific one Exampl e 5.2 is i ntroduced.
I MPORTANCE OF THE ONE- DI MENSI ONAL PROBLEM 5. 3
(5.15)
f(xl X
2
' X
3
) = (xl )2+ { ~ - x
3
)2+ 3(x
3
- 1)2
Exampl e5. 2
Mi ni mi ze
5.3 I MPOATANCEOF THEONE. DI MENSI ONAL PAOBLEM
5
up
' t
'
E
a
' J
s
NUMEAI CAL TECHNI QUES -THEONE. DI MENSI ONAL PAOBLEM
20
18
18
216
14
12
10
8
6
{
6
}
h

4 3
Gol den5ectlon- startingvaluas.
d= 3. 0557- 15279 = 1. 5278
2

Flgur 5.4
Iteration2:
4
-2
-1
2
The final iterationis reproducedby runni ngthefol l owi ngcode:

i terati on17
2. 6395
-0. 6197
2. 6406
-0. 6197
2. 6402
- 0. 6197
2. 6399
- 0.6197
lel ol erance2 I inthefinal iterationis about 0.001as expecl ed.
Gol dSecti on_1Var. m: The codc lr slates eal gori thmfor the gol den s tion
m od( A5.3) in TLABcode. The i nput 10thefuncti onis ename of thcfuncti on
t nCtname) whose mi ni mumisbei ngsought; etol crance(tol) of theapproxi mati on;
the val ue (l ow bo und) of e s c pa ed to Up perBo und_1Va rr. m m n1;
scanmnginterv ( u t 1) pa55edtoUp per Bou md_1Va r' .m;them u mE

ps11 p to Upper Bound_1 V .m. The output of programis ;
218 NUMERI CAL TECHNI QUE5 - THE ONE- DI MENSI ONAL PROBLEM
A c orygl ance at Exampl e 5.2i ndi cates atthesol uti oni satxl = l ; x2=I ; X3=1;
and themi ni mumval ueoffi s O. Threevari abl es arechosen because we ar notgoi ng
touse any graphi cs for illustration(i t is essenti al tofol l owthevector al gebra).
Generl c Al gorl t hm(A5. 4): The generi c al gori thmis an iterativeone and is al so
refer dt oasearchal gori thm. as thei terati ons tak pl aceby movi ng al ong a search
direction. These searchdi recti ons can be determi ned inseveral ways (Chapter 6). The
al gori thmwi thout any convergencel stoppi ngcriteriacanbe expressedas
Step 1. Choose X
o
Step2. For each iterationi
Determi ne searchdi recti onSJ vector
Step3. Cal cul atesX; =j Sj
Note: sX; is nowa functi onof thescal ar clj as Sj is known f romStep2
clj is cal l edthestepsizeas it establ i shes thel engthof s Xj
Xj 1 =X/ +s X
i
is determi ned by Mi ni mi zi ng F(Xi +l)
I i s is referredtoas one- di m nsi onal stepsizecomputati on
i i + 1; 00 To Step2
Appl i cal i on0/ Generi c A hm( A5. 4) EXI l I Tlpl e5.2
~ }
..

n
u
x

-
-
-
a
u
r
J

e
t

o

t

n

9
u
w M

V
e
b

1
I
l
l
1
t
1

1
0
0

d
r
l
l
M
L

E
B
E
B
E
E
-
-
E
E
E
J
.
0
0
6
.
g
r
i
l
l
-
L

u
.
n
J
L
z
r
E
-
'
E
E
E
E
E
L
-
x
U
1
1
1
1
1
1
1
1

M
M
M
F
1
1
1
1
1
1
1
L
=

=
'
J
g
u
A
U

3
n
r
n
r

e
e

-
-
p
a
n
a

f ( X1) = f(1) = (0- 0)2+ 2(0- 61)2+ 3(61 1)2= 72l +3( 6J - 1)2
Mi ni mi zi ng j{1) c val ue of 1 = 0. 1. Thi s is a one-di mensi onal
opti mi zati onprobl em.
Whi l e thesol uti onabove was obtai nedanal yti cal l ybecause it was a si mpl e l i near
exp ssi on epol ynomi al approxi mati onor thegol den secti onmet hod coul d have
been used inst ad.
5. 4 AODI TI ONAL EXAMPLES 219
Xl =l L
(5.16)
and soon.
h is secti on addi ti onal exampl es of si ngl e-vari abl e opti mi zati on probl ems arc
expl ored. The avai l abl ecode for thepol ynomi al approxi mati on and gol densecti on
shoul dhel psol veany and all si ngl e-vari abl eopti mi zati onprobl ems. This is l argel y
left tothereader toexpl oi t . Inthis secti ononl y extensi ons modi fi cati ons or creati ve
appIi cati ons of the one-di mensi onal opti mi zati on probl emar consi dered. First
Exampl e 5.2is revi si tedtomodi f y thegol den secti onmet hod so at it canbe used to
l cul ate S psi zeinmul ti di mensi onal probl ems. Exampl e 5.3is a sol uti ontothe
Bl assi us probl em. lis is an exampl e of the exact sol uti on to e Navi erStokes
uati on for fl owover a flat platc. Exampl e 5.4is an exami nati on of thei nequal i ty
constrai nt in Exampl c 5.l a by the gol den secti on met hod so that equal i ty and
i nequal i tyconstrai nts can be handl ed.
5.4.1 Exampl e 5. 2- l I l ustratl on of Gener al Gol den Sect l on Met hod
'he exampl e f rom eprevi ous secti onis vi si tedagai ntomodi f y egol densecti on
method to d tcrmi ne e stepsi ze for mul ti di mensi onal probl em. is book is al so
about getti ngMATLAB towor k for you. Ina scnse ssubsecti onis l argel yanexerci se
indevel opi ngthecodc will seeconsi derabl euse inthenext chapter. An ui val ent
devel opment wi th respect to e pol ynomi al approxi mati on method is left as an
exerci sefor thestudent.
'he gol dcn secti on met hod requi res the establ i shment of the upper bound.
'h reforet wo functi ons wil1 need to be changed to handl e dcsi gn vectors and the
search di recti on vectors. The code is avai l abl e Upper Bound_nVar . m d
Gol dSect l on_nVar . m. The modi fi cati ons are nOl very chal l engi ng and can be
i nferredby compari ng theusage of et wo functi ons.
For thesi ngl evari abl e:
Usage: Upper Bound_1Var (' Exampl e5_1' 0110) (5.12)
For many vari abl es:
U ge: Upper Bound_nVar ( ' Exampl e5_2' x s Ol 0) (5.17)
x: cur r ent posi t i on vect or or desi gn vect or
s: pr escr i bed sear ch di r ect i on
220 NUMERI CAL TECHNI OUE5- -THE ONE-DI MENSI ONAL PROBLEM
Of course Exampl e5_2. mretums the val u of e functi on correspondi ng to a
vector of desi gnvari abl es. lis i mpl i es thefuncti oncall toExampl e5_2. mcan
onl y take pl ac after the desi gn vari abl es areeval uated f rom e val ue of stepsi ze
appl i edal ong eprescri bedsearchdi cti onf romthecurrent l ocati onof edesi gn.
The usage of Gol dSecti on_nVar. mmust accommodat e functi on (5.17) and e
need todeal wi tha desi gnvector. Compari ng wi ththesi ngl e-vari abl ecase
U ge:
Val ue Gol dSect i on_1Var ( ' Exampl e5_1' 0. 0010110)
(5.14)
U ge:
Val ue Gol dSect i on_nVar ( ' E xampl e5_2' 0. 001x s 0110)
(5.18)
Vi si t codetos thed 15.
Compar / son w/ th Sectl on 5. 3: The code Gol dSect l on_nVar . mwas runf rom
eCommand wi ndowusi ngthefol1owi ng listing:
>>x [0 0 0 ];
>>S [ 0 0 6] ;
>> Gol dSect i on_nVar ( ' Exampl e5_2' 0. 001x s 0110)
After 11i terati ons thefinal Ji ne intheCommand wi ndowwas
ans CI
0. 1000 1. 2000
The above is i nterpretedas:
o o 0. 5973
1= 0. 1000; f (1) = 1. 2 x
1
=0; A2=0. 0;
whi ch matches Secti on5.3and Equati on(5.16).
x
3
= 0. 5973 (5.19)
5. 4. 2 Exampl e 5. 3 -Two- Pol nt Boundar y Val ue Pr obl em
A rea1exampl e for one-vari abl eopti mi zati oncan be f ound in enumeri cal sol uti on
of thel ami nar f l owover a flat pl ate[5]. leprobl em usual l y attributd to Bl assi us
represents an exampl e of theexact sol uti onof theformi dabl e Navi er-Stokes cquati on.
le mathemati cal f oml Ul ati on al l owi ng for si mi l ari ty sol uti ons is exp ssed by a
thi rd-order nonl i near ordi nary di fferenti a1 equati on wi th boundary condi ti ons
sp ified at t wo poi - a two-poi nt boundary val ue pr obl emPBVP) . What
folI ows is theessenti a1mathemati cal descri pti on. leinte tedreader can fol l owthe
detai l s inmost books on fluidmechani cs inc1udi ng suggestedrefe nce.
Mathematl cal Formul atl on
ff" + 2f " = 0
x=O; f (O)= 0; f ( O) =O
x= f ( }=1
The sol uti on nondi mensi onal vel oci tyintheboundary l ayer is obtai nedas
=f (x)
(5.20)
(5.21a)
(5. 21b)
(5.22)
The sol uti on to Equati ons (5.20) to (5.21) is l argel y through speci al iterative
techni qucs. These techni ques use a numeri cal i ntegrati on procedure l i ~
Runge- Kutta met hod to i ntegrate thesystemby guessi ngand adj usti ngthemi ssi ng
boundary condi ti ons at theiniti a1poi nt such thefinal poi nt boundary condi ti ons
arerea1ized. This is due to efact that thei ntegrati onprocedu susual l ysol veinitial
val ueprobl ems onl y. Inthe Bl assi us probl em is woul d i mpl y the mi ssi ng initial
val ue can be regarded adesi gnvari abl e.
( ' ( 0) =
The obj ecti vefuncti onthereforewi l l be
(5.23)
Mi ni mi ze: ( ) 1)2 (5.24)
I mpl i edintheformul ati onis l f () isobtai nedby i ntegrati ngthedi rential uati ons
(5.20) ( e can be consi dered as differential cons'ain) . lere is one othcr
consi derati onto takeinto account. lei ntegrati on methods requirethe ubl emto bc
expressed in state spa f ormwhi ch r n - ord differenti a1 uati on 10be
expre edas a systemof n l -ordcr uati ons. leconversi onis fairlystandardand is
done throughu l Sformati onand i nu uci ngaddi ti onal v les. For this exampl e:
y=f
y ~ =f =Y2
Y; =f' =Y3
Y; =f " =' / 2
The opti mi zati oncan be formul ated: Fi nd
Mi ni mi ze: [Y2() l f
usi ngtheRunge- Kutta met hod (or y other method) toi ntegral e
(5.25)
222 NUMERI CAL TECHNI QUE8 -THE ONE. DI MENSI ONAL PROBLEM
-
E
E
E
E
E
E
E
E
E
E
E
3 v
d
E
J
U
J
U
J
g
J
I

A
U

r
i
-
-
I
r
-
-
J
I
1
1
1
1
1
J

r
E
E
E
E
E
E
a
E
E
E
E
E
J
o
o
r
E
E
-
E
-
E
L
-
-
1
B
E
E
t
-
E
E
E
B
E
E
E
E
J

)
)
)

n
v
n
U
4
v
i
(
(
(

-
a
'
-
f
w
l
w
l
v
(5.26)
Fr ompubl i shedwor k [5J theval ueof =0.33206. I nsl eadof enal poi nt at it
is consi dered 10be 5.0. Leavi ng is val ueas a vari abl ecandefi nea correspondi ng
two-vari abl eopti mi zati onprobl em. In ei mpl ementati ontofol l ow epower d
conveni ence of MA1L AB ar readi l yapparent
Exampl e5_3. m: Thi s m-fi l e is lI sed to i mpl ement expressi on (5.25). It calls the
MATLAB built-in functi on ode45 10 use the RungeKutta 4/5 order method. The
functi onode45 requi res thestateequati ons tobe avai l abl einanother m-fi l e. It a1so
E ui res edefi ni ti onof thei nterval of i ntegrati onas wel l as theinitial condi ti on. The
fol l owi nglinef romthecod dcscri bes theusage ( ecode is welI comment ed too)
[t y ]
ode45( ' Ex5 3 st at e' t i nt val bci ni t ) ;
t is thei ndependenl vari abl eand y thedependent vector.
Ex5_3_st at e. m: The stateequati ons in(5.26) areavai l abl ein is file.
st st [ y (2) 0. 5*y( 1) *y( 3) ] ' ; y( 3)
Refer tol hecode for details. Gol dSect l on_1 Var . mis used f romthecommand line
for is probl emas
U ge:
Gol dSect i on_1Var ( ' Exampl e5_3' 0. 00100. 110)
Start i terati on:
st ar t
al phal ( l ow)
o
1. 0000
al pha( )
0. 1528
0. 1904
al pha( 2)
0. 2472
0. 0378
The fi nal i terati on:
i t er at i on 12
al phal ( l ow)
0. 3359
0. 0000
al pha( l )
0. 3364
0. 0000
al pha( 2)
0. 3367
0. 0000
al pha( up)
0. 4000
0. 0161
al pha( up)
0. 3371
0. 0000
5.4 ADDI Tl ONAL EXAMPLES 223
The resul t expected i s = 0.33206. However the tol erance speci fi edwas onl y
0. 001. The initial boundi ng i nterval was onl y 0.4. The gol den secti on has
produced an i mpressi ve sol uti on to e TPBVP. Al so note the conveni ence of
modul ar pl ' Ogrummi ng. No changes we made to Upper Bound_1Var . mor
Gol dSect l on_1 Var . mtorunthis exarnpl e.
5. 4. 3 Exampl e 5. 4 - Root Fi ndl ng wi th Gol den Sect l on
The appl i cati onof thegol den secti onmet hod totheroot fi ndi ngprobl emis i mportant
toaccommodat e cqual i tyand i nequal i tyconstrai nts. Consi der Exampl e 5.4:
g():0.75 2 _ 1. 5 1=0
0 $ $ 4
(5.27)
(5.2)
is is ei nequal i ty cons int of Exarnpl e 5.l aexcept t it is transformedtoan
equal i ty constmi nt . In is f ormthenecessary tomake it i ntoanacti veconstrai nl
can be establ i shed. The si mpl est way toaccompl i sh this is toconvert theprobl emto
a mi ni mi zati on probl emfor whi e numeri cal techni ques have al ready bcen
cstabl i shed. Squari ngEquati on(5.27) ensurcs that themi ni mumval uewoul d be zero
(if it exists). Therefore thesol uti on10Equati on(5.27) is thesame as
M mi ze: ): [0.75a
2
- 1. 5- 1] 2
(5.28)
whi ch can be handl edby Gol dSect i on_1Var . m. While is was qui ck and pai nl ess
therear some attendant probl ems usi ng this approach. the nonl i n rity has
i ncreased. This is usual l y not encouraged in numeri cal i nvcsti gati ons. Second the
number of sol uti ons has sati sfyi ngF OC i ncreased. Fi gure 5.5rep sents thefuncti ons
inEquati ons (5.27) and (5.28).
'
Sol utl on Usl ngGol dSect i on_1Var . m: The fol l owi ng di scussi on is based on
theexperi ence of attempti ngtorunthc gol densecti onmet hod wi ththel ower bound
at the val ue of 0 e case wi th e othcr exampl es. The is a probl emsi nce the
sol uti oncannot be found. The probl emapp rs tobe intheca1cul ati onof theupper
bound. lnmany of thenumeri cal techni ques thereis an assumpti on at theobj ecti ve
functi onbei ngdea1t wi this modal at is has a si ngl emi ni mumintheregi onof
i aterest. For this to happen. it is assumed at e functi on bei ng mi ni mi zed wi1l
al ways decreasef romthel ower bound. Take a l ook at efuncti onfi nFi gure5.5. It
is i ncreasi ngat thc va1ue of zero. Upper Bound_1Var . massumes that einiti a1
i nterva1is tool argeand conti nues tobi sect it tofi nda ncti onva1ue is less an
theone at thel owcr bound. Thi s dri ves thei ntervaI tozeroand thefuncti onis exi ted
wi ththel ower bound and wi th no sol uti on.
"
4
Remedy 1: The easi est way out of thedi ffi cul tyis tochangc theval ueof thel ower
bound and tryagai n. It is useful toundcrstandthat probl emis wi ththecal cul ati on
224 NUMERI CAL TECHNI QUE5 - THE ONE-DI MENSI ONAL PROBLEM
18
-
E
-
-
-
e
-
-
a
' a
-
-
a
'
'
o
a
-
-
-
-
-
a
-
-
v
J
-
M
h
e
-
e

.
16
14
12
10
(
(
H
M
)
s
'
p
-
t
'
-
-
e
4
sol utl on
2
g
0.5 2.5 3 3.5
Fl gure5.5 Exampl 5.4.
of the upper bound (bracketi ng the mi ni mum) ra er than wi th gol den secti on
method. Changi ng the l ower bound to 1 and rerunni ng th gol densecti on me od
provi des a successful sult whi ch matches egraphi cal sol uti on. Whi l e success was
real i zed. such a pr edureneeds user i ntcrventi onwel l as some knowl edge about
th probl cm- i nthis case therewas egraphi cal sol uti on_ These arenot acceptabl e
qual i ti for an automati c proc lure.
Remedy 2: Si oce the posi ti ve sl ope at the l ower bouod is i ndi cati ve of the
probl em. it shoul d be possi bl etosense this posi ti ve si gn 01 the sl ope and move the
l ower bound to the poi nt where the sl ope is negati ve. Such a poi nt can be
di scovered by thesam scanni ngprocedure. Thc sl opecomputati on is numeri cal
and is based on f orward first di fference. Oncc mor c is change inthel ower bound
is onl y needed for the uppcr bound cal cul ati on. Upper Bound_m1Var . mis the
modi f i ed upper bound cal cul ati on whi ch rel ocates the o w ~ r bound to the poi nt
where the sl ope is negati ve. The usage and i nformati on f romrunni ng the
Gol dSect l on_1 Var . mi ncorporati ng the chaoged upper bound cal cul ati oo are
shown bel ow. The samc usage wi th thc ol d Upper Bound_1Var . mdoes oot
produce any useful rcsul ts.
4
PROBLEMS 225
U ge: Gol dSect i on_1Var ( ' Exampl e5_4' 0. 0010 . 120)
st ar t
al phal ( l ow)
al pha( l )
al pha( 2)
al pha( up)
0. 9931
1. 6069
2. 6000
1. 0000
3. 0624
2. 1720
0. 0289
i t er at i on 16
al phal ( l ow)
al pha( l )
al pha( 2)
al pha( up)
2. 5266
2. 5270
2. 5273
2. 5278
0. 0000
0. 0000
0. 0000
0. 0000
REFERENCES
1. Burden R. L. andFaires J. D. Numeri cal Anal ysi s. 4thed. PWS_KENT Publishing
Company Boston 1989.
2. Hosleller GH. . sami na- M. s. . and MontaI OU- p. D. Anal yti cal Nwneri cd d
computati onal Methodshrsci ence and Engi neeri ngprmti cc-Hal l -Ensl ewoodCliffs.
NJ . 1991.
3. V 1 G. N.. Nu Opl i l 11011 Tecl mi q for Engi neeri ng Desi gl l
McGraw-Hi l l NewYork. 1984.
4. Huntl cy. H. E. The DlvIlle Proporl i on: A Sl udy in Muthemal i r : al Beauty. Dover
Publ i cauons NewYork. 1970.
5. Schl i chung. H. Bounda-Layer Theory McGr aw- Hi U. NcwYo 1979.
PROBL EMS
5.1 Extend Exampl e 5.1toi ncl udc t wo i nequal i ty constrai nts for whi ch a sol uti on
exi sts-Di spl ay fu l (
5.2 Extend Exampl e 5 .1 toincI ude t wo i n uaalityconstrai nts for whi ch t herl 1sno
sol uti on. Di spl ay thefuncti ongraphi ca11y.
5.3 Reprogram ecode Sec5_2_2. mobt ai n eval ues of f and ct' through
MA' 1. ABfuncti onprograms dappl y toExampl e 5. 1.
54 22J;;13 222:l:::Z347t t
5.5 I denti fy the val ue of wher e e constr nt (5.3) bcomes acti ve usi ng
Ncwton- Raphsoo techni que.
5.6 I mpl emcnt a bi secti on procedure for the rni ni mi zati on of e functi onof onc
vari abl e. Appl y it tosol veExampl e 5. 1.
5.7 For thepol ynorni a1apprOl dmati on i mpl ementati on cxtend c code todi spl ay
eori gi nal funcon d eapproxi mati ngpol ynomi al.
5.8 For c gol densecti oni mpl cmentati on extendthecodc todi spl aytheori gi nal
functi onand thc anal external bounds on thevari abl e.
226
NUMERI CAL TECHNI QUE5 -THE ONE. DI MENSI ONAL PROBLEM
5.9
Veri r at ebounds on thevari abl earedecreasi ngby esame ratio. Why is
this better anthebi secti onmethod?
5. 10 How woul d you set up thegol den secti onmet hod todetrmi nethezeroof
functi oni nsteadof themi ni mum.
5. 11 Combi ne e gol den secti on d e pol ynomi al approxi mati on to fi nd the
mmi mumof efuncti on.
5. 12 I mprov theaccuracy of thesol uti ontotheBI ius 1over that presented
inthetext .
5. 13 Obtai n the sol uti on to
approxi mati on method.
the Bl assi us quatl On usmg the pol ynomi al
6
NUMERI CAL TECHNI QUES
F OR UNCONSTRAI NED
OPTI MI ZATI ON
Thi s chapter iIl us ates many numeri ca1 techni ques for muJ ti vari abl eunconstrai ned
opti mi zati on. Whi l e unconstrai ned op ati on is not a common occurrenc in
engi neeri ngdesi gn neverthel ess thenumeri ca1techni ques incI uded heredemonstrate
i nteres i deas and al socapturesome of theearl yi ntensi vewor k in eareaof desi gn
opti mi tion[1]. They aI so provi de themeans tosol veconstrai nedprobl ems after they
have been transformedi ntoan unconstrai nedone (i ndi rect m ods Chapter7).
Thi s book is uni que for thepresentati onof nongradi ent techni ques (Secti on6.2)
whi ch can be empower ed by the ubi qui tous avai l abi l i ty of incrdi bl e desktop
computi ng power. Unti l very recentl y. appl i cati on of desi gn opti mi zati onrequi red
mai nf rame computers that rel i edon l argeprograms wri tteninFORTRAN. Today' s
desktopcan runprograms atreq reonl y Ii mi ted programmi ng skills. The desi refor
gl obaI opti mi zati on has brought i nto focus numeri caI techni ques at are l argel y
heuri sti c have l i mi tedneed for sophi sti catedgradi ent-basedmethods requi rel i mi ted
programmi ng ski Hs. and requi re l i mi ted programmi ng resources. Usual l y such
techni ques requi re the sol uti on space be scanned usi ng a l arge number of
iterations. Thesemethods can i deal l ybe run on personaI desktops for unl i mi tedti me.
The press of l i mi tedcomputi ng resources is no l onger a si gni fi cant constrai nt on
thesetechni ques. Whi l e they may compromi se on el egance and sophi sti cati on it is
i mportant to recogni ze opportuni ty to engage desktop resources for greater
useful ness.
6.1 PROBL EM DEF1NI TI ON
The uncons ai ned opti mi zati on probl emrequi res onl y the obj ecti ve functi on. The
book has chosen toemphasi ze anaccompanyi ng set of si deconstrai nts torestrict
227
228 NUMERI CAl TECHNI QUES FOA UNCONSTRAI NED OPTI MI ZATI ON
sol uti on to acceptabl e i acel gi on. Most numeri cal i qu in S
chaptEr agpo e ?ns ints in I ng out method. It i s u su dIy
e
Ponsib iJity0 fth 1ed e tover t hedcco nl I I nts as part of his expl orati on
of opti mum. is checki ngof csi dens nt canbe easilyprogrammed inthe
code. Thus theprobl emfor is chapter canbedefi nedas
Mi ni mi ze f (X); ] "
Subj ectto: xf $; XI $; xr; i = 12 n
6.1.1 Examp' e 6.1
; ; 2 3 1 2 : : z z ; 2 2 2 Z2 ; 2 J Ze mrpossi bl e hicaJ descri pti or
Mi ni mi ze
f ( X) =f (xl X2) = 3 + (XI - 1. 5x2l
2
+ (X2_ 2)2
Subj ect to:
OSX( S5;
OSx2S5
(6.4)
Fi gure 6.1 provi des thecontours of eprobl emand e)lutioncanbe l ocatedat
fJ i ZL 2; ar v al uei s f =3 n adrati c hesol uti on
6. 1. 2 Necessar y and Suffl cl ent Condl tl ons
2; trzz;::2jr2zs ;tt:!sZ;:?
condi ti ons) are egradl ents must vani shat thesol uti on
21=2( Xt - 15 ) =0
1 ) + 2(X2- 2) = 0
(6.5a)
(6.5b)
Equons (6.5) c beved to obtai n xj = 3xi = 2. This matches e graph
2i l j i tM223 iC :tt; s z
= l
(6.6)
(6.1)
(6.2)
(6.3)
6.1 PAOBLEMOE NITl ON 229
'jJ
T
I
l
!
l
l
'
l
l
i
4
4
w
h
H
M
l
'
I
ta--
0.5 2.5
X
1
5 3 1.5 2 3.5 4 4. 5
Fl guro6.1 Exampl e6.1.
The Hessi an is positive definite consi deri ng the deten nants alone. Sec6_1_1. m
'
provi des Edefor thesymbol i c calculationandalsodi spl ays gphi l d cription
t he probl em. The resultsof ecalculationarcdi spl ayedinthecommand wi ndow. le
satisfactionof theSOC identifies sol uti onfor thedesi gnis a mi ni mum.
6. 1. 3 El ement s of a Numerl cal Technl que
'he el ements of a typi 1num 1techni quc or al gori thm canbc associ atedwi ththe
generi c aori n in duced duri ng e di scussi on of the rel evance of the
one-di mensi ona1op ni zati onfor mul ti vari abl eprobl ems (Chapt 5). us al gori thmis
iterativeandisalsoreferredtoasa searchalgoritlvn astheiterationstakepl aceby movi ng
al onga searchdirectionor a searchvectorduri nganiteration. Thes hdirections can
bedetem .edinmany ways. The different hni ques at arep sentedinthis chapter
pri mari l ydiffer howthcsearchdir tioni stabl i shed. le mai ni ngel ements of the
algori mex pt for convergenccJ stoppi ngcriteria arethesame for al most all of
themethods. The aJ gori thmcanbe exprl scdas fol l ows.
IFileslObe downl ondedrromIhcwebsi nreindicaledbyboldracesonsserir Iype.
230 NUMERI CAl TECHNI OUES FO UNCONSTRAI NEOOPTI MI ZATI ON
General Al gori t hm a
Step1. ChseX
o
Stcp2. For each i terati oni
Determi ne searchdi recti onvector 8j
(lt woul d be ni ceif theobj ecti vedecreasedal ongthis di recti on)
Stcp3. Cal cul atesX =si
Note: aXj is nowa functi onof thescaI ar <<j as S is known f romStep2
is cal1ed estepsi zeas i ttabl i shes el engthof AXj
j is dtermi nedby Mi ni mi zi ngf ( whcreX' +I = Xj +X1
Here the convergcnce criteria ( FOC/ SOC) and thestoppi ngcritcria(I s
desi gnchangi ng? Exceedi ng i terati oncount? etc.) must be checked.
i i + 1;
OoTo Step2
ln this chapter this al gori thmi c structure will be uscd for aI l of the methods. 'he
one-di mensi onal opti mi zati onwil1be i mpl emented usi ngthegol den secti onmetbod.
Exampl e 6.1 wilbe used todcvel op thc di ffercnt numeri cal methods.
6. 2 NUMERI CAL TECHNI QUEs - NONGRAOI ENT METHOOS
'hesemethods areaJ so caJ l edzero-order methods. 'he order refcrs to eordcr of thc
deri vati ve of theobj ccti ve functi on necded to establ i sh thesearchdi recti onduri ng
any iteration. Zero order si gni fi es that no deri vati ves used [2] whi ch in t um
i mpl i es onl y functi on val ues are used to S bl i sh tbe search vector. There is
another i mportant si gni fi cance that accompani es tbe mi ssi ng deri vati ve
computati on- the F OC shoul d not be appl i ed in these methods. If wc were
computi ng deri vati vcs we mi ght as wel l incude this i nformati onfor establ i shi ngthe
searchvector. Onl y thechanges in eobj ecti vc functi onor thedesi gnvari abl es can
provi d convergence d / or stoppi ngcriteria.
etcchni ques arei ncl udedinthis secti on. The first is a heuri sti c one basedon
randomsearchdi recti ons. The sccond one cycl es througha searchdi rccti onbasedon
each varl abl c. le last is Powcl l ' s met hod whi ch has tbc property of quadrati c
convergence.
6. 2. 1 RandomWal k
The searchdi recti onduri ngeach i terati onisa r andomdi recti on. Randomnumbers and
sets of randomnumbers areusual avai1abl ethroughsoftware. MATLAB i ncl udes the
ab ty to generate severaJ typcs of r andomnumbers and matri ces. Most
computer-generatedr andomnumbers ar cal l edpseudorandomnumbers as they cycl e
after a suffj ci ent number of t hemhav been created. The one di mensi onal stcpsi ze
computati on is donc by thc Gol dSect l on_nVar . m ncti on(Chapter 5).
62 NUMERI CAI. TECHNI OUE - NONGAAOI ENTMETHOOS 231
H
i
l
A orl thm: RandomWal k (A6. 2)
Step1. Choose X
o
N(number of iteratlons)
Set i = 1
Step2. For eachiterationI
8
1
= Randomvector
Step3. Xj +
1
= Xj +Sj
'j is determi nedby mi ni mi zi ngj{Xi+I)
i i + 1
lf i < N 00 To Step2
el sc SP (i rati ons exceeded)
EZ2222zte szz:;12rz::;; i
l; f: ;
si gnbased on a test of randomnumber.
RandomWal km: Tl mr andomwal k al gori thmt A62) i s i mpi ementcd in this
OP6mi adon- TMcodeof the randomwal k functi on al so i ncl udes thE fol l owi ng
1'1
~
~
econtour of thc obj ecti veis drawn.
For t wo- vari abl eprobl ems tn
For t wo vari abl Eprobl ems thesuccessful i terati ons Me trackedon thecontour
d theiteration" number identified.
. 2; : u16v m d 10 C dfor
theone-di mensi onal stepsi zecomputati on.
. The mul ti vari abl eup rboundcal cul ati on(devel opedinchaptcr5)i s used for
. ;?::; E ::: 2:zr1;z;;11i In1 here 0 h d di iI1 r ec
.::: i : :;: e L i gnvari abl es and cti ons for all of
thc i tcrati ons arc pri nted.
232 NUMERI CAl TECHNI OUES FORUNCONSTRAl NEDopnMI ZATI ON
Use of previ ous code iII us ates code reuseand modul ar programmi ng. Once more
r ders ar strongl yencouraged 10visit thecode tounderstandth ansl ati onof the
aJ gori lJ l mand subsequent appl i cati on.
U ge:
Z
RandomWal k( ' Exampl e6_1' [ 0. 5 0. 5] 200. 0010120) (6.7) ~
I nput: (They areth i tcms encl osedwi thi ntheoutermost parenthesi s inexpressi on
(6.7).) a
' Exampl e6_1:
[0. 50.5]
20
0. 001
0
20
lCfileExampl e6_1. mwhere lJl e obj ecti veis descri bed
starti ngdcsi gnvecl or (dvarO)
number of i terati ons of themet hod (niter)
tol erancefor egol den secti onproced e(tol)
initial stepsi zefor scanni ngupper bound (I owbound)
sl epi nterval for scanni ng(intvI)
number of scanni ngsteps (n aIs)
Output:
A vector of n + 1 vaJ ues of desi gnvari abl es at thelast iterationand thevaJ ue of the
obj ecti vc functi on.
Executi ng functi on (6.7) in lJl e Command wi ndowproduces the fol l owi ng
i nformati on( ecomment s areadded later):
- The desi gn vect or and f unct i on dur i ng t he i t er at i ons
0
0
1
1
1
1
2
2
8
8
4
4
4
4
4
4

0
0
9
9
9
9
8
8
0
3
0
0
0
0
0
0

u
n
u
n
u
n
u
n
u
n
u
n
3
n
E
n
o
n
o
n
u
n
D
n
D
=
R
J
R
d
'
J
n
u
n
v
n
u
n
u
n
u
n
u
n
u
....
4
m
0
0
0
0
0
0
0
0
2
3
3
3
3
3
3
3

a

0. 5000
0. 5000
1. 0440
1. 0440
1. 0440
1. 0440
0. 9756
0. 9756
2. 1093
2. 0481
2. 0350
2. 0350
2. 0350
2. 0350
2. 0350
2. 0350
5. 3125
5. 3125
4. 6481
4. 6481
4. 6481
4. 6481
4. 6349
4. 6349
3. 0663
3. 0025
3. 0020
3. 0020
3. 0020
3. 0020
3. 0020
3. 0020
st ar t
di d not move
decr eased f unct i on
di d not move
di d not move
di d not move
decr eased f unct i on
et c
6.2 NUMERI CAL TECHNI OUES - NONGRADI ENT METHODS 233
3. 0804
3. 0804
3. 0804
3. 0804
ans
3. 0804
2. 0350
2. 0350
2. 0350
2. 0350
3. 0020
3. 0020
3. 0020
3. 0020 20 i t er at i ons
2. 0350 3. 0020
Compare vaI ut o e actual uti on of xi =3xi =2j* =3. Ts is
remarkhl efor a method at has no senseof preferreddi recti on. If theexecuti on15
resated wi th Menucal i nputm ent i mnewsol uti onwi l l beobtai ned. Fi gure62tracks
th; Iges of the d nvari son theco l f pl ot. The movem nt of ilie si gn
vari abl es are pai red wi th the corzespondi ng i terati on number . Ther andomwal k
functi on can bEExECUted for several di ffErent starti ng sets of desi gn vari abl es.I t
shoul dnot di sappoi nt. The Gol dSect i on_nVar . mis esarneone as inChapl er 5
except iliepri nti nghas been suppressed.
Whi l e executi ng e RandomWal k. mthe pri nti ng of i terati on mb ?
cmates a mess as you approach thesol uti on. You can swi tchit of f by comment t qgme
( )statement- Thgeodei nRandomwal k- mal soseI ves as a templ ateforother
al gori i l i ms. lt is worth expl ori ngindetai l.
k
0.5
1.5
Fl gure6.2 Randomwalk: Exampl e6.1.
i
j
L
i
l
i
-
-
1
1
J
i
l
l
234 NUMEAI CAl TECHNI OUES FO UNCONSTAAl NEDOPTI MI ZATI ON

6. 2. 2 Pattern Sear ch
The PattemSearchmet hod is a mi nor modi fi cati on10l he Utlivariatemet hod wi tha
maJ or I mpact . As the PattemSearch uses the Uni vari ate met hod the latter is not
di scussedseparatel y. IntheUni vari atemet hod a1so known theCycl i c Coordi nal e
Descenl method each desi gn vari abl e (consi dered a coordi na) provi des a search
di recti on. Thi s is a1so refe edto as a coordi natedi recti on d is easi l y expressed
throught h1eu n 1
n umbr o f variabless
desig nvari abl es) i teratio ns onef or achof then searchdi recti ons. It can be shown bv
appIi cati on t for probl ems wi thconsi derabl enonl i nea theUni vadatemet130J
tends toget l ock i ntoazi gz agp attEemof sma ma
sol uJt 110 n.
l l ePat mSearchprc dureattempts todi srupt thiszi gzagbebavi or by executi ngone
addi ti onal iterationfor eachcycl e. Ineachcycl e at eend of 11Uni vari atedi recti ons the
n+l s ea h di recti on is assembl 1as a I II n 1 ar c o m 11b
dc ons deopt i mum u eof t hle p ze fo at direction. one-di m ional
opti mal stepsi zeis enmputed dthenext cycl eof iterationbegIl1s.
A ori thm: Pattern Sear ch 6.3)
Step 1. Choose X
1
N
c
(number of cycI es)
j (1) =j ( X) ; X
c
(1) = X
E E2: tol erancefor stoppi ngcriteria
Setj = I (initializecycl ecount)
Step2. For eachcycl ej
For i =111
S; =(uni vari ate p)
X; + =X; +
/ is determi ned by mi ni mi zi ngj (X; +)
(storeval ues of.i and S/)
cnd of For l oop
n
= : ; S; aX - X (Pattcm
Xj =Xn+ + CiSj
X+1) Xs J "+1) =j () (stor cycl eval ues)
Step3. 6j =1rU+ 1) - 1r A X = X.u+I ) - X
c
If l.1jl S E" stop
I f AX
T
.1xS stop
If j =N" stop
X ~
6.2 NUMEAI CAl TECHNI OUES- ONGAAOI ENTMETHODS 235
j j +l
GoToSt ep 2
ThePatternSearchmethodhasaf cwaddi ti onal stoppi ngcri tcri a. Thcscarebasedon
howmuch thef uncti on i sdecreasi ngfor each cycl eand how much change istaki ng
pl acei nthcdesi gn var i abl ei tsel f. For thel atter thi sIl1f ormati on i sbasedon thel ength
of thechangei n thedesi gn vector for thecycl e. Onceagai n theapproach i ssi mpl eand
the i mpl ementati on is di rect. Pr ogr ammi ng such a techni que wi l l not bc too
forbi ddi ng.
Pat t ernSearch. m: To conlI nue eexposure 10MATu. B programmi ng i smethod
is i mpl emented very di fferent 1y f romtheother al gori thms that have bcen transl ated
intocode sofi . It is a stand- a1one program. Therc areprompts for er i nputs wi
appropri ate defaul t val ues in cas the user deci des not to enter a val ue. The
one-di mensi onal stepsi ze computati on is i mpl emented by e mul ti vari abl e gol den
secti onmcthod. The i nteresti ngfeatu sareas fol1ows.
. It is a freestandi ngprogram.
lefilecontaIl1i ngtheobj ecti vefuncti onis sel ectedthrougha list box.
Scveral programcontrol par neters are entered f romthe keyboard after a
sui tabl eprompt i ndi cati ngdefllult val ues.
Dcfaul t val ues arc initializedif theuser chooscs not 10entcr theval ues.
Thc desi gnvcctor is trackedand pri ntcdaftcr thc programis stopped.
Si nce theupper bound cal cul ati ons arc based on posi ti vc val ucs of stepsi ze thc
search di recti on is vcrs if t umed stepsi ze is e same as the l owcr
bound.
Iterati oncounts arerecorded.
For two-vari abl eprobl emtheobj ecti vefuncti oncontours arepl otted.
. The first ni ne or Icss i terati ons aretracedon contourpl ot . Thc Uni vari ateand
thepattemsteps areco)or codcd.
U ge: Pat t er nSear ch (6.8)
The prob)emis Exampl e 6.1. lefi]eis Exampl e 6_1. m.
Out put ( MATl .AB Command wi ndow): Wi t h e defaul t va)ues and the starti ng
vector [43] thefol l owi ngarewri ttentotheCommand wi ndow.
>> Pat t er nSear ch
The f unct i on f or whi ch t he mi ni mum i s sought must be a
MATwa f unct i on M - Fi l e. Gi ven a vect or dependent var i abl e
i t must r et ur n a scal ar val ue. Thi s i s t he f unct i on t o be
MI NI MI ZED. Pl ease sel ect f unct i on name i n t he di al og box
and hi t r et ur n:

236 NUMEAI CAL TECHNI OUES FOAUNCONSTAAI NEDOPTI MI ZATI ON
The f unct i on you have chosen i s : : Exampl e6_1
maxi mumnumber of cycl es [ 1000] :
conver gence t ol er ance f or di f f er ence i n f [ l e 8] :
conver gence t ol er ance on change i n desi gn x [ l e- 8] :
I nput t he st ar t i ng desi gn vect or . Thi s i s mandat or v as
t her e i s no def aul t vect or set up. The l engt h of our
vect or i ndi cat es t he number of unknowns. Pl ease ent er i t
now and hi t r et ur n:
[4 3]
The i ni t i al desi gn vect or [ 4. 00 3. 00]
Conver gence i n f : 3. 999E 009 r eached i n 10 i t er at i ons
Number of usef ul cal l s t o t he Gol den Sect i on Sear ch
Met hod: 31
Tot al number of cal l s t o t he Gol den Sect i on Sear ch
Met hod: 40
The val ues f or x and f ar e
4. 0000
4. 4992
4. 4992
4. 3868
4. 1432
4. 1432
3. 8229
3. 3338
3. 3338
3. 1750
3. 1980
3. 1980
3. 1944
3. 1473
3. 1473
3. 0346
2. 9958
2. 9958
2. 9971
2. 9972
2. 9972
2. 9972
2. 9977
2. 9977
2. 9999
2. 9998
3. 0000
3. 0000
2. 6924
2. 7616
2. 7616
2. 5287
2. 2224
2. 2224
2. 1543
2. 1321
2. 1321
2. 0916
2. 0980
2. 0980
2. 0682
1. 9969
1. 9969
1. 9981
1. 9981
1. 9981
1. 9986
1. 9988
1. 9988
1. 9990
1. 9999
1. 9999
4. 2500
4. 0000
3. 6916
3. 6398
3. 5801
3. 4022
3. 2890
3. 0494
3. 0343
3. 0180
3. 0175
3. 0121
3. 0118
3. 0096
3. 0067
3. 0016
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
0
0. 4996
0. 3081
0. 2254
0. 2446
0. 2339
1. 3148
0. 4896
0. 0681
0. 3248
0. 0234
0. 0405
0. 1579
0. 0471
0. 0298
2. 3916
0. 0391
0. 0013
0. 0322
0. 0001
0. 0005
0. 4563
0. 0005
0. 0002
4. 9305
0. 0002
6.2 NUMEAI CAL TECHNI OUE9- NONGAADI ENT METHODS 237
2. 9998
2. 9998
2. 9998
2. 9998
2. 9998
1. 9999
1. 9999
1. 9999
1. 9999
1. 9999
i
t
i
l
l
-
-
0. 0000
0. 3398
0. 0000
0. 0000
0. 2020
3. 0000
3. 0000
3. 0000
3. 0000
3. 0000
To veri fy eprogramis executi ngthea1gori thm sc thedesi gnvecl ors E
first l Wo col umns. The uni vari al esteps can b recogni zedwh none component of the
vec10r changes whi l e other mai ntai ns its val ue. The pattern di recti ons can be
;1. recogni zedwhen bothcomponents change.
~ The number of i terati ons is a si gni fi cant amount . One reasonis that thetol erances
aremuch smal l er. It can be notedthat di fferencebel ween l wo i l erati ons ch ges
I i nearl y. Such methods are expec1ed to run for a l arge number of iterations. The
1 : programcon 01par ters control theacc cy of eresul ts as wel l as enumber
of iterations. The sol uti onis mor e accurat thantheRandomWa1k me od.
Fi gure 6.3 iII ustral es e a1gori thmgraphi cal l y. The uni vari ate and the paUern
di recti ons arequi tedi sti ngui shabl e. When executed on your machi ne th steps are
expl i ci tl y col or coded. Readers areencouraged 10go over thecode. and modi f y i1to
E
4.5
0.5 1.5 3
Rgure 6.3 Pal l emsearch: Exampl e6.1.
238 NUMEAI CAL TECHNI QUES FOAUNCONSTAAI NEDOPTI MI ZATI ON
refl t th ir preference. The number of l i nes is around 250. We at rti ngtoget i nto
senous programmmg.
6. 2. 3 Powel l ' s Me' od
lf there were onl y one zero-order met hod must be progranuned the
overwhel mi ng choi ce woul d be Powel1's met hod [3]. The pri nci pal reason for e
deci si on woul d be that it has the property of ql l adrati c convergence namel y for a
quadrati c probl emwi thn vari abl es convergenc wi l l be achi evedinless thanor equal
to n Powel1 cycI es. A quadrati c probl emis an uncons i ned mi ni mi zati on of a
functi on that is expressed as a ql.drati c pol ynomi al - a pol ynomi al wi th no term
havi ng a degree greater than two. Exampl e 6.1 is an exampl e of a quadrati c
pol ynomi al in t wo vari abl es. Engi neeri ng desi gn opti mi zati on probl ems are rrel y
descri bedby a quadrati c pol ynomi al . sdoes not i mpl y that you cannot use Powel1' s
method. What this means is that the sol uti on shoul d not be expected to converge
quadrati cal l y. For nonquadrati c probl ems as the sol uti on is approached i terati vel y
eobj ecti vecan be approxi mated very wel l by a quadrati c functi on. It is at is stage
at thequadr cconvergenc propertyis reaJ i zedinthecomputati ons.
The actual aJ gori thm( A6.4) is a si mpl e (the wor d is not bei ng used Ii ghtl y)
modi fi cati ontothePattemSearch al gori thm. l neach cycJ e (a er efirst) i nstead
of usi ng uni vari ate di recti ons in the first n i terati ons. the search di recti ons are
obtai ned f romthe previ ous cycJ e. The newsearch di recti ons areobtai ned by I
shi fti ngthedi recti ons of the previ ous cycJ e. In is way a hi story of the previ ous
searchdi recti ons is used toestabl i sh the next . Wi t h this change. Powel l ' s met hod
shoul d converge at most in three cycJ es for Exampl c 6.1 rather than the several
listed intheprevi ous secti on.
A ori thm: Powel l ' s Met hod 6. 4)
Step 1. Choose X" N
c
( number of cycl es)
Jt l) = j{X
1
); Xc{l ) = X
E E2: tol erancefor stoppi ngcriteria
Setj = 1 (initializePowel l cycl e count)
For i =1 n
S; (uni vari atestep)
Step2. For each . cycl ej
For i =l . n
If j 2S; S;+
X;+ =X; +I SI
; is detenni ned by mi ni mi zi ngJtXi+')
end of For l oop
Sj =S; + =I ;S; = X
n
+ -XJ ( pattemstep)
62 NUMEAI CAL TECHNI QUE ONGRADI ENTMETHODS 239
1
1
1
f
H
{
Xf =Xn+ +j
XcU+1) Xf ; f cU+ f (Xj) (s cycI e
Step3. !:if=f cU+ 1) U); d1{ = XcU + 1) - Xc(j)
lf ldfl S E" stop
lf d1{T dX S E2. stop
If j=N
c
sp
X X
J
j
j +l
GoToSt ep2
APPMeat t on of poweI I ' s Met hod: The transl ati onof theal gori thmi ntoMATLAB
COdei sl ef t as m exerci sefor thestudent. Step-by-stepappl i caGon t oExampl e 6.1 is
shown here. Exampl e 6.1 is res das
Exampl e6. 1
Mi ni mi ze f(X) = f (x" X2) = 3 + (x -1.
5 X
2)2+ (x2 (6.3)
X E =
S =[ 6] ; S2
(6.4)
j =l
Step2. j = 1
X2 4 3 1 +l = ? ? l
2. =1/4; X2= l

4 4 4 6 1 5 ; X 4 z j ; ) =4. 5576
S) = 0. 25[6]
X4=[ 03251 i 5]
i = 2:
i =3:
241 6.3 NUMERI CAL TECHNI QUEs - GRADI ENT.BASEDMETHODS
i
l
i
l
l
i
t
-
-
-
Inlight of theprevi ous definition. thesewoul d bereferredtoasfi rst. order m ods.
The searchdi recti ons will beconstructedusi ngthegradi ent of theobj ecti vefuncti on.
Si nce gradi ents are bei ng computed. e Kuhn- Tucker condi ti ons (FOC) for
unconst nedprobl ems. Vf = O. canbeusedtocheck for convergence. The SOC are
hardl yever applied. One of thereasons is at it woul d i nvol vethecomputati onof an
n X 11 second deri vati ve matri x whi ch is consi dred compu tionally expensi ve.
particularlyif eeval uati onof the0 tivefuncti on qui resa call toa finiteel ement
method for generati ngrequi r i nformal i on. Another reason for not cal cul ati ngthe
Hessi anis at theexi sl enceof esecondderi vati veina real desi gnprobl emis not
certaineven thoughit is computati onal l y possi bl eor feasible. For probl ems that can
bedescri bedby symbol i c calculations. MA11. ABshoul dbeabl etohandl ecomputati on
of secondderi vati v at thepossi bl esol uti onandi ei genval ues.
Wi thout SOC thesemethods requi reuser' s vi gi l ancetoensure thesol uti on
obtai nedis a mi ni mumrather ana maxi mumor a saddl epoi nt. A si mpl way 10
verify is is to perturb the obj ctive functi on through pe urbati on in th desi gn
vari abl es at thesol uti onand verifyit is a local mi ni mum. Thi s bri ngs up ani mportant
prope of theseme ods- t heyon1y f i nd l ocal opti mums. Usual l y this will be c10se
to th desi gn where the iterations are begun. Before concl udi ng the desi gn
expl orati on. it is necess Y toexecute the method f romseveral starti ngpoi nts to
di scover if other mi ni mums xist and select the best one by head to head
compari son. The bul k of exi sti ng unconstrai ned and constrai ned opti mi zati on
methods bel ongto is category.
Four mcthods arepresented. The first is thc Steepest Descent method. Whi l this
method is not usedinpractice. it provi des anexcel l ent exampl e for understandi ngthc
al gori thmi c pri nci pl es for gradi ent-basedtechni ques. The secondis theConj ugate
Gradi ent techni quewhi ch is a classical workhorse pa cularlyini ndus usage. The
thirdand fourthbel ongtothecategoryof Vari abl e Metri c methods. or Quasi - Newt on
me odst heyarealsocalled. These methods havc beenpop for some ti me. d
will possi bl ystaythat way for a l ongti metocome.
Thc gcneral probl emand specific exampl e ar producedfor conveni ence
NUMERI CAL TECHNI QUEs - GRADI ENT- BASED METHODS 6.3
240
NUMERI CAL TECHNI QUESFORUNCONSTRAI NEDOPTl MIZATl ON
L=11:570;
f ( X
4
) =4.4843
=- 0.14111'' Y .
8558
1
1. 2 = 11. 01041; f ( X
2
) =4.4145
; =0.4235;
Step3. l.df l = 1.353>EI' conti nue
ldX
T
dXl = 1.8992> Ez. conti nue
} = 2: One cycl eover
GoToSt ep2
Step2: }=2
xJ 0. 855
J =11 1 c' 7n 1 :
11.l 570 l '
M
-
-
g
u
111iZ] ; S 2 2=[ 0 =[ 0 0 22 5J =0 4 0 3 5
i =3:_
v _10. 95671 r 0.1011 r'l l
- U. 1966 J = /2/; f ( X
4
) = 3
: : i ; ; 32! emd C utati ons in iercise stopped
i t E : 2 7 ; : J r : at J ; ; : : : : woul dit here? You agedto
- l e method onl y took two cycl es toconverge (as expected)
T heextent of compu onis sameas thePattemS chb lenumberof
iterations is significantlysmal 1er.
2: ; ; 212S22; : r s econdml ei S tive. H E n ce
The stepsi ze for the last itcration is 20 . The supporti ng al gori thmfor
; ; : : : !1gand gol densecuonmust be abl etO tp the mi ni muma t elar
Whi i l eno oti l h u Islm t dhmerE.th hel saxi ml 3 u 1m n m Ilu umbe ?r '0 f cyc1es toconver:ge n 1ceflor
a
z::rL ::::z21;f: : f g poi nt . Y
(6.1)
(6.2)
(6.4)
(6.3)
j (X); [X]"
i =I . 2. . . . . n
j (X) = f(XI . X2) = 3 + (XI - 1.5x2)2+ ( X2 - 2)2
OSXI S5: o S X 2 S 5
Mi ni mi ze
Subj ect
Subj ect to:
Mi ni mi ze
2jz:32ifi:5132ti: ;
matri x. lf sea di recti ons S . j and dieco n E
c on.ugacyarc
Steepest Descent Met hod
This method provi des a natural evol uti on for the gradi ent based techni ques [4].
The gradi ent of a functi onat a poi nt is thedi recti onof themost rapi di ncreasein
6.3.1
(6.9)
SJ[ U] Sj =0
Veri if it is truc for thcexampl e.
242 NUMERI CAl TECHNI QUES FORUNCONSTRAI NEDOPTI MI ZATI ON
6.3 NUMERI CAl TECHNI QUEs - GRADI ENT-BASEDMETHODS 243
theval ue ofthefuncti onat tllat poi nt. The descentdi rcti oncan be obtai ned reversi nl!
thegradi ent (or mul ti pl yi ngit by -1). The next stepwoul d be toregardtl1e descent
vecl or as a search di recti on after all we are aUempt i ng 10 decrease tl1e functi on
Out put (wri ttento eCommand wi ndow): A coupl e of i terati ons at estart and at
EEnd(out of 20i terati ons)Me copi ed b-l ow.
through successi ve i terati ons. Thi s seri es of steps gi ve riseto eSteepest Descent
al gori thm j
A ' ori thm: Stl P t Descent 6.
Step J. Choose XJ N (number ofi terati ons)
!.(l ) = j {X1); X. (l ) = X
1
(storeval ues)
EJ E2' EJ: (tol erancefor stoppi ngcriteria)
Set i = 1 (i niti a1i zei terati oncounter)
Step2. Sj = - Vf (X/ )( is is comput ed inStep3)
X;+l
is determi ned by mi ni mi zi ngf (X; +I)
Xs(i + 1) X 1; (i + 1) = f ( X
i
+
l
) (storeva1ues)
Step3. aJ =!. (i +l ) a X= Xs(i + 1) - Xs(i)
Ifl aJ l (functi onnot changi ng)
If a X
T
a X E2; stop(desi gnnot changi ng)
[fi + 1 = N; stop
If Vf( X; +l l Vf (X
i
+
l
) E3; converged
i i +1
GoToSt ep2
St eepest Descent. m: Thi s is an m-fi l e a1execut a1gori thmA6. 5. It uses e
gol den secti onand tl1e upper bound scanni ngPI 1 ess. le ares ofthi s programare
as fol l ows.
For t wo vari abl es it wiJI drawtl1e contour pl ot .
For t wo vari abl es the desi gn vector changes can be seen graphi cal l y in sl ow
moti on wi tl1steps indi fferent col or.
The desi gn vari abl es efuncti on va1ue and e SqUar of tl1e l engtl1 of
gradi ent vector(ca11edKTval ue) at each i terati onaredi spl ayedintl1e Command
wi ndowat compl eti on of tl1e number of iterations.
The gradi en1of tl1e functi onis numeri cal l y comput usi ngfirst f orward fi sI te
di fference. The gradi cnt computati on is tl1erefo automati c.
Fr oma programmi ng perspecti ve basi c stri nghandl i ngtochange linecol or is
i ntrocl uced.
Usage: St eepest Descent ( ' Exampl e6_1' ( 0. 5 0. 5]
200. 00010120) (6.10)

j
0. 5000
0. 5000
5. 3125 5. 2969
0. 5996
0. 9483 4. 7832 2. 8411
1. 0701
0. 8429 4. 3766 3.1379
2. 6901
1. 8659 3. 0298 0. 0507
2. 7455
1. 8502 3. 0233 0. 0462
2. 7585
1. 8960 3. 0181 0. 0316
Fi gure 6.4rep sents thegraphi cal rnoti onof thedesi gn vari abl es for 20 i tErati ons.
The graphi cal pi ctureand thechanges inthedesi gnarEnot i mPEESSi VETheSteepest
Descent mEt hod iSWOEf ul l y MadequatE Compazed toPowel l -s met hod even i fthelatter
is a zero-order metl1od. MO[ 1 ver is concl usi on is dr awn wi th rpect lo an sy
probl em(quadrati c) Exampl e 6. 1. is perf ormance j usti fi es tl1e l ack of seri ous
interest intheStecpest Dcscent method.
5
4.5
0.5
gure6.4 Sl e pest desnt : Exampl e 6.1.
244 NUMEAI C LTECHNI QUES FOAUNCONSTAAI NEDOPT1MI ZATl ON
r J ; ? 12: :JI :241 2EZ
xpected to overcom this pattem. In thecase of the Uni vari ate m odthis was
achEeyed through patFrn Search met hod in the zero-order f ami l y. An i terati on
breaki 0 of the zi gzag pa m (or preventi ng getti ng l ocked i nto oInEPi5
::: : r r o: liz::: : 22:eme e od Su ch
6. 3. 2 ConJ ugat e Gradi ent (Fl etcher- Reeves) Met hod
TheCo ugateGradi ent method ori gi nalI y due F1etcher and Reeves [5] is a smaI l
e izi i ; ?:; ; 5 525g:;:2;qi:?:1izi5;;
c onv erge nc b ecau sethe s earchd l Ir.ec tu1o E n 3s aMr c01 Ij j ugate w ith sp ttotheHessi an
::?xi : E: :: lu uo n. A qu la
A 'or/l m: ConJ uga Gradfj t a
Step1. Choose X" N (number of i terati on)
!s(l ) =j ( X); X. (l) = X(storeval ues)
E E2E3: (tol erancefor stoppi ngcriteria)
Sel i = I (i nilializeiterationcounter)
Step2. If i =I Sj =- Vj ( X
j
)
El ses= ' (xj f Vj (X
j
)
Vj ( Xj_l) T Vj ( Xi_')
Si =- Vj ( ) + sSj _
X' +I =X/ +t
I is determi nedby mi ni mi zi ngj (X
i
+)
X.(i + 1) X; +; (i +1) =j ( X ) % (s V s)
S 3. !J.j =f s(i +l ) 1) ; X=X. ( i +1 X. (
Ifl !J.j l s EI; (fu mJn n E c ti i onnE
If !J. X' !J. X: $E2 stop (desi gnnot changi ng)
I f i +I =N; 5top
I f Vj (Xi +f Vj ( X) :$E3; convergd
i +1
GoTo Step2
c; : : P22J t : : 2J Ul z ; i t t r t t ; 1 2 3 t z r : r n ;;;:;
6.3 NUMERI CAL TECHNQUEs - GRAD' ENT. BASEDMETHODS 245
computati on--s. and th succes5i ve adj ustment of search di recti on
i ncorporati ngthis val ue. represents therati oof thesquare of thecurrent gradi ent
vector tothesquare of theprevi ous gradi ent vector. The first thi ngtonOl i ceis at
a degree of robustness is bui l t i ntothe met hod by carryi ngi nformati onf romthe
previ ous iteration. Thi s is Ii ke mai ntai ni nga hi story of themet hod al bei t for j ust
one i terati on. Si nce the F OC is based on the l ength of the gradi ent approachi ng
zero at thesol uti on. is P ti cul ar f onn of i ncorporati onof is i ngeni ous. If the
previ ous i l erati onis cl osetothesol uti on. thens is l argeand previ ous i terati on
pl ays a si gni fi cant rol e in the current i terati on. On the other hand if s is l arge.
suggesti ng the sol uti on is still far away then the current val ue of the gradi ent
determi nes thenewsearchdi recti on.
Appl l catl on 0 ' ConJ ugat e Gradl ent Met hod: This method is i mportant to
warrant worki ng through the iterations. These caI cul ati ons si mi l ar to Powel l ' s
mthodcanbe done usi nga hand caI cul ator. usi nga sp adsh tor usi ngMATLAB
itsel f. The author recommends thecaI cul ator as eprocess is sl owenough todevel op
an i ntui ti on worki ng wi th the pal temof numbrs. In the fol l owi ngand unl i ke
Powel l ' s mthodon1y the rel evant caI cu1ati ons are recorded. The reader shoul d
fol l owaI oog veri fyi ngthenumbers arei odeedcorrect.
10.51
Step 1. X1 =1 o:sl; f (X) = 5. 3125
1 0.5 1 _. ___ . . . . . ___. __ 10. 59961
Step2. S = = 0. 1993; f(i ) = 4. 7831; X2 =1 O: 94841
t = 2 - 1 . 6 4 5 9 1
Step2: s = 0. 5351
11. 91351 131
2 = I O: 83831; i = 1.2544; f( ) =3. 0; X3 =121
Con) ugat eGradl ent . m: This m-fi l ewill executetheConj ugate Gradi ent method.
For two ari abl e probl ems there will be a contour pl ot over whi ch e iterative
progress inthedesi spaceis recorded. The property of qua 8tiCconvergence can
be observed the sol uti on is obtai ned after t wo iterations. This is defi ni tel y
i mpressi ve compared wi th St pest Descent m od. K p in mi nd at the
modi fi cati ontotheaI gori thmis mi nor. As evari ous codes areruninorder. some
changes inthefi gurecanbe observed. Thesearelarg lydue totheuseof some string
processi ng functi ons. Y ou are encouraged to see how ey are i mpl emented. The
author is certai nthereare i ngs you woul d do di fferentl y. Do not hesitat to yas
is theon1y way tol earntouse MATLAB eff ctively.
U ge: Conj ugat eGr adi ent ( Exampl e6_1' [ 0. 5 0. 5] (6.11)
200. 00010120) ;
246 NUMERI CAL TECHNI QUES FO UNCONSTRAI NEDOPTI M ATI ON
Out put ( Command wi ndow)
The probl em: Exampl e6_1
The desi gnvector functi onval ueand K T val ue duri ngthei terati ons
0.5000
0.5996
2. 9995
0.5000
0. 9483
1. 9982
5.3125
4. 7832
3. 0000
0. 2490
2. 8411
0. 0001
Keep in mi nd the sol uti on is dependent on the tol erance for the one-di mensi onaI
stepsi ze searchand that thederi vati ves arecomput ed nurneri cal l y. The executi onof
ecode shoul ddi spl ay Fi gure 6.5inthefi gurewi ndow.
6. 3. 3 Davl don- Fl et cher- Powel l Met hod
The Davi don-Fl etcher-Powel l ( DFP) [6] met hod bel ongs to e fami l y of Vari abl e
Metri c Met hods ( VMM) . I t was nrst i ntroducedby Davi don and seVEral yEMB later
was devel oped in its current f ormby Fl etcher d PowelI. Bi ng aware of the
Conj ugate Gradi ent met hod thesemet hods woul d not merI t incI usi oninthis book if
they di dnot have theproperty of qu. adrati c convergence whi ch they do. General l y
2 3.5 4 4.5
Fl gure6.5 Coni ugategradl ent method: Exampl e 6.1.
5
6.3 NUMERI CAL TECHNI QUE - GRADI ENTBASEDMETHODS 247
this far lyof met hods go beyond that. As thesol uti onis approached they behave l i ke
Newton' s met hod (Secti on6.4). The quadrati c convergence of Newton' s met hod is
qui l ei mpressi ve- i t wi l l l ocate sol uti ontoa quadrati c probl eminolleiteration.
Thi s Newt on- I i ke behavi or of theV MM as thesol uti onis approached has gi ven t hem
anot hernam termedQuasi -Newtol l or Newton-l i ke methods.
The DFP met hod is presentedhere for hi stori cal i nterest d because it is a little
easi er tounderstandthan others. I nfact Huang (7) documents a generi c procedure
f romwhi ch most of epopul ar methods can be obtai ned and f romwhi ch you coul d
al soderi veyour own.
The Conj ugateGradi ent method' s i rnprovement over theSteepesl Descent met hod was
possi bl e because of the i ncl usi on of the hi story m the previ ous iteration. ln the
quasi -Newton methods th hi story f romaIl previ ous iterations is avai l abl e. Thi s
i nform onis col l ectedinann X n m ixcal l ed emetri c. The metri c is updatedwi
eacbiteration dis used toestabl i shthes hdi recti on. An initial choi cefor themetri c
is al so u d. It must be a tet'c posi ti ve deftnite matri x. For the m od to
converge themetlI c must hol don toits posi ti vedefi ni tepropertythroughtheiterations.
h eDFP method themetri c approaches i nverseof theHessi anat Ul esol uti on.
A ori thm: Davi don- F ' cher- Powel l (A6.
Step 1. Choos x.. [A1l (initial me ic}N
Eh e2' eJ: (tol erancefor stoppi ngcri ia)
Set i = 1 (i nitializ i terati oncounter)
Step2. 8= [Aj ]Vf (Xj)
X
i
+
1
= X
i
+ su AX=j S
i is determi ned by mi ni mi zi ngf (Xi +l)
Step3. If Vf(Xi +I )TVf ( X
i
+
1
) e3; converged
If ! f( X
i
+
l
) f(Xj )1::;; el; stop (functi onnot changi ng)
If AX
T
(desi gnnot changi ng)
If i + 1 = N stop (i terati onI i mi l )
El s
Y = Vf (Xj +l) - Vf ( Xj}
z=[A; ]Y
AXa X
T
[B]=---:-::::;:::
AX' Y
ZZT

V' Z
[ Ai+Jl = [ A ] + [B] + [ q
i i + 1
GoTo Step 2
248 NUMERI CAL TECHNI OUES FORUNCONSTRAI NEDOPTI MI ZATI ON
Intheabove ematri ces areencl osed by square brackets. The initial choi ce of the
metri c is a posi ti vedefi ni tematri x. The i denti tymatri x is a safechoi ce.
DFP. m: The DFP al gori thm(A6. 7) is coded in is m-fi l e. Si mi l ar 10theprevi ous
programs for two vari abl es edesi gnchanges aretrackedon a background contour
pl ot Fi gure 6.6 . The defaul t initial metri c is Ihe i denti ty matri x generatedusi nga
MATLAB built-infuncti on.
Usage: DFP( ' Exampl e6_1' [ 0. 5 0. 51 40. 00010120) ; (6.12)
Output: The output f romthe above i nvocati on (copi ed f romthe Command
wi ndow) can be obtai nedby removi ng thc scmi -col onafter thestatement . The vari abl e
defi ni ti ons match thosedefi nedintheal gori thm. For Exampl e 6.1
0
1

=

n
U
A

4 4.5
FJ gure6.6 DFP Mel hod: Exampl e 6.1.
5
6.3 NUMERI CAL TECHNI QUE 3RADI ENT. BASEDMETHODS 249
i t er at i on number : 1
s
0. 4990 2. 2467
del x =
0. 0996
0. 4483
y
- 1. 1458
2. 6153
z
- 1. 1458
2. 6153
B
0. 0094 0. 0422
0. 0422 0. 1899
c
0. 1610 0. 3676
0. 3676 - 0. 8390
A
0. 8483 0. 4097
0. 4097 0. 3509
s
1. 2443 0. 5446
The pr obl em: Exampl e6_1
The desi gn vect or f unct i on val ue and KT val ue
dur i ng t he i t e at i ons
0. 5000 0. 5000 5. 3125 5. 2969
0. 5996 0. 9483 4. 7832 2. 8411
2. 9996 1. 9987 3. 0000 0. 0000
6. 3. 4 Broydon- Fl et cher- Gol df art ShannoMet hod
If you were to programonl y one gradi ent-bascd m thod. then thc Broydon-
Fl et cher - Gol df ar b- Shanno( BFGS) 181met hod woul d be the one. I t is a
quasi - Ncwton met hod and currentl y is the most popul ar of th Vari abl e Mctri c
methods. It enj oys l he propcrl y of quudrati c convcrgencc and has robustness by
carryi ngf orward i nformati onf romI13Eprevi ous i terati ons. Thedi f f erencEbEtween
DFPand BFGS is theway themel ri c is updal ed. The f ormer converges tothe
i nverseof theHessi an. whi l e thelatter converges totheHessi an itself. Ina sense
e BFGS is mor c di rect. The BFGS has repl aced the Conj ugate Gradi enl
techni ques as a workhorse insol vi ngnonl i near equati ons.
H
j
i
l
-
-
J
4
1
J
H
A
H
i
l
l
-
J
250 NUMERI CAL TECHNI QUES FORUNCONSTRAI NEDOPTl MI ZATl ON

For convergence the metri c must be posi ti ve defi ni te. An initial choi ce of
posi ti vedefi nIl matri x for themetri c is usualJ y suffi ci enl l oensure this property
for quadrati c probl ems. The i denti ty matri x is usual l y a defaul t choi ce.
Al gori thm: Br o F er- Gol df arb- Shanno( BFGS) Met hod (A6.
Step 1. Choe X" [Atl (i nitial metri c) N
E" E2E3: (tol erancefor stoppi ngcriteria)
Sct i = 1 (initializeiterationcounter)
Step2. The searchdi recti onis obtai nedas a sol uti onto
[A;]S = - Vf ( X )
C l S ; dJ ( = S
i is detenni nedby Mi n zi ngf (X )
Step3. If Vf ( X )TVf ( X) E3converged
If I r( Xi+l) f ( X)1 EI sp (functi onnot chungi ng)
If !1X
T
dJ ( E2; stop (desi gnnot changi ng)
If i + 1 = N stop(iterationlimit)
El se
Y = Vf ( X
i
+
l
) - Vf ( X)
yyT
[B) =
y' dJ (
Vf ( X ) Vf ( X?
[ C ] =
Vf(X;)' Sj
[Aj +d= [A;] + [B) + [c]
i f- i +1
GoTo Step2
Appl i c tl on0 ' BFGS Met hod: The met hod is appl i cd to Exampl e 6. 1. Onl y
cssenti al computati ons areshown.
cpI XI =l s l ; 1) = 5.31 ; 1 Vf ( X1) l

s t e p 3 Y =1 2 J r i ; m=l t ml
6.4 NUMERI CAL TECHNI QUEs - 5ECONDORDER 251
R
U

i E c l = 2 m- 3 2 1
- 3. 0429 6. 5095
S 2 4 2 2 1 ; =1 44; X3=[ H =3. 0; ) =[ g]
Converged
The transl ati on of the BFGS al gori l hmi nto codc is left as an exerci se for the
student. It is r ecommcnded that Ul ecal cul ati onof thesearchdi recti oninStcp2 be
accompl i shed as a sol uti ontoa set of l i near equati ons (as listed) raUl er thani nverti ng
the matri x A. One other thi ng to note for Exampl e 6.1 is that if the pol ynomi al
approxi mati on was used for the one-di mensi onul process the cal cul ati on of -
woul d have becn exuct (why?).
6. 4 NUMERI CAL TECHNI QUES - SECOND ORDER
Scond-order methods for unconstrai nedopti mi zati onarenot uscdbcaus val ual i on
of the Hessi an matri x duri ngeach iteration the Hcssi an matri x is consi dered to be
computati onal l y expensi ve. A second-order met hod wi th e property of quadrati c
convergence is very i mpressi ve. An n-vari abl eprobl emcan convergc inone iteration.
As menti oned before for real desi gn probl ems where deci si ons arerequi redto be
made on Ul e di screte nature of some of e vari abl es exi stence of the first
deri vati ve not to menti on second deri vati ves is questi onabl e. Moreovcr Ul e
quasi - Newton methods of last t wo secti ons are abl e to effccti vel y functi on as
second-order methods as ey approach sol uti on- and they do not need the
esti mati onof secondderivatives. One second-order met hod is prl sentedherefor the
sake of compl etenss.
I ndeed there is onl y one basi c second-order techni que for unconstrai ned
opti mi zati on. It is based on the extensi on of the Newt on- Raphson techni que to
mul ti vari abl e probl cm. Many di ffcrent cxtensi ons are avai l abl e but in this text a
253 6.5 ADDI TI ONAL EXAMPLES
(i terati onI i mi t) stop If i + 1 = N
El se
i i + 1
GoToStp 2
Appl i cati on of Modi f i ed Newt on Met hod: Al gori thm(A6. 9) is used to sol ve
Exampl e 6.1. Onl y essenti al computati ons arei ncl uded:
f ( X
t
) =5.3125
-
E
E
E
-
E
0
5

&
n
U
Q
U

s =1 1 : u = 1. 0: X? =1 1 ; 1=1. 0; X;
1 1. 5000 l ' U ot - J . v - 1 2 1
Asexpect ed onl y a si ngl ei terati onwasnecessary toobtai n thesol uti on. Transl ati on
of theal gori thmi nt oMATLAB code isonce agai n left as an exerci se.
converged f ( X
2
) = 3.0;
Three addi ti onal exampl es arepresentedin is secti on. Wi t h many of thenumeri cal
techni ques al ready pr ogr ammed obtai ni ng the sol uti on is a matter of j udi ci ous
appl i cati onof thenumeri caI procedures. Thi s secti onal sopresents some creati veuse
of the methods and expl ores nonquadrati c probl ems. le frrsl exampl e is e
Rosenbrock probl em[9]. l I s exampl e was among those created to cha11eng e
numeri cal techni ques for unconstrai nedmi ni mi zati on. It is al sosometi mes referr dto
as thebananafuncti on. I f you have a copy of th Opti mi <.ati onTool box f romMATLAB
you wi1l see is as part of tool box demo. You wiJ1 al so note at the tool box
contai ns many of techni ques devel oped inthis secti on. The second exampl e is a
sol uti ontoa nonl i near two-poi nt boundary vaue probl emthat isdue 10 eNavi er-Stokes
uati ons descri bi ng f l owdue to a spi nni ng disk. 1 last is an unusua data fi ng
exampl e usi ngBezi er curves. Once agai ntheOptirnizationTool box presents its versi on
of thedatafittingexampl e when runni ngoptdemo at theMA TLABprompt
ADDI TI ONAL EXAMPL ES 6. 5
(6.15)
Exampl e 6. 2- Rosenbr ock Pr obl em
f(xl X2) = 100(x2- xT)2+ (1 - XI)2
-2::;;xl ::;;5;
S t e p 1 4 3 1 ;
izl
-2::;;X2::;;5
6. 5. 1
The Rosenbrock probl emis
Mi ni mi ze
Subj ect to:
3
3
3
3
3
3
4
1
1
3
1
4
1
1
f
f
i
t
s
?
h
i
t
i
t
i
-
-
1
i
i
f
t
J
NUMERI CAL TECHNI QUES FO UNCONSTRAI NEOOPTI Ml ZATI ON
di rect extensi onrecast in egeneral al gori thmi c s uct (A6. 1) is presented. Once
agai nthegeneral probl emand speci fi c exampl e areas fol1ows.
252
(6. 1) f(X); [X]n
xl Xi xj t
Mi ni mi ze
(6.2) i =12... 11 Subj ect to:
(6.3)
(6.4)
f (X) = f(XI' X2) = 3 + (XI - 1.5x2)2+ (X2- 2)2
O::;;XI: 5; 0::;;x2::;;5
Mi ni rni ze
Subj ect to:
Modi f l ed Newt on' s Met hod: The Newt on- Raphson met hod used for a si ngl e
vari abl e sol ves the probl em(X) = O. Thi s equati on can present e F OC for an
unconstrai ned probl emin one vari abl e (Chapter 5). The i terati ve change in e
vari abl eis comput ed through
(6.13)
dx= L=
'(X) f"(x)
where f is th si ngl e-vari abl e obj ecl i ve functi on. The mul ti vari abl e extensi on lo
computi ng a si mi l ar change in evari abl evecl or is
(6.14)
The ori gi naI Newt on- Raphson is not known for its robustness or stability and
Equati on (6.14) shares thesame di sadvantage. To control thedesi gnvari abl echanges
and tobri ngit under eschem of a1gori thm( A6.l) thel eft-handsi deis defi ned
the search di recti on v ctor S f ol l owed by a standard one- di mensi onaI stepsi ze
computati on. Thi s is terrnedthemodi fi edNewt on method. The compl ete al gori thmis
as fol l ows.

Al gori m: Modl f l ed Newt on Me od{A6. 9)
Step 1. Choose XJ N
E" E2' E3: (tol erancefor stoppi ngcriteria)
Set i = 1 (initiali7..ei terati oncountr)
Step2. The searchdi recti onis obtai nedas a sol uti onto
[H(Xj )]Si = - Vf ( X
1
); [ is eHessi an
Xi =Xi +/ Si ; AX= .SI
" i is deterrni nedby Mi ni mi zi ngJ(X
i
+
l
)
Step 3. If Vf ( X ) TVf ( Xi+I) E3' converged
If I r( X 1) f(Xi )1::;; EI' stop (functi onnot changi ng)
I f AX
T
s X::;;E2 Sl Op (desi gnnot changi ng)
254 NUMERI CAL TECHNI QUES FORUNCONSTRAI NEDOPTI MI ZATI ON
The si deconstrai nts areused for drawi ng contours. The sol uti ontothis probl emis
x ; = 1. 0; x ; = 1. 0; I = 0.0
Thc probl emis notori ous for qui ri nga l argenumber of i l erati ons for converg ces.
Changes in desi gn are smal l as the sol uti onis bei ng approached. Here the Random
Wal k and Conj ugate Gradi enl methods areused toexami ne thesol uti on. Both these
methods are changed 10 drawappropri al e conl our levels. To avoi d c1utter di sabl e
pri nti nglexl i nformati onon l i nes representi ngdesi gnchanges. Al so si nc ese
methods are run for 200 i terati ons the f mal pri nti ng 10 the Command wi ndowis
avoi ded. The trac of thedesi gnvari abl es on thecontour pl ot is provi ded as theoutpul
f roml heprogram al ongwl thIh linal val ues. As thefi gurc is bei ngdrawn it is c1ear
that theprogress tothesol uti onis sl ow.
Sol utl on Usi ng RandomWal k Met hod: Fi gure 6.7 presenl s the resull of
executi ngRandomWal k. m for thls exampl e. Th programi nvocati onand output f rom
eCommand wi ndowis
N
M
-1 2 3
X
Fl gure6.7 Randomwal k method: Exampl e 6.2.
4 5
>> RandomWal k( ' Exampl e6_2' [3 2] 200l . Oe- 080l 20)
The pr obl em: Exampl e6_2
ans
2. 0000e+002 1. 0000e+000 1. 0000e+000 1. 2327e- 007
Si nce thre are no stoppi ng criteria bui 1t i nto the met hod (was suggested as an
exerci se) themet hod ranfor 200 i l erati ons (first val ue). tefinal val uefor thedesi gn
V or and eobj ecti vefuncti onarevery i mpressi ve:
r andomwal k : x ; = 1. 0; x ; = 1.0; I = 1. 23e- 07
Sol uti on Usl ng Co ugat eGradl ent Met hod: The i nvocati onand output f rom
theCommand wi ndowis
nj ugat eGr adi ent ( ' Exampl e6_2' 2) 2001. Oe 080120)
The pr obl em: E xampl e6_2
No. of i t er at i ons: 200
ans
9. 6B07e- 001 9. 3700e- 001 1. 0222e- 003
The sol uti onis
conj ugate gradi ent : x; = 0. 96807; x; = 0. 937; r = 1. 0222e - 03
Thi s appears no mat ch for theRandomWal k met hod for is i nvesti gati on. Fi gure 6.8
tracks edesi gnchanges on thecontour pl ot.
6. 5. 2 Exampl e 6. 3Thr ee- Dl mensi onal Fl ownear a Rot at i ng Di sk
Thi s exampl e represenl s another of Iheexact sol uti ons to c Navi er-Stokes equati ons
f romfl ui dmechani cs. The nonl i near two-poi nt boundary val ue probl emdescri bes a
vi scous f l owaround a flat di sk at rotates about axi sperpendi cul toitspl ane wi
a constant angul ar vel oci ty. After sui tabl e defmi ti on [10] the mathemati cal
descri pti on of the probl emis the fol l owi ngset of coupl ed nonl i near equati ons wi th
boundary condi ti ons at t wo poi nl s:
2F+H' =0
(6.16a)
F
2
+ F' H - G
2
- F " = 0
(6. 16b)
2FG+ HG' - G" = 0
(6.16c)
256 NUMEAI CAl TECHNI QUES FORUNCONSTRAI NEDOPTI MI ZATI ON
. . .
M
x
t
2 3
Fl gure6.8 Conj ugategradl ent method: Exampl e 6.2.
4 5
F(O) = 0; G(O) = 1; H(O) = 0
(6. l 7a)
F() =0; G( ) =0
(6. 17b)
Equati ons (6.16) areconvertedstate spa f onn(s Chapter 5 Exampl e 5.3). The
desi gn vari abl es arE the mi ssi ng boundary condi 6orts on F' (O) G' ( O) and the final
val ueof thei ndependent vari abl e(inlieuof ). The obj ecti vefuncti onis esquar
crror inthei ntegratedvaJ ue al thefinal condi ti on.
The Optl ml zati on Pr obl em
Mi ni mi ze f (xl' x
2
x
3
): YI (X
3
)2+ Y3(x
3
)2 (6. 18)
where thc slalc vari ab1es areobl ai nedf romthesol uti onof thc fol l owi nginitial val ue
probl em:
YI
Y2
YI (O) O
Y2
Yi +Y
Y2(O)
~
Y3(0)
(6.19)
Y3 Y4
Y4 2YI Y3+ Yv' 5 Y4(O)
x
2
Y; -2YI Y5(O)
Thc statcequati ons (6.19) .c i ntcgral edusi ngMATLAB functi onode45 .

Sol utl on to the Pr obl em: The three-vari abl e probl emis sol ved usi ng the DFP
method. Exampl e6_3. mr t ums the obj ecti ve functi on. It calls the functi on
Ex6_3_st at e. mwhi ch rcturns the systemquati ons in state fonn. The call to the
DFP met hod wi th einitial desi gnvector of [1 - 1 8J wi th20 i terati ons of theDFP
met hod and a tol erancefor thegol den secti onof 0. 0001 is shown bel ow. AJ so inthe
call is thei nput tol heupper bound cal cul ati on.
Usage: DFP (' E xampl e6_3' [1 18] 200. 00010120) (6.20)
The final val ue for thedesi gnafter 20 i terati ons was
x;( ) = 0.5101; .t;(t.t) = - 0. 6157; xj ( =10. 7627; j = 5 08e 06 (6.21)
111e tol erance for the gol den secti on was onl y 0. 0001. h is possi bl c l hc obj ecti vc
cannot decrcasc unlil a smal l r tol eranceis speci fi ed.
The publ i shedval ues for thedesi gnvari abl es ar
x ; = 0.5101; ( ) =- 0. 6159 (6.22)
whi ch is remarkabl y cl ose to I he val ues obtai ned by the DFP method. The vaJ ue of
xj is ILslLallynot speci fi edbUI is about 7. Si nce it is a substi tutefor i nfi ni ty theJ arger
number shoul d not be a probJ em.
Exampl e 6.3 is not trivial probl em. The i ntegrati on is hi ghl y sensi ti ve to the
initial val ues. A reasonabl e starti ng poi nt is essenti al to prevent the i ntegrati on
f romgenerati ng NaN's (nol a number). The second desi gn vari abl e musl have a
ncgati ve si gn. The exampl i l l ustrates the appl i cati on of standard opti mi zati on
techni que to soJ ve a nonl i near di fferenti al system. A si mi l ar appJ i cati on can be
made to probl ems in sysl emdynami cs and opti ma1 control . It is essenti al to
understand theal gori thms outl i nedhere arenumeri cal too1s at transcendany
parti cuJ ar di sci pl i ne.
There arenumeri caJ techni ques at address two-poi nt non1i near boundary val ue
probl ems mor e effi ci entl y. However computi ng resources especi al l yon a PC arenot
a concern. Therefore t11eproccdure adopted in is cxampl e is an acceptab1eapproach
258 NUMERI CAL TECHNI QUES FORUNCONSTRAI NEOOPTI MI ZATI ON
for this cl ass of probl ems. Even the RandomWal k met hod is i mpressi ve in is
mstance.
6. 5. 3 Exampl e 6. 4 - Fi ttl ng Bezi er Par amet r l c Cur ves
The exampl e i ntroduces a nontradi ti al and an unusual curve fiuillg procedure.
Gi ven a set of datapoi nts a Bezi er curve 01 user-chosenorder is fil 10 thedata. Bezi er
curves are parametri c curves a speci al case of uni f ormB-spl i nes. Bezi er
parametri zati on is based on the Bemstei n basi s functi ons. Any poi nt P on a
I wo-di mensi onal Bezi er curve (any parametri c curve) is aCl ual l y obtai ned P(x(v)
y(v)) where 0 S v S 1. leactual rel ati ons are
P (v )=[x y ) =B Jn.1( v
;=0
JII.;(v) ) (1 -)". 1
(6.23)
whereB; E everti ces of epol ygon that determi n sthecurve. B; represent a pai r
of val ues in two-di mensi onal space. The order of thecurve is lI - t h hi ghest power
in l he basi s functi ons. The acl ual computati ons of the poi nl Son I heCUl ve' e ellsier
usi ngmatri x al gebmand arewel l expl ai ndand document ed inRef erence 11. Fi gure
6. 9 a cubi c B zi er curve. The fol l owi ng are some 01 ' l he most useful
prop ies of thecurve as obs rvedinthefi gure:
The curve is compl etel y defi nedby thepol ygon obtai ncdby j oi ni ngtheverti ces
inorder.
'hedegree of epol ynomi defi ni ngthecurve is one less anthenumber of
verti s of thepol ygon.
. The firsl and last poi nts of thecurve arecoi nci dcnt wi ththefirst and last vertex.
Thc remai ni ng vertex poi nts do not l ypi cal l y I i e on thc curvc.
. The sl opcs al theends of thecurvc have the same di recti onas thesi des of the
pol ygon
. The curve is contai nedwi thi ntheconvex hul l of epol ygon.
Inthis ex npl etheconveni ent matri x representati ons for thc curve arc used. They are
not di scussedhere however. but areavai l abl c inthecodc. For isexampl e. thedcsi gn
vari abl es arethei nsi depol ygon verti ces. For n = 5 thcrearc 6 ve c. The first and
last vcrtex areknown f romthedatapoi nts. Thi s l eaves 4 verti ces or 8 d si gnvari abl es
f orn = 5.
Exampl e6_ 4. m: lis m- fiI c is a stand-al one programto run thc cxampl c in this
sccti on. Typi ng Exampl e 6_4 al the prompt shoul d start thc program. Actual l y the
0.8
0.7
0.6
8 0 5
p
0.4
0.3
0.2
0.1
6.5 AOOI TI ONAL EXAMPLES

'1
"
8
2
/
a '
/
/ i
f /
. / '
.

1 J .
- - -

1

1 / "
1 . / l '
1 . / .
1 / J
/
/
.' / C!I 1 /
-
/ E3
' "
B
0.2 0.4
Fl gure6.9
x data
Acubl c B zl rcurve.
1.6
259
2
exampl e is generi c as most deci si ons a basedon etext file that contai ns thedata
tobe fi ued. 11calls on m y m-fi l es. The Bezi er caIcul ati ons arefairlymodul ar. Thi s
programis not vi al. A mor e i mportant asonfor its i ncl usi onis atif understood
it si gnal s a reasonabl e maturi ty wi th MATLAB programmi ng. Mor e i mportanl l y it
exempl i fi es eway toexpl oi l the i mmense power of bui l t-i n MATLAB suppo. The
author encourages theuser toexert somc effort traci ngtheprogress of thecal cul ati ons
rough ecode. A 101of programcontrol is based on euser' s choi ce concemi ng
the order of the Bezi er curve. Al so the initial val ues for the desi gn vari abl es are
generated automati cal " based on thedata10 be fitted. The functi oncal l i ngsequence
used in is exampl e can be bettcr understood by thefol l owi ngstructure:
Exampl e6_5_3. m
CaUs ui getfi l c topi ck up e(* .txt) Ic at contai ns t wo- col umn data at wi l l be
fit (Ex6_5_3. txt)
'hrows up an i nputdi al ogbox toobtai ntheorder of fit f romtheuser (0 = 5)
Cal l s coef f . mwi ththeorder i nformati on
Cal l s combl nat l on. m- cal cul ati on01 ' combi nati on for Bcrnstei nbasi s
Cal l s Factori al . m- caIcul ates factori al of ani nteger
260 NUMERI CAL TECHNI QUES F OR UNCONSTRAI NED OPTI MI ZATI ON
Cal l s Curve_f i t. m whi ch handl es theopti mi tion
CalI s DFP
CalI s BezSq_Er r . m cal cul ati onof functi on
Cal l s gri' sdfun. m- cal cul ati onof deri vativ s
Cal l s Gol d_Sect i onnVa r ' . m - 1 D stepsi ze
Cal l s Upper Bound_nVar . m- bracket themi ni mum
Si nce there are ei ght desi gn vari abl es a11 of the i nformati on except for iteration
number. convergence criteria and obj cti ve functi on is suppressed f romthe
Command wi ndowtoprevent scrol l i ngand loss of comprehensi on.
lesi gni fi cant el ements in is codear as fol l ows:
Pi cks up edatafilethrougba filesel ecti onbox.
Uses an i nput di a10g box toquery theuser on theorder of thec ve.
Theordr determi nes thesizeof thecoeffi ci ent matri x A and it is automati ca11y
generat .
The number of desi gn vari abl es is automati cal l y determi ned. Once agai nit is
basedon order of thecurve.
The initial val ues for thedesi gnareautomati cal l ygeneratedbasedon theda
tobe fi t .
TheDFP method is usedtofi ndtheopti mum.
The obj ecti vefuncti onis computed inthegeneri c fiI e Bezi ecSq_Er r . m. The
dataspeci fi c to exampl earepassedthrough eMAT L AB gl obal statement .
The ori gi nal dataset and Bezi er curve fittingthedataar di spl ayedat the
end.
Theobj ti ve functi onand th convergencedatafor eachiterationarepri nted
totheCommand wi ndow.
ObJ ectl ve Functl on: Th obj ecti ve functi onis the1 st squarederror over all of
thedatapoi nts between eactual dataand thefittedBezi er curve. It is expr' ssedas
Mi ni mi ze:
I I Da
X): Iy -YBT
1<=1'" . J
(6.24)
The fol l owi ng i nformati on is copi ed f romthe Command wi ndow (the file is
Ex6_5_3. txt and eorder is 5):
Bxampl e6_4
gr adi ent l engt h squar ed:
obj ect i ve f unct i on val ue:
gr adi ent l engt h squar ed:
5. 0001
0. 5302
5. 6890
obj ect i ve f unci on val ue: 0. 2497
gr adi ent l engt h squar ed: 0. 0546
obj ect i ve f unct i on val ue: 0. 1830
i t er at i on number :
obj ect i ve f unci on val ue: 0. 1401
UI
Fr omthedataabove it appears that thee or is around 0. 14and is difficult tori uce
further. It al soappears thereducti onintheobj ecti veis accompani ed by thei ncrease
inthegradi ent. after a certai nnumber of iterations.lt is movi ng away f romsati sfyi ng
theFOC. The best val uefor theF OC is about 0. 007and starts toworsen wi thfurther
iteratI ons. It is qui tepossi bl e numeri cal i naccuraci es may be a probl emas there
are xtensi ve computati ons i nvol ved in each iteration. It is al so l i kel y at e
numeri ca1gradi ent computati on may need tobe refi ned. Currentl y egr ients e
computed usi ngforwardfinitedi fferencewi tha stepof O. OO1. These areparameters
tltat n d to b xpl ored in opti mi zati on as there is no uni versal set of val ues
for all cl asses of probl ems.
Fi gure 6. 10illustrates eori gi na1data dtheBezi er curve a er elast iteration.
The pol ygons arenot shown as our effort is togen ratethec ve. hefit is defi ni tel y
acceptabl eand thereis probabl y little eBezi er curve can do about a coupl eof poi nts
that arenot very smoothl y l ocated. It is probabl y thesepoi nts that do not a110wthe
obj ecti vefunctI on todecreasefurther wi thout i ncreasi ngth gradi ent . This concl udes
another i nnovati ve use of unconstrai ned opti mi zati on. le author has used these
curves toopti mal l y desi gnairfoils for l ow-speedflight .
' "
'11'
I
d
-
-
R
4
3
t
m
i
t
t
H

E
l
i
-
-
Not e on F OC for Unconst rai ned Mi ni mi zati on: exampl e has alerted us
tothequesti onof FOC Vf = O. If anal yti cal computati onof thederi vati veis possi bl e
for a probl em. thenFOC has the likeliltoodof bei ng met. ln desi gn probl ems like
Exampl e 6. 4 where theel ements of thegradi ent vector have thef orm
51
oo
4
) o. e..._ 0
- e ...
9
2
263
5. Fl etcher R. and Reevcs R. M. Functi on Mi ni mi zati onby Conj ugal e Gradi ents. The
Comput er J oumal . Vol . 7 pp. 149- 180 1964.
6. Davi don W. C. Vari abl e Metri c Methods for Mi ni mi zal i on U.S. Al omi c Energy
Commi ssi on Research and Devel opment Reporl No. ANL - 5990 Argonne Nati onal
Laboratory 1959.
7. Huang H. Y. Uni fi ed Appr oachQuacaJ l y Convergent A1gori thms for Functi on
Mi n .tion J oumal of Op u wtI on ' aoryand Appl i cati o/ls Vol. 5 pp. 405- 423 1970.
8. Vanderp1aats G. N. Nl /l neri cal Opti mi zati o/l Techni ques f or Engi neeri ng Desi gn
McGr aw- Hil1 New York 1984.
9. Hock W. andSchi ttkowski K. Test Exampl es f or Non Li near Programmi l!g Codes
LectureNotes InEconomi c and Mathemati cal Systems 187 Spri nger-Verl ag Berl i n
1980.
10. Schl i chti ng H. BoundaI Layer Theory McGr aw- HilI New York 1979.
11. Rogers G. F. andAdams J. A. Mathemati cal El ementsf or Comput er Graphi cs 2nd ed.
McGr aw- HilI New York 1990.
PROBLEMS
NUMERI CAL TECHNI QUES FOR UNCONSTRAI NED OPTI MI ZATI ON
E
d
-
t

a
u
-
d
u
f

o
- :

a
E
F
-
-
n
"
a
u
-
-
;

O
B
-
h
v
-
-
O
262
e
a
a
a
q
{
S
E
E
E
)
S
PROBL EMS
Appl y theRandomWal k met hod toExampl e 5.2.
Sol ve usi ngtheRandomWal k method:
Mi ni mi ze !(X. x2) = x1- 2 2 x 1 ~ 2 x . +4
Appl y thePattemSearch met hod toExampl 5.2.
Sol ve usi ngthePattemSearch m od:
Mi ni mi ze !(.x.x2) = x1- 2x1X2 + xi +.q-2x. ... 4
Modi the Pate mSearch so that programcontrol p neters for contour
pl otti ng gol den secti on and upper bound cal cul ati on can be set by er.
I ncl ude prompt and al l owfor defaul t val ues incase theuser deci des not to
takeadvantage of it .
Transl atethePowel1met hod i ntoworki ng MATLAB code. Veri fy thesol uti onto
Exampl es 6.1 and 5.2. Start f Tomseveral poi nts and veri fy that the number of
cycl es toconverge is esame.
Veri fy esearchdi recti ons areconj ugate wi th spect totheHessi an matri x
for Exarnpl es 6.1 d5. 2.
Sol ve usi ng Powel l ' s m od d at veri fy any t wo search di recti ons are
cougatewi respect totheHessi an matri x at thesol uti on:
Mi ni mi ze !(.x.x2) = x1- 2x1X2 + xi +.q-2x. + 4
Sol ve Exampl e 5. 2usi ngtheSteepest Descent Conj ugate Gradi ent DFP and
BFGS me ods.
6. 10 Sol v usi ng the Steepest Descent Conj ugate Gradi ent DF P and BFGS
methods probl em
ze f(x. OX2) = x1- 2x1X2 + xi +.q-2x. + 4
6. 6
6. 8
6. 9
6. 7
6. 3
6. 4
6. 2
6.5
6.1
2.5
and whi ch are best comput ed numeri cal l y some speci fi cati ons areneeded regardi ng
deri vati ve computati ons if they aretobe used for enforci ngthenecessary condi ti ons.
REF ERENCES
1.5
x data(readf romflle)
Orlglnal dataand Bezl er curve.
i )
1. Fl etcher. R Practi cal Metl l Odsf or Opti mi wti on Vo1.1 Wi l ey New York 1980.
2. Brent R. P. Al gori thms f or Mi l l i mi zati on wi thout Deri vati ves ti ce-Hal l
Eng1ewood Cliffs NJ 1973.
3. PowelJ. M. J. D. NOl l l i near Opti mi Wl i on Academi c Press New York. 1981.
4. Fox R. L.. Opti mi zati ol l Methodsf or El!gi neeri ngDesi gn. Addi son-Wesl ey. Readi ng
MA. 1971.
Fl gure6. 10
0.5
11-
264 NUMERI CAL TECHNI OUES FORUNCONSTRAI NEDOPTI MI ZATI ON

6. 11 Veri fy theval ues in Scti on 6.3.3 usi nga cal cul ator. Veri fy if thematri x A at
esol uti onis ei nverseof theHessi an.
6. 12 Veri fy if the matri x A at the sol uti on is the i nverse of the Hessi an for DFP
me odfor
Mi ni mi ze f(XI x2) =x1- 2xtx2+xJ x ~ - 2x1 + 4
6. 13 Devel op e BFGS met hod i nto MATLAB code and veri fy the caJ cul ati ons in
Secti on6.3.4.
6. 14 Devel op the Modi f i ed Newt on mct hod i nto MATLAB code and appl y it to
Exampl e 5.2.
6. 15 How wi l l you i ncorporate esi deconstrai nts i ntothecod for aJ l of thevari ous
methods inthis secti on? I mpl emcnt t heminthc numeri cal procedures.
6. 16 Sol ve Exampl es 6.2 6.3. and 6.4by one other mthodof thesecti on.
6. 17 I denti fyand sol vea systemdynami cs probl emusi ngany met hod ofthi s secti on.
6. 18 I denti fyand sol vc your own curve fit probl cm.
7
NUMERI CAL TECHNI QUES
F OR CONSTRAI NED
OPTI MI ZATI ON
lis chapter expl ores aJ gori thms/ methods that handl e e gcneral opti mi zati on
probl em. Bot h equ ity and i nequaJ ity constrai nts are i ncl uded. For an engi neeri ng
probl mthis wi l l i nvol ve bri ngi ngseveral nonl i near rel ati ons i nto thedesi gn space.
Thi s is certai n to i ase the degree of di ffi cul ty in obtai ni ng c sol uti on. For the
desi gner ere is an addi ti onal burden of bei ngmor e attenti veto thedesi gnchanges
and thecorrespondi ngnumbers tocoax thesol uti onif themathemati cal defi ni ti onof
theprobl emis parl i cul arl ysevere. InaJ l of l heseprobl ems l herearet wo outcomes that
al gori thms seek to accomp i sh. le first is to ensure at the desi gn is feasi bl e
(satisaJ l constrai nts) and thesecond t it is opti mal (satisfies the Kuhn l cker
condi ti ons). Whilethefocus is on determi ni ng esol uti on inti mes of di ffi cul ty it is
essenti al 10remember Ihatfeasi bi l i is mor e i mportant at opti mal i ty. Al so unstated
in rest of echapter is at eopti mal sol uti on must be feasi bl e. Two di sti nct
approaches wi11 be used to handl e the constrai ned opti mi zati on probl em. The first
approach is termed ei ndi rect approach and sol ves probl emby transformi ng it
i nto an unconstrai ned probl em. The second approach is 10 handl e e cons ints
wi thoul nsf ormati on- thedi rect approach.
The i ndi rect approach is an expressi onof i ncremental devel opment of thesubj ect
totakeadvantage of ecurrent methods. For ex npl eit l everages theDFP met hod
to handl e constrai ned opti mal probl ems. Two i ndi rect methods are presented. th
Exteri or Penal ty Functi on (EPF) met hod and Augment ed Lagrange Mul ti pl i er
( ALM) method. The di rect approach handl es theconstrai nts and theobj ecti ve together
wi outany nsformati on. Four methods p sented. lemethods in i sbookare
Sequenti aJ Li near Programmi ng (SLP) Sequenti al Quadrati c Progr ammi ng ( SQP)
General i zedReduced Gradi ent Met hod ( GRG) and Sequenti aJ Gradi ent Restorati on
Al gori thm( SGRA) .
265
266 NUMERI CAL TECHNI QUESFORCONSTRAI NEDOPTI MI ZATI ON
7.1 PROBLEM DEFI NI ON
The standardformat of thenonl i near programmi ng probl em(NLP) isreproducedhe
for conveni ence:
Mi ni mi ze f (x" X2 X
n
) (7.1)
Subj t to: hk(x" X2' . . . X
n
) = O. k = 12 .1 (7.2)
gj (X" X2. . . X
n
) 0 j = 1. 2 . m
i =I . 2. . . . . 1t
Invector notati on
Mi ni mi ze f(X). [Xl "
Subj ect to: [b(X)]f = 0
[g(X)] " S 0
X1nwSX: ; ; X"1'
(7.3)
(7.4)
(7.5)
(7.6)
(7.7)
(7.8)
For this chapter thefol l owi ngi ndi ces arercserved: It. number of variabl; 1. number
of equal i tyconstrai nl s; m. number of i nequal i ty constraints. Many of Iheal gori thms
and methods present a reasonabl e level of difficulty. A two-vari abl e prob1em(for
graphi cal i1l ustration). wi th a single equal i ty and i nequal i ty constrai nt is used to
illustrate the vari ous algori ms. Al1 of the functi ons are nonl i near a1though the
constrai nt functi ons have si mpl egeometri c shapes.
7.1.1 Pr obl emFor muJ at l on- Exampl e 7.1
Exampl e 7.1 is a si mpl e mathemati cal formul ati on. It does not present any
engi neeri ngprob1em. It is cons ctedtohave a sol uti onof xj = 1 dxi = 1 for
nstrai nedprob1emand xj = 0. 8520and xi = 0. 8520for theuncons i nedprob1em.
The two constrai nt functi ons have si mpl e non1i neari ty. The uality constrai nt is a
ci lewhi l ethei nequal i tyconstrai nt is aneli pse.
Mi ni mi ze f (x" X2): x1- + xr + XI - 2x1 + 4 (7.9)
Subj ecl l o: h(X"X2): xr +xi - 2=0 (7. Oa)
g(x" X2): 0.25xT + 0. 75 - 1 S 0 (7. l Ob)
O:5xl :55; 0:5x2:55 (7. Oc)
F
7.1 PROBLEMDEFINITION 267

9<0
3 4
F1gure7.1 Graphi c solu1ionfor Exampl e7.1.
Fi gure7.1 di spl ays thegraphi cal sol uti on to probl em. It is cl ear f romthe figure
that thesol uti onis at xT= 1. xi = 1 (why?). The val ueof thefuncti onis 3. A si mpl e
a roachtothesolutionof theprobl emis 10u uati on(7.l Oa) 10el i mi nateone of e
variables m eprobl em erebyreduci ngit toa singlevariableconstrai nedprobl em.
lis is not done he but it is a good exercise. FI g7 _l . m
1
will createthefigure.
7. 1. 2 Necessar y Condl t i ons
le necessary condi ti on-or the Kuhn- Tucker (KT) condi ti ons for e
probl emwas devcl oped in Chapter 4. It is based on the mi ni mi zati on of E
Lagrangi anfuncti on(. The condi l i ons are
Mi ni mi ze
F(xl .. 'X
n
' AI ... Af' sI ... s
m
) = f (xp
x
n
) +I hl ++ +
P 1+ +s" g" (7.11)
1Ft! 10be downl oadedfrom ewebsi areindicaledbyboldfacesans serif Iype.
A
4
L
/
269
(7. l7c)
Equati on st (7.17) is sol ved symbol i cal l y in MATLAB. Thcre are t wo sol uti ons tha!
sati sfy thesi dc cons ints (7.l 0c)
PROBLEMDEFI N ' ON 7.1
xi +xi - 2=0
NUMERI CAL TECHNI OUES FO CONSTRA' NEDOPTl MI ZATI ON
subj tto econs aints. le aren + 1 + m unknowns intheLagrangi an. The same
number of equati ons is requI I tosol ve probl em. Theseareprovi ded by theF OC
or Kuhn- Tucker condi ti ons. Here' l equati ons areobtai nedas
268
j = 1. 2. . . . . n
(7.12)
dF dr dh. . dh 0 d81. . 11 d8m
= + + + L - 4 1 + +PJ S7=0;
axi a x a I a x t t a x i t axi dxt
x; = 1. 4142;
x; = 1. 0676; =- 0. 1217
=0 X; =0;
x ; = 0. 9275;
80th of these vi ol ate e i nequal i l y constrai nt. I n Fi gure 7.1. the first sol uti on
defi ni tel y do but l hesecond is a liltledi ffi cul t 10see. Zoomi ng shoul d hel p. Case a
docs not provi de a sol uti on.
1 equati ons areobtai neddi rectl y t1lfOugh eequal i tyconstrai nl s
(7.13)
k = 1. 2. . . . . 1
hi xX2.... . xn)=O;
lis establ i shes four equati ons as g =O Case b:
8j <0 then
m cquati ons areappl i cdthroughthe2
m
cases
sj8j =Oif =0
(7. 18a) 4xt +0.5s X
1
=0
(7. 18b) - 2xf +2xx
2
+2X2+1. 5
(7.14)
>0 l hen if 8j =0
(7.18c) xi - 2 =0
The K T condi ti ons appl y onl y if thepoi nts areregul ar (seeExampl e 4.3).
(7. 18d)
Equati ons (7.18) sol v symbol i cal l y and l he onl y sol uti on sati sfyi ng Ihe si de
constr nts s
0. 25 - 1 =0
s " =O
whi ch was expected(for thedesi gnvari abl cs not l hemul ti pl i crs). The val uc of s" = 0
is not expecl edas lhis is anacti veconstrai nt. The K T condi ti ons can be consi dered10
be weakl y sal i sfi edas s is not strict1y posi ti ve. As inChapter 6. wherever possi bl e.
eK T condi ti ons areused tocheck for convergence.
i.." =- 0.5000; =1. 0; xi = 1. 0;
a

;
t
i
-
-
Appl l cati on 0' the K T Condi ti ons: eLagrangi an
F( x"
- 2xT x
2
+xT+x -2x + 4 +
xi + 0. 75 - 1)
(7.15)
(7.16a)
(7. 16b)
Threeequati ons areset up through
dF
E; =44- 4xt X2+2XI +xi - 2+2hi +05PXl =0
dF
= 2xT+2xx
2
+2 i.. x2+ 1. 5 s X
2
= 0
C1X
2
El ement s of a Numer i cal Technl que
There is a di sti ncI di erence in the numeri cal techni ques wi th i ncorporati on of the.
constrai nl S. even if l hei deaof usi nga searchdi recti onvecl or is still useful . The maj or
di fferences inthea1gori thms of theprevi ous chapter was waythesearchdi recti ons
were establ i shed. To understand the necessary changes. the general approach for
unconstrai nedopti mi zati onas seenintheprevi ous chapter was:
7.1.3
(7.16c)
h =xi - 2 =
Wi t h this i nformati onEquati ons (7.16) become
The fourl hequati onis deve10ped through
s =O Case a:
General Al gorl t hm 6. 1)
Step 1. Choose X
o
(7.17a)
(7. 17b)
4x1 - 4 X x
2
+

- 2xT + 2x x
2
+ 2 x
2
=0
270 NUMERI CAL TECHNI OUES FORCONSTRAI NEDOPTI MI ZATI ON
Step 2. For each i terati oni
Detenni ne searchdi recti onvector Sj
(It woul d be ni ceif theobj ecti vedecreasedal ongthis di recti on)
Step3. Cal cul ateA Xj = Sj
Note: A Xj is now a functi onof thescal ar ( Xj as Sj is known f romStep2
( Xj is cal l ed estepsi zeas it establ i shes el engthof AXj
a; is deterrni nedby Mi ni mi zi ngf(X; +I )' where
Xj +1= X + A Xj
As theabove steps erevi ewed inStep 2 a di l emma n ds tobe resol ved: Shoul d
the searchdi recti onSj decrease eobj ecti ve functi on0' shoul dit attempt tosati sfy
theconstrai nts? (It is assumed at it cannot do both though very oftenit wi l l.) Si nce
feasi bi l i ty is mor e i mportant an opti mal i ty it is si er to i denti fy the latter as a
si gni fi cant i nfl uenceindeterrni ni ngthedi recti on. Thi s is mor e di ffi cuIt thanit sounds
as each constrai nt can have its own favori te di recti on at the current poi nt . Such
occasi ons call for a trade-off and toseeif it is possi bl etonegoti atea decrease inthe
obj ecti vefuncti onas wel l . It is cl ear that Step 2 wi l l i nvol ve i nvestment of effort and
some sophi sti cati on.
Step3 is not far behi nd inthe qui ment f orsp ial handl i ng. Once thedi recti on
is establ i shed (Step2) what ki ndof stepsi ze wi l l be acceptabl e1Several resul ts are
possi bl c. First thc obj ccti ve functi oncan b decreased al ong the di recti on. Second
current acti ve constrai nts (i ncl udi ng equal i ty constrai nts) can become i nacti ve
vi ol ated or can still p serve eacti vc state. Thi rd current i nacti veconstrai nts can
undergo a si mi l ar change 01 " state. Fourth probabl y most i mportant current vi ol ated
cons aints can become acti veor i nacti ve. It is di ffi cul t toencapsul atethis di scussi on
in a generi c al gori thm. Somc al gori thms combi ne Steps 2 and 3 i nto a si ngl e step.
Some di vi dethei terati oni ntoa feasi bi l i ty componenl and an opti mal i ty component .
Inessence thegeneri c a1gori thmfor is chapter whi ch is not parti cul arl y eful is:
Generl c Al gorl t hm(A7. 1)
Step 1. ChseX
o
Step2. For each i terati oni
Detenni ne searchdi recti onvector Sj
Step3. Cal cul ateAX; =j Sj
Xj+1 = Xj + A Xj
7. 2 I NDI RECT METHODS F OR CONSTRAI NED OPTI MI ZAT10N
These mc ods were devel oped to take advantage of codes at sol veunconstrai ned
opti mi zati on probl ems. They are al so referred to as Sequenti al Unconstrai ned
Mi ni mi zati on Techni ques ( SUMT) (1). The i deabehi nd eapproach is torepeatedl y
call the unconstai 1/ ed opti mi zati o1/ al gori thmusi ng the sol uti on of the previ ous
7.2 1NDI RECTMETHODS FORCONSTRAI NEDOPTI MI ZATl ON 271
r
t
4
4
j
5
3
2
iteration. le uncons ai ned al gori thmitself executes many i terati ons. Thi s woul d
r ui a robust unconstrai ned mi ni mi zer to handl e a l arge cl ass of probems. The
BFGS met hod is robust and i mpressi ve over a l argecl ass of probl ems.
A preprocessi ng l ask i nvol ves tr f ormi ng the constrai ned probl emi nto an
unconstrai ned probl em. Thi s is l argel y accompl i shed by augmenti ng the obj ecti ve
functi on wi th addi ti onal functi ons refl ect e vi ol ati on of thenstrai nts very
si gni fi cantl y. These ncti ons are referred to as e penal f cti ons. le W
si gni fi cant acti vi ty in is at one ti me whi ch l ed to enti re farnilies of di fferent
types of penal l y functi on methods [2). The firsl of these was the Ex1eri or Penal ty
Functi on (EPF) method. The first versi on of ANSYS [3] i ncorpora1ed
opti mi zati onin its finiteel emenl programrel i edon the EPF. The EPF (presentedin
e next secti on) had several shortcomi ngs. To address those e I nteri or Penal ty
Functi on (I PF) methods were devel oped l eadi ng 10 the Vari abl e Penal ty Functi on
( VPF) methods. I n is text . onl y the EPF is addressed l argel y due to academi c
i nterest . I nvi ewof eexcel l ent perf ormance of di rect methods these methods
wi l l probabl y not be uscd today for conti nuous probl ems. They arc once agai n
i mportant in gl obal opti mi zati orl techni ques for constrai ned probl ems. The second
me odp sentedinthis secti on l heAugment ed Lagrange Met hod ( ALM) is l hebesl
of the SUMT. lts exceedi ngl y si mpl e i mpl ementati on its qual i ty of sol uti on and its
ability to generate i nfonnati on on the Lagrange mul ti p1i ers al l owit to seri ousl y
chal l enge di rect techni ques.
7. 2. 1 Exterl or Penal t y Funct l on ( EP Met hod
The transformati on of thc opti mi zati on probl em(7. 1) (7.4) 10 an unconstrai ned
probl emis made possi bl e through a penal ty functi on fonnul ati on. The transf onned
unconstrai nedprobl emis:
Mi ni mi ze F( X '11' 'g) = f (X) + p(X '11 ') (7.19)
(7.4) X: :S; X/ :S; i = 1 2 1/
where P( X 'h 'g) is thepenal ty functi on. '11 and 'garepenal ty constants (al socal l ed
mul tJlies.)
The penal ty functi onis expressed
p( X ' h' ') = ' h1 L. "k(X)21+ '81( m {O (X) })21 (7.20)
1"'1 l t1
InEquati on(7.20) if eequal i tycons ints arenot zero their val ue gets squared and
mul ti pl i edby thepenal ty mul ti pl i er and thengets added to eobj ecti vefuncti on. If
th i nequal i ty constrai nt is invi ol ati on it toogets squared and added 10 obj ecti ve
functi onafter bei ngampl i fi cdby penal ty mul ti pl i ers. Ina sense if theconstrai nts
areno1satisfied then eyarepeni zedhence thefuncti on' s n ne. It canbe shown
thc l sfonncdunns n probl emsol ves eoriginal m nedprobl emas the
multipliers r" rg approach . In order for P to remai n finile at e val ues of e
multipliers (1 dcdfor a validsolution) econstraints must be satistied. Incomputer
i mpl ementati on is limit is pl aced by a large val ue ins ad of . Another facet of
mputer i mpl ementati onof this method is alargeval ueof themultipliers at the t
ilerationisbound tocreatenumeri cal diffi lties. l esem tiplier arestartedwi thsmal1
val ues du a gme lly chiteration. eunconslraI n chni quefor
ex DFPwi1 sol ve Equati on (7.19) for a known val ue of multiplicrs. The
solution ncdfromtheDFP canbe consi der as a functi onof thcmultiplier andcan
be thought of as
X. = X.(r"r
8
) (7.21)
The SUMT iteration i nvol ves updati ngthemul ti pl i ers and einitial desi gnvector
and cal l i ng eunconstrai nedmi ni mi zer agai n. In al gori thmit is assumed that e
DFP method is used (al though any method f romChapter 6 canbe used BFOS is
rcommended).
Al gorl t hm: Exterl or Penal t y Funct i on ( EP Met hod 2)
Step 1. Choose X
I
N. (no. of SUMT iterations) N
u
(no. of DFP iterations)
E;'S(for convergence andstoppi ng)
r! (initial penal tymul ti pl i ers)
Ch. c/I (scal i ngval uefor mul ti pl i ers)
q = 1 ( SUMT iterationcounter)
Step2. Cal l DFP tomi ni mi ze F( X
q
r%
OutpUl: xq.
Step3. Convergence for EPF
If hk = 0 for k = 1 2. . . . 1;
If gj :5O. for j = 1. 2. . . . m;
If all si deconstrai nts aresatisfied
Then Converged Stop
Sl oppi ngCriteria:
d. F = F
Q
- F 0-" d X = X
q
* - X( q- I ) *
q q-- I
If (d.Fl :5EI: Stop(functi onnot changi ng)
El seIf d. X
T
d. X:5EI: Stop(desi gnnot changi ng)
self q = N.: Sl Op( maxi mumiterations reached)
Conti nue
qq+l
r: r1Cg
xqxq.
00 toStep2
1
7.2 I NDI RECTMETHODS FORCONSTRAI NEDOPTI MI ZATI ON 273
The EPF is very sensi ti vetothestartingval ueof themul ti pl i ers and toth scal i ng
factors as well. Di fferent probl ems respond favorabl y to di fferent val ues of the
mul ti pl i rs. It is recommended that theinitial val ues of themul ti pl i ers be chosen as
theratioof theobj ecti vefunclI on tothecorrespondi ngtennin penal tyfuncti ona1
theinitial desi gn. Thi s ensures at boththeobj ecti vefuncti onand theconstrai nts are
equal l y i mportant in deteni ni ng the changes in the desi gn for succeedi ng
lteration.
One reasonfor thetennExteri or Penal ty is that at theend of eachSUMT iteration
desi gn will be il asi bl e (until the sol uti on is obtai ned). Thi s i mpl i es that c
method detenni nes desi gn val ues areapproachi ng efeasi bl eregi onf romthe
outsi de. Thi s is a seri ous drawback if themethod fails p maturel yas it will oftendo.
Thc i nformati ongeneratedsofar is val uel ess as thedesi gns werc never feasible. As
seenintheexampl e bel ow. theEPF severel yi ncreases enonl i neari tyof theprobl em
creati ng condi ti ons for the me od to fai l . Il is expected the i ncrease in
nonl i neari ty is bal anced by a coser starting val ue for the desi gn as each SUMT
iterationstarts coser tothesol uti onthan cprevi ous one.
lnthefol l owi ngtheEPF is appl i edtoExarnpl e7.1througha series of cal cul ati ons
rather an through the translationof eal gori thminto MATl AB code. Thereare a
coupl eof changes wi threspect toAl gori thm(A7.1). To resol vethepenal ty functi on
wi threspect to ei nequal i ty constrai nt the constrai nt is assumed to al ways be in
vi ol ati on80 at ereturnf romtheml l xfuncti onis econstrai nt functi onitsel f. Thi s
wi1l dri ve the i nequal i ty constrai nt to be acti ve whi ch we knowto be true for this
exampl e. Numeri cal i mpl ementati on as outl i ned in the al gori thmshoul d al l ow
detenni nati on of i nacti ve constraints. I nstead of numeri cal i mpl ementati on of the
unconstrai nedprobl em ananalytical sol uti onis determi nedusi ngMATLAB symbol i c
computati on. Exampl e 7.1 is reproducedfor conveni ence.
Mi ni mi ze f(Xj oX2): +4 (7.9
Subj ect: h(x( X2): (7. l Oa)
g(x( :50 (7.LOb)
o :5Xl :55; 0 :5X2:55 (7.l 0c)
Fi gure7.2is thccontour plot of thetransformedunconstrai ncdfuncti onfor val ues of
rh= 1 and =1. The i ncreaseinnonl i neari tyis rcadi 1y appar nt . Fi gure7.3is theplot
for rh= 5 and rg= 5. Handl i ng esefuncti ons numeri cal1y evenwi ththeBFOS wi l1
be a consi derabl echa11enge. Both thefi gures suggest s veral poi nts at satisfy first
order condi ti ons. Thei r cl oseness m it difficult for any numeri cal techni quetofi nd
theopti mum. l t iscl ear theEPF severel yi ncreases thenonl i neari tyofthe probl em.
Sec7_2_1-pl ol mcontai ns the code t generates the pl ot . Si nce the code uses
symbol i c mani pul ati on. actual l y drawi ng tlle plot takes ti me. Eval uati ng the data
Dumeri cal l y will make a bi gdi fference.
274
4
NUMERI CAL TECHNI QUES FORCONSTRAI NEDOPTI MI ZAnON
7..;..;..:
..- -
x
t
z
. . . ' "
3 4
Fl gure7.2 E or Penal tyFuncl l onMel hod: Exampl e 7.1 (n. = 1 rg= 1).
4
5
Appl l cati on 0 ' EPF A orl thm(A7. 2): The correspondi ng unconstrai ned func-
ti onfor is probl emis

+4) +
+ xi - 2)2+ ' 8(0 xf +O 1)2 (7.22)
Sec7 _2_1_cal c. mis an m- fiI e that wi l l cal cul ate esoh onand theval ues of e
functi on. for Exampl e 7.1. for a predetermi ned set of val ues of thepenal ty mul ti pl i ers.
It requi res t wo i nputs mtheuser at di fferent stages. The first i nput is theval ues for
emul ti pl i ers for whi ch esol uti onshoul d be obtai ned. leI i st of sol uti on (there
areni ne for this probl em) is di spl ayed in Command wi ndow. The user fi nds e
sol uti onthat satisfies thesi deconstrai nts (usual l y one) whi ch must be enteredut e
prompt (note: it must be enteredas a vector). The programthenpri nts out eval ues
of e vari ous functi ons i nvol vedin e exampl e. The fol l owi ngis postedf romthe
Command wi ndowfor bothof thepenal ty mul ti pl i er set to25.
Sec7 2 1 cal c
ent er val ue f or r h [ def aul t 1] 25
3
l
N
M
7.2 I NDI RECTMETHODS FORCONSTRAI NEDOPTI MI ZATI ON
5
$everc nonllnearltywlthmany locol ml nl mums
41
3
..
x
t
Fl gure7.3 Exterior Penal tyFuncti onMethod: Exampl e 7.1 (n. = 5. fg= 5).
ent er val ue f or r g [ def aul t 1] 25
ans
- 1. 4125 - 0. 1015
- 0. 8692 1. 0980
- 0. 6263 - 1. 2107
- 0. 0278 - 1. 3266
- 0. 0124 1. 3267
- 0. 0090 - 0. 0000
0. 7893 - 1. 1267
0. 9775 1. 0165
1. 4184 - 0. 1318
I nput t he desi gn vect or chosen f or eval uat i on
[ 0. 9775 1. 01651
The desi gn vect or [ 0. 9775 1. 0165 1
obj ect i ve f unct i on: 2. 9810
equal i t y const r ai nt : 0. 0112
i nequal i t y const r ai nt : 0. 0138
275
5
277
Here ).. is themul ti pl i er vector ti edto the ual i ty cons ai ntsp is themul ti pl i er
vector associ ated wi l h the i nequal i ty constrai nts and rh and rs are the penal ty
mul ti pl i ers used si mi l ar to l he EPF method. F is sol ved as an unconstrai ned
functi on for predetermi ned val ues of p r" and rg. Theref ore the sol uti on for
each SUMT i terati onis
7. 2 I NDI RECT METHODS F OR CONSTRAI NED OPTI MI ZATI ON
NUMERI CAL TECHNI OUES F OR CONSTRAI NED OPTI MI ZATI ON
In the above run the equal i ty and the i nequal i ty nstr nt ar not sati sfi ed (
expected). Tabl e 7.1 documents theresul ts of theappl i cati on of eEPF met hod to
Exampl e 7.1throughSec7 _2_1_cal c. m. lescal i ngfactor is5 for bothmul ti pl i ers.
A gl ance al Tabl e 7.1cl earl y i1I ustrates thecharacteri sti cs of theEPF met hod . As the
val ues of themul ti pl i ers i nc ase:
276
X. =X( pr
h
rg)
. The desi gnapproaches eopti mal val ue.
The constrai nt vi ol ati ons decrease.
The sol uti onis bei ngapproached f romoul si dethefeasi bl eregi on.
At eend of theSUMT ilerationl heval ues of themul ti pl i ers and pnal ty constants
areupdated. The latter areusual l ygeometri cal l y scal edbut unl i keEPF do not have to
be i vento for convergence.
Al gorl thm: Augment ed Lsgr ange Mu ipller LM) Met hod 7.3)
Step1. Choose X
1
N. (no. of SUMT i terati ons)
N
u
(no. of DFP
E j'S (fi or c o nv rgenceand stoppi ng)
r ; . r4; (Oi n itia a1pena aIty ml II u llti i Plie
Ch c
g
(sca1ing val uefor mul ti pl i ers)
val uefor mul ti pl i er)
.JJI (initial mul ti pl i er vectors)
q = 1 ( SUMT iterationcounter)
Step2. Ca11 DFP tomi ni mi ze F ( Xq )..q p" rZ r:)
Out put: xq.
Step3. Convergence for AL M
If h
k
= 0 for k = 1 2. . . . 1;
l f g
j
: 0for j = 1. 2 . . m;
( >Of or =0)
( I f Vf +k ' kVhk
If all si d constrai nts aresatisfi
Then Converged Stop
Stoppi ng Cri teri a:
M' =F q- Fq- 1aJ ( = xq* - X<q- I)
lf ( M' )2 S EJ: Stop (functi onnot changi ng)
seIf aJ (T d. XS EI : Stop (desi gnnot changi ng)
El seIf q = N.: Stop ( maxi mumi terati ons reached)
An i mportant note ds to be mad. le anal yti cal computati on usi ng symbol i c
computati on appear sy. The same cannot be sai dfor thenumeri cal computati on.
especi a1ly l ooki ngat Fi gure 7.3.
Augment ed Lagr ange Mul ti pl i er ( ALM) Met hod
Thi s is the most robust of e pena1ty functi on methods. Mor e i mportantl y it a1so
provi des i nformati onon eLagrange mul ti pl i ers at esol uti on. Thi s is achi evedby
not sol vi ngfor the mul ti pl i ers but merel y updati ngt hemduri ngsuccessi ve SUMT
i terati ons [24). It overcom s many of the difficulties associ ated wi the pena1ty
functi onformul ati onwi thout any si gni fi cant overhend.
7. 2. 2
Tr ans rmat l onto the Unconst r al ned Pr obl em: The genera1 opti mi zati on
probl em(7.1)(7.4) is l sform i n odof Lagrange mul ti pl i ers
(7.23)
(7.4)
Mi ni mi ze
r8): f ( X) d X ) z d)+
hk
(X) )
Tabl e 7.1
i = 1. 2..... n
Exl eri or Penal l y Functi onMel hod: Exampl e 7.1
xls S x !
Conti nue
qq+l
+2rhh(X"O)
pqpq+2 r
ll
( max (xq*). -pqI 2r
8
])
r1. rrCh; r: r f Cg
s
0. 0227
0.0295
0.0138
0. 0037
7. 5097e004
0. 0687
-0. 0302
-0. 0112
-0. 27
-5. 9775e-004
f
2.9578
2.9651
2. 9810
2.99
2.9988
X2
1. 0391
1.0364
1. 0165
1. 0044
1.0009
X
1
0.9228
0. 9464
0.9775
0. 9942
0.9988
r
8
1. 0
5.0
25. 0
125
625
rh
1. 0
5.0
25.0
125
625
Iteration
'
a
q
J
A
-
-
J
278 NUMERI CAL TECHNI OUESFORCONSTRAI NEDOPTI MI ZATI ON
x
q
x
q
*
00 toStep2
The AL M algori m(A7. 3) is avai l abl e as a stand-al oneprogramin ALM. m. le
programis l ongwhi ch is not unexpected. Thesea1gori thrns arenot vial. To translate
al gori thm(A7. 3) intoworki ng code is a mendous accompl i shment parti cul arl yif
yourexpo toMATI . AB occurredthrough isbook. The reader shoul dwal k through
the code line by line. e organi zati on and structure are kept si mpl e and
strai ghtforwal' d. Conunenl s hav been liberally used as an cncouragement to
underst.1ndthecode.
ALM. m: The code ga ers i nput f romthe user through several prompts in the
Conunand wi ndow. The prompts and final iteration ar copi ed bel ow. The
si gni fi cant features of thecode are:
The startingdesi gnvector is thefirst i nput to progr n. The number of desi gn
vari abl es is automati cal l yextractedf romthis i nformati on.
le l ower and upper bounds on desi gn v abl es (side cons nts)
obtai ned roughuser prompts (though eyarenot usedin eprograrn).
Number of equal i ty constrai nts and correspondi ng initial mul ti pl i ers ()
promptcd.
Number of i nequaJ ity nstraints and correspondi nginitial mul ti pl i ers ( )
promptcd.
Val ues at cachSUMT itcration pri ntcdintheCommand wi ndow. Wami ngs
generatedby MATLAB swi tchedoff.
The DFP. mis used for uncons i ned opti mi zati on. Thi s programuses
Gol dSecti onnVa r
oeprogramexP ctsthefol l owi ng estobe avai l abl ei n ep a:
Obj ecti vefuncti on Of un. m
Equalityconstrai nts Hf un. m
Inequal i tyconstrai nts Gf un. m
Unconstrai nedfunctI on FALM. m
The programuses gl obal statements tocommuni cate mul ti pl i er val ues.
Theinitial penal tymul ti pl i ers arecomputcd automati cal l y.
Several parameters arecoded into eprogram. especi aIl y thoseneeded for the
gol densecti onand the upper bound cal cul ati on. These canbe changed by
usr. if necessary.
Appl i cati on to Exampl e 7.1: Thc fol l owi ng represents the initial prompts
secn in the Command wi ndowand the val ues for thc starti ng iterations. Other
i terati ons are si mi 1arly di spl ayed. Al so inc1uded bel owis a consol i datedpri nti ng
7.2 I NDI RECTMETHODS FORCONSTRAI NEDOPTI MI ZATI ON 279
ofval ues for a11ofthe SUMT i terati ons at thetermi nati onofthe program. The initial
desi gnvector is [ 32 ]. Initial = 1 and = 1.
AL M
1nput t he st ar t i ng desi gn vect or
Thi s i s mandat or y as t her e i s no def aul t vect or set up
The l engt h of your vect or i ndi cat es t he number of
unknowns (n)
Pl ease ent er i t now and hi t r et ur n f or exampl e [ 1 2
3 . . . ]
[ 3 2]
t
The i ni t i al desi gn vect or :
3 2
1nput t he mi ni mumval ues f or t he desi gn vect or . .
These ar e i nput as a vect or . The def aul t val ues ar e
3 * st ar t val ue bel owt he st ar t val ue unl ess i t i s zero.
1n t hat case i t i s - 5:
[0 0]
The mi ni mumval ues f or t he desi gn var i abl es ar e 0 0
-
1
!
c
i
-
-
1nput t he maxi mumval ues f or t he desi gn vect or .
These ar e i nput as a vect or . The def aul t val ues ar e
3 * st ar t val ue above t he st ar t val ue.
1f st ar t val ue i s 0 t hen i t i s +5:
[5 51
The maxi mumval ues f or t he desi gn var i abl es ar e 5 5
Number of equal i t y const r ai nt s [0] 1
1ni t i al val ues f or l ambda 1
Number of i nequal i t y const r ai nt s [0] 1
1ni t i al val ues f or bet a 1
ALM i t er at i on number : 0
Desi gn Vect or (X) 3 2
Obj ect i ve f unct i on 64
Squar e Er r or i n const r ai nt s( h g) 1. 2100e+002
1. 8063e+001
Lagr ange Mul t i pl i er s ( l ambda bet a) : 1 1
Penal t y Mul t i pl i er s ( rh r g) : 5. 2893e- 001 3. 5433e+000
280 NUMEAI CAL TECHNI QUES FOA CONSTRAI NEDOPTI MI ZATI ON
( ot her i t er at i on omi t t ed)
X not changi ng O. OOOE+OOO r eached i n 5 i t er at i ons
The val ues f or x and f and 9 and h ar e
3. 0000e+000 2. OOOOe+OOO 6. 4000e+001 1. 8063e+001
1. 2100e+002
7. 9603e- 001 7. 1223e- 001 2. 9443e+000 0 7. 379ge- 001
9. 9937e- 001 1. 0092e+000 2. 9995e+000 1. 8223e- 004
2. 9434e- 004
9. 9607e- 001 1. 0013e+000 2. 9962e+000 0 2. 731ge- 005
9. 9940e- 001 1. 0002e+000 2. 9994e+000 2. 5872e- 011
6. 2772e- 007
9. 9940e- 001 1. 0002e+000 2. 9994e+000 2. 5872e- 011
6. 2772e- 007
The val ues f or l amda and bet a ar e
1. 0000e+000 1. 0000e+000
- 9. 0876e- 001 0
- 1. 6493e 001 9. 5663e- 001
9. 1192e- 002 9. 5616e- 001
7. 6430e- 002 9. 9221e- 001
Th programtook fi vei terati ons (lifthwas not necessary) tofi ndthesol uti onas:
x;=O. 94; x ; = 1. 0002; r = 2. 9994; h. = 0. 0000; g. = 0. 0007 (7.24)
whi ch is very c10se totheactual sol uti on. The val ues for emul ti pl i ers
A.' = - 0. 0763; s" = 0. 9922
(7.25)
Thi s satisfies the K T condi ti ons. However in Secti on 7. 1. 2 the val ues of the
mul ti pl i ers obtai nedusi ngsymbol i c computati on arequi tedi fferent. It is di ffi cul t to
expl ai nthedi screpancy except tosay thesymbol i c cal cul ati onfai l edtodi scover
this parti cul ar sol uti on whi ch is i ndeed possi bl e. le val ues in (7.25) ar mo
acceptabl e anthose inSecti on7.1.2consi deri ng eK T condi ti ons.
TheAL Mme odas demonstral ed above is effecl i veand useful met hod insDi te
of bel ongi ng tothecl ass of i ndi rect me ods. It has several si gni fi cant advantages over
theother penal l y functi onformul ati ons [2]. Som of t hemarelisledbel ow:
The met hod is not sensi ti ve to the initial penal ty mul ti pl i ers or thei r scal i ng
strategy.
T he met hod does not qui re e penal ty mul ti pl i ers to be dri ven to extreme
val ues 10tabl i sh convergence.
The equal i ty and acti veconstrai nt can be sati sfi edprl isely.
3
7.3 DI AECT METHODS FOA CONSTRAI NEDOPTI MI ZATI ON 281
. The starti ngdesi gn can be feasi bl eor i nfeasi bl e.
The initial choi ces for th mul ti pl i ers nbe rel axed.
Thereis onl y a modest i ncreaseinthecal cul ati ons.
At convergence theLagrange mul ti pl i ers wi1l be dri ven10tlleir opt i mumval ues.
Thi s aJl ows veri fi cati on of K T condi ti ons through the si gn of the mul ti pl i ers
associ atedwi eacti vei nequal i tycons ai nts.
7. 3 DI RECT MET HODS F OR CONSTRAI NED O PTl MI ZATl O N
: 1
The di rect met hods inc1ude both the obj ecti ve d .e constr nts to search for the
opti mal sol uti on. Whi l e emet hods do not i nvol ve conversi on toa di fferent c1ass of
probl ems (l i keSecti on7.2) most of t hemareb edon l i neari zati onof thefuncl i ons
about the currenl desi poi nl. Li neari zati on is based on expansi on of the functi on
about thecurrent vari abl eval ues usi ngtheTayl or seri (Chapter 4).
Unesrl zatl on: The Tayl or seri es for a l Wo.vari abl e expanded functi onj{x y)
quadrati cal l y about iliecurrent poi nt (xp' Yp) is expressed as:
1 ) = f (xf" y/) -1- [ J 13{J +
! I +2 + )
L _.. I (x.JI) ----" I (xpJ') -" I ( J
r
i
y
c
J
u
u
d
I f e di spl acements areorgani zed a col umn v [ ax 8y]T. expansi on in

Equati on(7.26) can be expressed ina condensed manner
f ( ) = f
L[ A y ]T[H(;Xn.Yn IJ] ]I ( 7.27)
. . I sYI
For It vari abl es. wi th X
p
thecurrent poi nt and 8 x thedi spl acement vector
f ) = f ( X
p
) + Vfl p)T8 X+t d XT H )
Equati on(7.28) c bewri l teninterms of edi fferenceinfuncti onval ues
=f ( X
p
+ f ( X
p
) = Vf (Xp)T dX + t dX
T
X
p
) )
df =Of +o
2
f
282 NUMERI CAL TECHNI QUES FOR CONSTRAI NED OPTI MI ZATI ON
where of = Vj T a X is tenned Ji rst variation. o2j is thesecond vari al i onand isgi ven
by es ondtenn intheabove uati on. Inl i neari zati onof thef uncti onf about the
current val ueof desi gnX
p
' onl y the fi vari ati onis used. Th nei ghbori ng ueof
thefuncti oncanbe expressedas
l ( X
p
) = f ( X
p
) + V f ( X
p
) T a X (7.30)
Al l of the functi ons e probl emc be l i neari zed si mi l ly. It is essenti aI to
understand di fference bet weenf (X) andl ( Xp)' Thi s is ilI ustrated inFi gure 7.4
usi ngtheobj ecti vefuncti onof Exnmpl e 7.1expanded about thecurrent desi gnx = 3
x2=2. Th curved I i nesf ( X) are the contours of eori gi naI functi on. The s Bi ght
l i nes J (X
p
) arethecontours of thel i neari zedfuncti on(l i nes of constant vaI ue of the
functi on) obtai nedthroughthefol l owi ng:
Ori gi naI i on: f(x). X2): x1- 2 xr X2 + xi + ~ - 2x
1
+ 4
Li neari zedfuncti on: ](!J. XI ' !J. X2) =+92!J. X I - 6!J. X2
wher e the coeffi ci ents in second expressi on i ncl udes the eval uati on of the
functi onand gradi ent at x = 3 X2 = 2 and substi tutedinE ati on(7.30). InFi gure
7. 4several contours areshown al though those of intrest areinthenei ghborhood
of ecurrent desi gn. Any nonl i near functi oncan be expanded i nthesame manner.
If another poi nt was chosen. then esl opes 01 " Lhe lines woul d be di fferent .
The quadrati c expansi on of thefuncti oncan be obtai nedby usi ngEqu on(7.28).
leexpanded curves will be nonl i near (quadr c). l eyappear el l i psi nFi gw 7.5
where severaI contours of e 0ecti ve functi on of Exampl e 7. 1 expanded
quadrati l I ya ut XI = 3 X2 = 2. It is i mportant to ogni zethat econtour woul dbe
di fferent if another poi nt we chosenfor expansi on.
Four di rect methods ar di scussed The first is Sequenti aI Li near Prograrnmi ng
(SLP) where the sol uti on is obtai ned by successi vel y sol vi ng the correspondi ng
l i neari zed opti mi zati on probl em. The s ondSequenti aI Quadrati c Progr ammi ng
( SQP) uses equadrati c expansi on for eobj ecti vefuncti on. Li ke e SUMT e
current sol uti on provi des e starti ng vaI ues for the next iteration. The third. e
GeneraI i zed Reduced Gradi ent Met hod ( GRG) . devel ops a sophi sti cated s h
di recti on and fol l ows it wi th an el aborateone-di mensi onal process. The fourth. Lhe
Sequenti a1 Gradi ent Restorati on Al gori thm( SGRA) uses a two-cycl e approach
worki ng on feasi bi l i ty and opti mal i ty a1tematel y to fi nd the opti mum. l ere are
severa1other methods due to many researchers but ey di ffer f rom ose listed in
smal l details. To k p the l ength of this chapter reasonabl e. ey have not been
incJ uded. Wi t h ecode dother procedures avai l abl ef romthis secti on. it shoul d not
bedi fcul t toprogramth addi ti onal techni ques at areworth expl ori ngfor parti cul ar
cJ asses of probl ems.
Except for speci al cl asses of probl ems inthefol l owi ngt wo chapl ers. is chapter
is the pri nci pal reason for the book. It provi des a resol uti on of the compl ete
opti mi zati onprobl emas posl ated. For a l ongti me now therehave not been any new
or greatl yi mproved numeri cal techni qu. lereis a lot of wor k on gl obal opti mi zati on.
7.3 DI RECT METHODS F OR CONSTRAI NED OPTI MI ZATI ON 283
N
M
5
4
3
t
2
2
2
d
t
4
A
f
F
d
- 1
x
t
F1gure 7.4 lII ustratlonolllnearizatlon.
n Current i nterest in tradi ti onal desi opti ti on ( is chap) is di rectedtoward
actual desi gnappucadons and mi grati onf mmt hemai nf mme and UNI X Envi ronment
to e P C (like is book) For compl eteness. the standard f onnat of the NL P is
reproduced herefor conveni ence:
Mi ni mi ze !(x1oX2. x)
(7. 1)
Subj ect:
ht<X" X2. X
II
) = o. k = 1 2. . . . 1
(7.2)
~ X I X 2 ) O. j =12. . . . m
(7.3)
Xl S: Xi S: Xr . i =1. 2. . . . n
(7.4)
Invector notati on
Mi ni mi ze f( X). ] "
(7.5)
Subj tto:
[h(X)]=O
(7.6)
284 NUMERI CAL TECHNI OUESFORCONSTRAI NEDOPTI MI ZATI ON
5
4
3 J

2"'"
- 2
2
I

- 1
x
v
2 3
FJgure7.5 lII ustration01quadratlc9xpanslon.
[g(X)]mS 0
X10wSXSXup
4
(7.7)
(7.8)
Exampl e 7.1 will beusedtoilI ustratethevari ous al gori thms.
Mi ni mi ze f(x. xi): x1- 2xi x2+xi +xl r z- 2xl +4 (7. 9)
Subj ectto: h(XI x:z): xi + rz - 2 = 0 (7.10
g(X..X2): SO (7.10b)
OSXI ::;5; OSX2SS (7. lOc)
7.3.1 Sequentl al Li near Pr ogr amml ng (SLP)
5
IntheSLP [5] all of thefuncti ons areexpanded linearly. If X; isconsi dered ecurrent
desi gnv tor en elinearizedopti mal probl emcanbeset up as
7.3 DI RECTMETHODS FORCONSTRAI NEDOPTI MI ZATl ON 285
Mi ni mi ze: l(aX) = f(X;) + Vf(X; ) T a X (7.31)
Su tlo: aX) : "k(X;) + vhI (X;)a X = 0; k = 12 . . . 1 (7.32)
(AX): (X;) + V8J{X;)TAX S 0; j = 12.. .. m (7.33)
Ax!ow ax; S ax)IP; i = 12 .. . n (7.34)
Equati ons (7.31) (7.34) represenl a Li near Programmi ng (LP) probl em(Chapter 3).
Al l of thefuncti ons inEquati ons (7.31)(7.34) except for aX. have numeri cal val ues
after substi tuti ng a numeri cal vector for X;. Assumi ng an LP programcode is
avai l abl e it nbe cal l edrepeatedl yafter thedesi gnis updatedas
X" =X+AX
Inorder toi ncl udea searchdi recti onand stcpsi zecal cul ati on X inEquati ons
(7.31)(7.33) is consi dered as a sol uti on for S (a search di recti on at the currenl
desi gn). Stepsi ze computati on slra gy is no l onger si mpl e. I f re are vi ol ated
constrai nts a1pha attempts toreduce is violation. If ecurrent sol uti onis feasible
estepsi zewill attempt toreducethe ncti onwi thout causi ngtheconstraI nts tobe
excessi vel y violated. Such strategies arel argel yi mpl emented at thedi screti onof
al gori thmor code devel oper.
A orl thm: Sequentl al Ll near Pr ogr amml ng ( SLP) (A 4)
Step 1. Choose X' N
s
(no. of iterations)
E;'S(for convergence andstoppi ng)
q = 1 (iterationcounl er)
Step2. Call LP toopti mi ze(7.31)(7.34)
Output: S
Use a constr ned cal cul ati on(X; = xQ)
AX=. . S
Xq+I =Xq+AX
Step3. Convergence for SLP
If h
k
= 0 for k = 1 2 . . . 1;
If gJ S 0 for j = 1. 2 . . . m;
[f all si deconstrai nts aresatisfied
If KT condi ti ons satisfied
Then Converged Stop
Stoppi ngCriteria:
AX=xq+l - x
q
If AX
T
AX S EI: Stop(desi gnnot changi ng)
If q = N.: SI Op( maxi mumiterations ached)
286 NUMERI CAL TECHNI QUESFORCONSTRAI NEOOPTI MI ZATI ON
Conti nue
qq+l
00 toStep2
The SLP is appl i ed10Exampl e 7.1bel ow. InChapter 3 no explicit programfor sol vi ng
eLP programwas devel opedal thougha command- by- command wal k roughwas
i ntroduced. A programfor appl yi ng eSi mpl exme odpsl mpl ex. mis i ncl udedon
ewebsi te(wi thout any comment or di scussi on). Usi ng i nstructi ons f romChapter 3
addi ng somc programcontrol and organi zi ng some bookkeepi ng can provi de a
MATLAB programfor LP probl ems. Al tematel y M ... . AB'Sl i nprogcanbe usedina
l oop to hel p lranslate e SLP al gori thmto code. In is secon e l i neari zed
subprobl cmin each iteration is graphi calI y presented. The area of interest can bc
zoomed and thesol uti onreadfromthepl ot . Thi s is treatedas dX (i nsteadof usi ngit
as S). The desi gnis updatedandanother pl ot canbe obtai nedby runni ngthemfile
for a newval ue of thedesi gn vector. Sec7 _3_1- p1ot. m assists in di spl ayi ng the
I i neari zedfuncti ons f romwhi ch sol uti oncanbe obtai ned. The equal i tyconstrai nt is
egr nline. leredis i nequal i l y constr i nt ledesi gn vector is putto e
program. Thdeuses symbol i c cal cul ati on10sol veand di spl aythesubprobl em.
Appl l catl on of SLP
sP 1. X
l
= ( 32)
Step2. Li neari zedsubprobl em r .
Mi ni mi ze 1(dX) = 64 + [92 -6] I I
I
s X
21
ct II(dX): 11+ [6 4] I 1=0
I !l X21
g(dX): 4.25+ [1. 5 311 I so
I !l X21
=ax
l
=1. 335&2 = ax
2
= - O.75.For
thenext iterationX2 = [ 1. 665 1. 25]. Repeat Step2.
Step2. Li nari zedsubprobl em
Mi ni mi ) = 6. 7985+ [13 05 -1. 382] 1
I LU21
S : ( ) : 23347+[ 333251l h=0
I
s X
21
l ax. l -
g(d X 9+ 25 1. 8751/ I SO
7.3 OI RECTMETHODS FORCONSTRAI NEDOPTI MI ZATI ON 287

5
4
-4 - 3 - 2 4 5 2 3 o
OX
1
Fl gure7.6 Exampl e7.1 SLP ileration1.
-
-
i
3
i
i
f
z
j
-
Fr omFi gure7.7(zoomed) thesol uti onis ax
l
= 0. 54ax2 = 0.23. For thenext
iterationX
3
= [1.125 1. 02]. Repeat Step2.
Step2. Li neari zedsubprobl em
Mi ni m 1(. ) = 3. 206+ [2.3957 -0. 2363] I
l "'21
bject 10: ( ) : 0.3060 + [2.25 2.04] 1 1=0
l "' 2 l
E)::00 6
Fr omFi gure7.8(zoomed ) thesol uti oni sdx ( = 0.125dx2 = 0.02. For thenext
iteration X
4
= [ 1. 0 1. 0 ]. Si nce this is the known sol uti on further iterations
stopped.
leSLP took threeiterations t. ofindthesolution. Thi s is very i mpressi ve. It is true
onl y for th. osecases where thenumber . of equal i typl us acti vei ncqual i tyconstrai nts is
l I i
288 NUMEAI CAL TECHNI QUES FOR CONSTRAI NED OPTI MI ZATI ON
5
4
3
N
>< 0
<0
- 2
- 3
4
-5
5
Fl gure7.7 Exampl e 7.1 SLP l I erallon2.
4 5
l he same as e number of desi gn vari abl es- a ful l comp[ement. The mai n
di sadvantage of l hemel hod appe si f ereis no full compl ement. Inl hat case si de
cons ints (7.34) arecritical to edetenni nati onof l hesol uti onas eydevel op i nto
acti veconstrai nts. Inl hesesi tuati ons l he l i mi ts on l hesi deconstrai nts e cal l edl he
move limits. ln seof acti vesi deconstrni nl s I hemove l i mi l s establ i shtheval ues of
l hedesi gnchanges. If l hedesi gnchanges areconsi der as l hesearchdi cti on(s
al gori l hm) enbo vaJ ueand di recti onarea cted. For l hesechanges tobe smal l
as l hesol uti onis approached emove l i mi ts have tobe adj usted(l owered) wi thevery
i terati on. lestrntegyfor is adj ustrnent may i nfl uence esol uti on.
Exampl e 7.1 wi l hout the i nequal i ty constrai nt but wi l h e equal i ty constrai nt
provi des an illus l i on of the move limits. Fi gu 7.9 represents l he lin ri zati on
about l hepoi nt [ 3 2 J . The move limits inl hefi gu (box) represent l hesi deconstrai nt
l i mi ts
- 2 S2
lt is cl ear f romFi gure 7.9 Ihat wi l houl Ihe move l i mi ts thc sol uti on is unbounded
whi ch is not very hel pful . Usi ng l hesol uti onobtai nedinl hefi gure l henext i terati on
1
7.3 DI AECT METHODS FOA CONSTRAI NED OPTI Ml ZATI ON 289
5
4
3

3
1
-2
- 3
- 4
--55L
- 4 - 3 -2 -1
2 3 4 5
SX
1
Fl gure7.8 Exampl e 7.1 SLP l I eration3.
C beexecuted. Once agai nl hemove lirnits wi l l i nfl uence esol uti on. If themove
l i mi ts are left at l he same val ue the sol uti on wi l l al ways be on l he squ e wi th no
possi bi l i tyof convergence. These lirnits musl be l owered wi th chiteration. Usual l y
they are done geometri cal l y l hrough a scal i ng faclor. lis i mpl i es l heconvergence
wi l l depend on l hestral egy for changi ng l hemove l i mi ts not an appeal i ngsi tuati on.
The SLP is not one of l hepopul ar mel hods. 111emove lirnits areavoi ded wi l henext
me od.
7. 3. 2 Sequent l al Quadr at l c Pr ogr amml ng ( SQP)
The l ack of robustness in eSLPduet o eneed for move lirnits can be countel by
i ncl udi ngan el ement of nonl i n rity in eprobl em. The arc several ways lhis can
be achi eved. One way is 10 requi re l hal search di recti ons ( remember in aca1
i mpl ementati on s X isS) inEquati ons (7.31 ) (7.34) be l i mi tedby theuni t ci rcl ewhi ch
can be descri bed constrai nl :
STSS: 1 (7.35)
290 NUMEAI CAL TECHNI CUES FOACONSTAAI NEDOPTl MI ZATI ON
5
3
2
.
z o
- 1
- 2
-3
-4
-5;
- 4 - 3 - 2 o
6x
Fl gu 7.9 lII ustratlonof move IImlts.
stepsi zedetermi nati onshoul d then account for the actu changein thedesi gn
vector. The probl em expressed by Eq (7.31)(7.35) is a quadmti c
programmi ng (QP) subprobl em. An i temti vesol uti onof this subprobl emc beone
f ormof SQPme od. Thi s odis popul ar and is usedwi del y. Whi l e theQP
subprogramis a si mpl e nonl i near probl emwi th l i near cons ints. it is still a
mul ti vari abl c nonl i near probl em. Fortunatel y QP probl ems appear in many
di sci pl i nes and thereareeffi ci ent procedures and al gori thms toaddress theseprobl erns
[6-10]. One of thse methods is an extensi on of the Si mpl ex met hod of Li near
Programmi ng.
lesol uti onto eQP is wel l documented. It woul d r l ui remuch di scussi onand
thereforeis not i ntroducedhcre. Readers arereferredto thevari ous references. Thi s
is al so e pri mary reason why the QP met hod is avai l abl e in the Opti mi zati on
Tool box inMATLAs. The formal methods for QP probl ems arebased on effi ci ency in
thevari ous stages of thei mpl ementati on. An i neffi ci ent but successful sol uti on10the
probl emcan be obtai nedby usi ngI heA L M of theprevi ous secti onto sol vetheQP
subprobl em. After all. computi ng resources arenot a constrai nt today. Inlieuof formal
di scussi on. an i ntui ti veunderstandi ngof enumeri cal met hod is attemp d inthis
book. The transl ati onof SQP i ntocode is not trivial and themet hod isusual l ythebasi s
of commerci al software. For exampl e SQP is the onl y al gori thmfor constrai ned
7.3 DI AECT METHODS FOA CONSTAAI NEDOPTl MI ZATI ON 291
opt i mum MATLAB i mpl ements (as of this wri ti ng). Havi ng access to the
Opti mi zati onTool box al l ows NL P tobe sol vedusi ngSQP. A detai l eddi scussi onof
thesol uti ontechni que is al soavai l abl e[11].
le QP subprobl emempl oyed in this subsecti on is based on expandi ng thc
obj ecti ve functi on quadrati cal l y about the current desi gn. le constrai nts are
expanded l i nearl y as in SLP [8]. is is cal l edsequenti al quadrati c programmi ng
(SQP).
Mi ni m 1(d.X) =f' ) + Vf ( Xi
r
d. X+ T V
2
fl d X )
Subj lto: ( ) : hk(Xj ) + VhI ( X) =0; k = 1. 2. . . . . 1 (7.32)
gj ( Xj) + VgJ{Xj)Td. XS 0; j = 1. 2. . . . . m (7.33)
sx}owS sxj S i = 1. 2 ... n (7.34)
In Equati on (7.36) V ( X) is theHessi an matri x. Inactual i mpl ementati on thereal
Hessi an wi l l not be used. I nsteada metri c ] is upda d wi eachi terati onis
used. s is based on success of the Vari abl e Me c Met hods ( V. f ) of the
previ ous chapter. Several researchers [6.9] have shown BFGSupdate for the
Hes8i an provi des an e i ci ent i mpl ementati on of the SQP method. Thi s QP is wel l
posed and convex and shoul dyi el da sol uti on. Sol uti onto this subprobl emis at
heart of e SQP method. A consi derabl e amount of research has been i nvested in
devel opi ng effi ci ent techni ques to handl e is subprobl em. In a formal
i mpl ementati on of SQP d . Xin Equati ons (7.36) d (7.32) (7.34) musl be
pl acedby thesearchdi recti onS. The QP for S al 80modi fi es theconstrai nt equati ons
80that a feasi bl edi recti oncan be f ound wi respectto ecurrent acti veconstrai nts.
Fi gure 7. 10 illus S e concemabout movi ng l i nearl y to a poi nt on an acti ve
con8 ai nt. XO thecurrent desi gnis on anacti veconstrai nt . If thesearchdi recti onSI
fol l ows l i neari zedfuncti onat XO (thetangent to the functi on at that poi nt). any
stepsi ze. al ong E h di recti on. however smal l will cause theconstrai nt to be
vi ol ated. Inorder todetermi ne a nei ghbori ngpoi nt that will stil sati sfytheconstmi nts.
a search di recti on sl i ghtl y less th e tangent is used. This i ntroduces a l ack of
consi stency as thedevi ati onf romtangency becomes a matter of i ndi vi dual experi ence
andpracti . Experi ments have suggested at 90- 95%of thetangent isa useful fi gure
al thoughmaki ng it as cl oseto100%is recommended [2]. be searchdi recti onfi ndi ng
QP subprobl emis
Mi ni mi ze 7(S) = f ( X
1
) + Vf(Xj )T ST[H] S (7.37)
Subj ecl : (S): c MX
i
) + VhI ( X) S=O; k = 1. 2. . . . 1 (7.38)
(S): cgi<Xi) + V 8.x)'f S S 0; j = 1. 2.. m (7.39)
s}OWSSj SSr"; i =I . 2. . . . . n (7.40)
Rep ci ngHessi an wi thBFGS Update: l n Eql n (7.37) cal cul ati ng the
actual Hessi an is strongl y di scouraged. Si nce the BFGS m od converges to the
Hessi an theBFGS metri c is consi deredtobe an excel1ent repl acement of theHessi
[9]. Theme cis based on theLagrangi an of efuncti on[12]:
Cal cul ati ng the Mul fl ptters: The popul ari ty of the SQP m od is that it
attempts to update e Hessi an based on the Lagrangi an of the probl em.
Cal cul ati ng the mul ti pl i ers is necessary to veri fy the Kuhn- Tucker condi ti ons at
th sol uti on. Cal cul ati ngthemul ti pl i ers for thecurrent val ue of thedesi gn vector
is necessary for addi ti onal operati ons i ncl udi ngsetti ngup thedescent functi onin
Equati on (7.43) as wel l as updati ng the metri c repl acement for the Hessi an
(fol l ows next). The cal cul ati onuses th F OCbased on theLagrangi an. The val ues
of thegradi ents of thevari ous functi ons arecomput ed at thecurrent desi gn. The
mul ti pl i ers then obtai ned as a sol uti on to a set of l i near equati ons. The
sol uti ons toQP and SQP probl ems usual l y takeadvantage of th activeconstrai nt
set. Thi s i ncl udes all equal i ty constrai nts and onl y those i nequal i ty constrai nts that
are acti ve. Thi s i mpl i es that SQP i terati ons start at a feasi bl e point. The
mul ti pl i ers correspondi ng to i nacti ve constrai nts are set to zero. The remai ni ng
mul ti pl i ers can be establ i shedconsi deri nga set of an appropri ate number of l i near
i ndependent equati ons f romEquati on (7.12):
Equati on (7.42) or (7.43) can be used toobtai n eval ue g stepsi ze. The functi onin
Equati on (7.43) requi res an i mportant observati on. Shoul d theLagrange mul ti pl i ers
be cal cul atedfor each val ueof ? I magi neusi ngthegol den secti ontosol veEquati on
(7.43) wi th e mul ti pl i ers cal cul ated o mthe sa facti on of e Kubn-Tucker
condi ti ons for each newval ue of X. Several updati ngstrategi es for mul ti pl i ers in
Equati on(7.43) can be f ound in elistdreferencs. A si mpl e one is tohol dtheval ues
of themul ti pi ers constant eprevi ous iterationval ue and change t hemat eend
of theiteration.
(7.43)
(7.41)
. 42)
293
i =12.. . n
(7.12)
7.3 DI RECTMTHODS FOR CONSTRAI NED OPTI MI ZATI ON
n
u
x

J
R
u
r
m
J
r
A
J
+

X

F
J

=

A
V

.
n
u
m
A
m

m
T
L
-
H

r
e
+

4
x

t
n
+

.
r
'
'
E

J

X

A
V

e J F eJ f " dh . oh n e}g 1. . n e}g
= + + +L - +P I ~ n . = O ;
axi a x i f t a x i t a x i f l a x i i a a x i
xj =x;'
1+S
Mi ni mi ze:
Mi ni mi ze:
The si deconstrai nts arenot necessary is is a wel l -posedprobl em. lefactor c is
the adj ustment for movi ng tangent to the constrai nt. le suggested val ue for c is
0. 9-0. 95. For thei nequal i ty constrai nt eval ue of c is 1 if theconstrai nt is satisfied
(shoul dbe abl etomove tangent totheconstrai nt).
Stepsi ze Cal cul ati on: On e search di recti on has been determi ned it is
necess
Y
to cal cul ate thestepsi ze. Si nce the methods of s chapter have seen an
enormous i ncreas inthenumber of cal cul ati ons thestepsi zecal cul ati ons arebased
on sl Ul taneousl y decl ' si ng e obj tive as wel l as i mprovi ng constrai nt
sati sfacti on. Inorder toaccompl i sh this goal a sui tabl efuncti onmust be devel oped and
estepsi zedetermi nati onbased on mi ni mi zi ng this unconstrai nedfuncti on. Several
authors refer toit as thedescent fllnction. Th descri pti onof this functi onvari es f rom
researcher to researcher. Some defi ne is as theLagrangi anfuncti oninwhi cb case
Lagrange mul ti pl i ers must be determi ned. Many others i ncl ude onl y the
constrai nts wi th a penal ty mul ti pl i er and adopt EPF f ormfor functi on. Still
others i ncl udeonl y themaxi mumvi ol atedconstrai nt. Two f orms of thefuncti onaz
i stedbel ow:
4 3
Movl nglinear toanactiveconstraint.
NUMERI CAL TECHN1QUES F O CONSTRA1NEDOPT1Ml ZATI ON
2
Fl gure7. 10
><" 1
-1
292
294 NUMERI CAL TECHNI OUESFORCONSTRAl NEDOPTI MI ZATI ON
p=Vf ( X) +L Ak Vhk(X 1) + sj VCj (X' ' )
1<=1 j=IVjeJ
- Vf( Xq) + ' kV + L s
j
V gi
xq
)
(7.44)
1<=1 j =l l tj eJ
y =Xq+I _ X
q
[Hq+l] = J +
p' y y[Hq]y
j E J i mpIi es theset of acti vei nequal i tyconstraints. It isrecommended me c
be kept posi ti vedefiniteas much as possi bl eeven if it is likely at it wiIl nol be
posi ti vedefiniteat thesolution. One way of ensuri ng is is toverifythat
pTy>O
for eachiteration.
A ' orl thm: Sequenti al Quadrati c Pr ogr amml ng ( SQ 7.
Step1. Choose X I 1 (no. of iterations).
E/S(for convergence andstoppi ng)
q = I (iterationcounl er)
Step2. Cal l QP toopti mi ze(7.37)(7.40) ( X
1
= xq)
Output: S
Use a constrai ned cal cul ati on
AX=" s
Xq+1 =Xq+AX
Step3. Convergence for SQP
I f hk=Of ork=1. 2....l ;
f orj =1. 2... m;
If K T condi ti ons aresatisfied
nConverged. Stop
Stoppi ngCriteria:
AX=xq+l _xq
If AX
T
AX EI: Stop(desi gnnot changi ng)
If q = N
s
: Stop( m umiterations reached)
Conti nue
Updat eMe c[
q q+I
nn rnStpn
l i
7.3 DI RECTMETHODS FORCONSTRAI NEDOPTI MI ZATI ON 295
Appl i cati on 01 SQP: Th SQP techni que is appl i ed to Exarnpl e 7. 1. There are
several ways toaccompl i shthis. The appl i cati oncanusegraphi cal support as shown
wi th the SLP (this is an exercise). The appl i cati on can use the QP programf rom
MATLAB (alsoanexerci se).l n is book QPprobl emis sol vedby appl yi ng eKT
condi ti ons-wbi ch is thefeatureofthi s subsecti on. Ati r all for is specificexampl e
efuncti ons areanal yti cal l ydefi nedand ei rstruc is si mpl e. We canhamess the
symbol i c operati ons MATL tocal cul ate ederi vati ves and eactual Hessi an. As
in e case of SLP the actual chang in tl X is computed [Equati ons (7.36).
(7.32)(7.34)] rather thanthesearchdi recti onS for theQP subprobl em.
Sec7 _3_2. mappl i es theSQP me odsymbol i cal l y toExampl e 7. 1. lt identifies
thesol uti onusi ng eLagrangemethod (Chapter 5). It promp for thecurrent desi gn
vector. It sol ves theQP anal yti cal l y appl yi ngtheKT condi ti ons andprovi di ng e
results for thetwo cases. C ea corresponds tos = 0 (gmust be less anzero). Case
b corresponds tog = 0 (s> 0). The output f romrunni ngtheprogramisrecordedbel ow
for several iterations in is i mpl ementati onof SQP. Ana1ytical sol uti onwi thuser
i nterventi onisus . Sec7 _3_2. mmust berunfor eachiteration. You shoul dbeabl e
tocopy and past desi vector in commandwi ndow.
Iteration1:
Iterati on2:
X
I
=[ 3
M =[2 l=[ ?
l
1;
d.x
l
=- 1. 0591; =1. 1613
=0; g = - 0.8225;
X
1
= 1.9409; =0. 8387
X
2
= [1. 9409 0.8387]
3.123; = U Z ; 1 2 2 2
705; Vh=[ J g=0. 4693; =1 1
The soIuti ontotheQP probl emis Case a:
d.x
l
=0. 6186; d.x
2
=- O.0041
h=O; g=1. 8281; f
296 NUMERI CAl TECHNI QUESFORCONSTRAI NEDOPTI MI ZATI ON
7.3 DI RECTMETHODS FORCONSTRAI NEDOPTI MI ZATI ON 297
1= 1.3222; =0. 7975
Iteration3:
X
3
= [1. 3222 0.7975)
f =4. 2126; = J = 2 2 8 1
=0 =! ? 0858; Vg ;;;l
7. 3. 3 General l zed Reduced Gradl ent ( GRG) Met hod
1 GRGmethod is anothcr popul ar techni que for constrai ned opti mi zati on. le
origina1 method the Reduced Gradi ent Method [13] has seen severa1 different
customi zati ODS due to severa1 researchers [14). Li ke SQP is is ao active set
method- deal s wi th active i nequal i ti es-hence the method iocI udes uality
cons ints only. The in uality cons ints E sformedto equa1ity co aints
usi oga linear sl ackvari abl eof thetypeusedinLP probl ems. ne genera1opti mi zati on
probl emis restructuredas
The SOIUti ODtotheQP probl emis Case a:
Iteration4:
Mi ni mi ze f(XI ' X2 . Xn) (7.45)
dx
l
= - 0.2685; dx
2
= +0. 2042
Subj ect to: ht(XI' x2' x) = 0 k = 1 2 . . . 1 (7.46)
(7.47)
h = 0; g = -0. 0190;
8J (XIoX2' . . . x) + xt+j = 0 j = 12 . . m
X
1
= 1.0537; =1. 0018
xl sx; sxr i =12... n (7.48)
Xn+j 0 j = 12 .. m (7.49)
X
4
= [1. 0537 1. 0018]
f = 3.0685;
; v h=I 2124; g=0 =l T Z l

The total number of vari abl es is n + m. The active !:et !:trntegy ao; sumes that it is
possi bl edef i ne or choose n subset of (11- 1) i ndependent vari abl es omtheset of
(l + m) equal i ty constraints. Once these vari nbl es are estabIi shed lhe dependent
vari abl es (l + m) canbe recoveredthrough consLraints. The number of i ndependent
vari abl es isbasedon coriginnl desi gnvari nbl cs andtheoriginal equa1ityconstrai nts
of theprobl em. The fol1owi ng devel opment is ndaptedf romReference 2.
The set of thedesi gnvari abl es [X] is parti ti onedinto ei ndependentset[Z] d
dependent set [11.
dx. =- 0.0523; L1xz=- O.0017; 0. 9275
=0; g=O;
-
E
J
Z
Y

r
E
-
E
E
a
h
=

x

1. = 1. 0014; =1. ;
The constrai nts areaccumul atedas
f =3. 0014; h=O. 27; g = 0. 0006
H=[ : ]
The sol uti onis a1most convergedand furthcr iterations areleft tothereader. The
next iterationshoul dproduce va1ues for convcrgence.
The SQP heretakes fiveiterations toconverge. This was basedoncomputi ng the
actua1 Hessi an. The SLP took four. The AL M took five. For Exampl e 7. 1 eSQP
does not app r tohave y significaot advaotage whi ch is unexpected. The AL M
slarts to appear qui attractive. The SQP in Sec7 _3_2. mis i mpl emented
anal yti cal 1y basedonKT condi ti ons. It is verydifferent thanAl gori thm( A 7.5) whi ch
rep sents a numeri cal i mpl mentati on.
Re1ations (7.45)(7.49) canbe recasl as
Mi ni mi ze f(Z Y) (7.50)
Subj ect 10: H(Z Y) = 0
z!S z; Szf i =12... n- l
(7.51)
(7.52)
298 NUMERI CAL TECHNI QUES FORCONSTAAI NEDOPTI Ml ZATI ON
A si mi l ar set of si deconstrai nts for Y can be expressed. 1nac al i mpl mental i onthe
i nacti ve in ua i ty constrai nts are not inc1uded in probl em. Al so the sl ack
vari abl es can b ng to the i ndependent . I n GRG lin zed 01 ve is
mi ni mi ztd suhdect tothel i neari zcdconstrai nts about thecumnt desi gnr.
Mi ni mi ze ](l:J.z' l:J. Y) = f ( X
i
) + V zf( Xi ? d. Z+ V yf ( X
i
? l:J. Y (7.53)
Subj ect to: (!:J.Z l:J. Y) = H. ( X
i
) + VzH. (Xi )Td. Z+V.;. (Xi )Tl:J. Y = 0 (7.54)
lf theGRG began at a feasi bl epoi nt. eB( Xi ) =0 elinari zedconstrai nts areaso
zero n i bl eari zedequati ons in ] detenni ne a set of l i near equati ons inl:J. Z and
d.Z. Consi deri ng vector [H] thel i near equati oncan be wri tten
where
[A] d. Z+]l :J . y=o
r r VvHT 1
I Y ZI I J I I ya &1 I
1 1 1 V
[A] = 1 - . - 1 ; [B] = 1 " - 1

I V
z mI I V)
(7.55)
Fr omEq ti on (7.55) the change in edepcndent vari abl el:J. Y can be expressed in
l erms of thechanges inthc i ndependent vari abl es as
l:J. Y =[ Br
l
[A] d. Z
(7.56)
Substi tuti ng Equati on (7.56) i nto Equati on (7.53) reduces i t an unconstrai ned
probl mind.Z:
] ( ) = f ( X
i
) + V zf(Xi )T d. Z- Vyf (Xi )T[Brl [A] d. Z
= f ( X
i
) + {v zf ( XT- Vyf l T[ Brl [A]) l:J. Z (7.57)
= X+{Vzf ( X
i
) ( [ l [ Al f vdxt )
= f ( X
I
) + [GR]T l:J. Z
[ G
R
] is ereducedgradi etJt of thefuncti onj{X)It provi des thesearchdirection. The
i zei s f ound by J 1 i ri ng t ngsi nl:J. YJ1J1e5pondi ng10 ch es
in hz ( S) determi ne a f l sible d gn. Thi s is i mpl emented rough Newt on' s
met hod as descri bed bel ow. Si nce t hehnct i ons ME l i neart hemis no quadrati c
'11111
7.3 DI RECT METHODS FORCONSTRAl NEDOPTI MI ZATI ON 299
convergence associ ated wi is choi ce. The i terati ons start wi th a feasi bl e current
desi gnvector [Xi] = [ Z Y
Stepsl ze Comput at l on: A r l t hm( GRG)
Step 1. Fi nd G
R
S = - G
R
; Sel ect a
Step2. q = 1
d. Z=S; Z = Zi +!:J.Z;
l:J. yq=[ Br
I
[A]d. Z
Step3. y q+l = Yq + l:J. yq
x1+1=1 ZI
I yq
If [ H( X) ] = 0; Stop Converged
seqq +1
l:J. Y9= r
1
[-H(Xi +I )]
GoTo Step3
E veagori mnei Z[A] [BJ nor i s nged. Thc si ngul ari ty of [B]
is a concemand can be con OI l edby thEsel ecti onof i ndependent vari abl es. I tapPE
at az zs responsi bl e for decrEasei n thE functi on whi l e A Y is rEsponsi bl Ef or
feasi bi1i ty. The above al gori thmis usual1y xecutedfor l wo vaues of . A quadrati c
i nterpol ati onbetween thethr poi nts ( = 0 is avai l abJ ef romtheprevi ous i terati on)
is to detenni ne e opt i mum psi ze. One f in al i tera on is perf ormed for i
opt i mumstepsi ze.
A 01 m: Genele!l l zedReduced Gradi ent ( GR
Step1. Choose X
I
(must be feasi bl e)
N. (no. of i terati ons)
E;'S(for convergence and stoppi ng)
p = 1 (i terati oncounter)
Step2. I denti fyZ Y
Cacul ate[A] [B]
CacuJ ate [ G
R
]
Cal cul ateOpt i mumStepsi ze (seeal gori thm)
Cal cul ateXP+l
Step3. Convergence for GRG
If hk = 0 for k = 1 2 . . . 1;
If 8j S; 0 for j = 1 2 . . . m;
If K T condi ti ons aresati sfi ed
Then Converged Sl Op
300 NUMERI CAL TECHNI QUES FORCONSTRAI NEDOPTI MI ZATI ON
Stoppi ngCriteria:
Ll X= X
p
+
1
- X
P
If Ll X
T
Ll X EI: Stop(desi gnnot changi ng)
If p = N.: Stop( maxi mumiterations reached)
Conti nue
p p+l
Go toStep2
Appl i catl on 0 ' the GRG Met hod: The GRG al gori thmis appl i ed 10 Ex npl e
7.1. Once agai n the iIl ustrati on is through a set of cal cul ati ons genernted usi ng
MA1LAB. For a 5tart Exampl e 7.1 is rewri ttenusi nga sl ack vari abl efor th i nequal i ty
constramt.
Mi ni mi ze f (Xh x2X3): x1- 2xrX2+ xt + - 2xl + 4
Su t H1(xl X2X3):
H2(XhX2): +X3=O

For is probl emn = 2 1 = 1 m = 1 n + m = 3" - l = l l + m = 2.
Exampl e 7.1 is not a good probl emtoiIl ustratetheworki ng of theGRG method
because onl y feasi bl epoi nt where g is acti veis al so esol uti on. Thi s makes Z
and refore G
R
a scal . The exampl e is retai ned because of uni formi ty. In

i mpl ementati onbel ow Z = [XI] and Y = [X2 X3)T.
Sec7 _3_3. m: l I s m- fiI e handl es Exampl e 7.1 through e GRG method
symbol i cal l y. It requests a startingdesi gnwhi ch shoul dbe feasible. It sets up e Z
and Y automnti cal l yand computes [A) nnd [8]. Experi menti ng wi ththecode i ndi cates
that not all feasi bl epoi nts provi dea nonsi ngul ar [8]. It thenprompts theuser for two
v uof whi chit uses toperforma quadrati c inte ol ati on. It computes anopt i mum
sl epsi zeand compl etes theiterationfor d. Y. Thismpl el es one iterationof GRG.
Experi menti ngwi th ecode iI1ustrates at thechoi ces of need some thought for
theiteration10succeed. The fol l owi ngis transcri bedf romtheCommand wi ndowfor
two iterations. Note eprompts for al pha:
l teraon 1
sec7_3_3
I nput t he f easi bl e desi gn chosen f or eval uat i on.
[ 1. 2 0. 74833] % Thi s i s x
1
The desi gn vect or [ 1. 2000 0. 7483 0. 2200]
f unct i on and const r ai nt s( f h1 h2) :
3. 6304e+000 - 2. 2111e- 006 0
A
2. 4000e+000
6. 0000e- 001
B
7.3 Dl RECT METHODS FORCONSTRAl NEDOPTI MI ZATl ON 301
1. 4967e+000 0
1. 1225e+000 1. 0000e+000
s
- 6. 0183e+000 ( Reduced Gr adi ent )
I nput t he st epsi ze f or eval uat i on.
( Pr ompt f or f i r st st epsi ze)
0. 1
al pha
1. 0000e- 001
No. of i t er at i ons and const r ai nt er r or : 29
1. 0000e- 001 8. 2187e- 009
desi gn vect or : 5. 9817e- 001 1. 2815e+000 - 3. 2110e- 001
f unct i on and const r ai nt s (f h1 h2)
3. 3548e+000 7. 2526e- 005 5. 4394e- 005
s ' "
- 6. 0183e+000
0. 05
al pha
5. 0000e- 002
( Pr ompt f or second st epsi ze)
No. of i t er at i ons and const r ai nt er r or : 12
5. 0000e- 002 6. 2635e- 009
desi gn vect or : 8. 990ge 001 1. 0916e+000 - 9. 5823e- 002
2. 9702e+000 6. 3314e- 005 4. 7485e- 005
al pha
S56595e 002 ( quadr at i cal l y i nt er pol at ed}
=
- 6. 0183e+000
No. of i t er at i ons and Const r ai nt er r or : 14
5. 6595e- 002 4. 3237e- 009
desi gn vect or : 8. 5940e 001 1. 1231e+000 - 1. 3072e- 001
2. 9903e+000 5. 2604e- 005 - 3. 9453e- 005
302 NUMERI CAL TECHNI QUES FOR CONSTRAI NED OPTI MI ZATI ON
Thi s mpl e onei terati on. Thi s is far f romsati sfactoryas X3 thesl ack vari abl e is
negati ve. The stepsi ze and desi gnchange cal cul ati onmust be mad sensi ti veto e
si deconstr ints. Inactual i mpl ementati onof theGRG si nceg(X) Was i nacti veat e
start it woul d nol have been inc1uded inthecal cul ati ons at all. Usi ng thesol uti onto
s t thenext addi ti onal i terati ons canbe obtai nedl eadi ng10thesol uti on.
l terati on2
Rerun Sec7 _3_3. mwi thX
2
= [8. 5940e-001 1. 1231e )00]
The rest of e i terati ons are left as an exerci se for reader. To compl ete
Sec7 _3_3. mthecal cul ati ons for K T condi ti ons must be i ncl uded. Intheabove si nce
g(X) is invi ol ati on theK T condi ti ons were not computed. This is currentl yone of the
more robust methods. For this pa cul ar exampl e theprogress of sol uti on is qui l e
sensi ti veto estepsi zechosen for i nterpol ati on.
7. 3. 4 Sequent l al Gr adl ent Rest orat l on Al gor l t hm( SGRA)
The SGRA is a two-phase approach to esol uti onof NL P probl ems. Starti ngat a
feasi bl edesi gn theGradi ent phase decreases theval ue of theobj ecti vefuncti onwhi l e
satisIIng l i neari zed acti ve constrai nts. This will cause cons ai nt vi ol ati on for
nonl i near acti ve cons ints. The Restorati on phase bri ngs back the desi gn 10
feasibilitywhi ch may establ i sha new ddi fferenl set of acti veconstrai nts. Thiscycl e
of two phases is peal eduntil theopt i mumis f ound [15 16]. The me odi ncOJ porates a
d cent property wi th chcycl e. Li ke other al gori thms in is sti on it uses an 8Ctive
nstrint strategy. In compari s the GRG combi nes the two phes togeth .
method has been favorabl ycomp81 toGRG and theGradi ent Proj ecti onmethod [17].
The SGRA as ori gi nal l yin ducedi n elistedreferences assumes thei nequal i ty
cons ai nts of thef orm
gj (X) 0; j = 12m
To be consi stent wi edi scussi oninthebook it is changed tothestandardf ormof
NL P probl ems defi nedin is text. Inthefol l owi ngdevel opment of theal gori thm
onl y thesalient features arei ndi cated. They areal so transcri bedconsi stent wi th e
other al gori thms in this secti on. It is commended the reader revi ewthe
references for a detaied descri pti on. The general probl emis
Mi ni mi ze
Subj ect to:
[X]n f (X)
[h(X)]1=0
[g(X)] " 0
X10wsXSXUP
(7.5)
(7.6)
(7.7)
(7.8)
E
F
5
7. 3 Dl RECT METHODS FOR CONSTRAI NED OPTI MI ZATI ON 303
In SGRAonl y acti vein ual i tyconstrai nts areof i nterest. Equal i tyconstrai nts are
al ways acti ve. Acti ve i nequal i ty constrai nl s al so inc1ude vi ol ated constrai nts. If 11
i ndi cates theset of acti veconstrai nts uati ons (7.6) d(7.7) arecombi ned i nto
a vecl or of acti vecons ints ():
( X) =[ J
(7.58)
The number of acti vei nequal i tyconstrai nts is v - l. The Lagrangi anfor theprobl em
canbe expressedinl erms of acti vecons i al one[si nce mul ti pl i ers for e
gJ {X) < 0 will be set to zero as part of K T condi ti ons]. le K T condi ti ons a then
expressedas
VxF(X ) = V x. f (X) + [ V (7.59a)
( X) =O (7. 59b)
O (7.59c)
where [ V X+1 = [ V X+I V
xz Vx'!>v]; =l 1 .JT
GrBdi ent PhBse: Gi ven a feasi bl edesi gnX' fi ndthenei ghbori ngpoi nt
s=Xi +AX
such that 5j < 0 =O. I mposi ng a quadrati c constrai nt on thedi spl acement dX
[16] eprobl emcan be set up as an opti mi zati onsubprobl emwhose K T condi ti ons
determi ne
tl X= - aV Fx(X
I
A. =S (7.60)
The i nterpretati on of the search di recti on S is used to connect to e other
al gori thms in is book. It is not intheori gi nal devel opment. The searchdi recti onis
opposi teto egradi ent of theLagrangi an whi ch is a novel feat . To comput e e
Lagrangi an l heLagrange mul ti pl i ers have tobe cal cul ated. Thi s is done by sol vi nga
systemof Ii near equati ons
[ V V xj (X
I
) + [ V = 0 (7.61)
St eps; ze f or Gr adl ent PhBse: The stepsi zecal cul ati onis basedon dri vi ng E
opti mal i tycondi ti ons inEquati on(7.59a) tozero. Therefo if
304 NUMERI CAL TECHNI QUES F OR CONSTRAI NED OPTI MI ZATI ON
=X
i
- ( J . V (Xi i.v)
thcnopti mum is foundby cubi c i nterpol ati onwhi l etryi ngsati sfy
VxF(')..l' l V
x
F i..I' ) =o (7.62)
Care must betaken this stepsizedoes not causesignificant constrai nt violation.
Thi s is nfol"cedby cappi ngthesquarederror intheconstrai nts by a suitableupper
bound wJ uch is set up
( g) T( ) S P
m
(7.63)
Reference 16suggests t Pm is relatedtoano er performance i ndex Q whi ch is
eerror intheopti mal i tycondi ti ons:

(7.64)
Restorati on Phase: It isexpec dthatat ndof thegradi ent phase thefuncti on
will have decreased but woul d be some constrai nt dissatisfaction (assurni ng
there was at least one nonl i near acti ve constrai nt at the begi nni ng of the gradi ent
phase). The restorationphaseestablishes a nei ghbori ngfeasiblesolution. 1t does this
by ensuri ng that the I i neari zed constrai nts are feasible. Pri or to is. e active
constrai nt set has tobeupdated (ii) si nceprevi ousl y feasi bl econstraints coul dhave
beC9me infeasible(andprevi ousl yinfeasibleconstrai nts coul dhave become feasible)
wi th desi gnchange causedby egradi ent phase. The desi gnvecr andthedesi gn
changes for is phasecanbe wri ttenas
X=Xg+dX (7.65)
The desi gnchange for this phasedX is obtai nedas a I st squaree.' or inLhedesi gn
changes subj ect to e sa faction of the lin r cons aints. Setti ng up NLP
subprobl emthis cal cul ati onfor edesi gnchanges is
dX
r
= - V xCT( X
g
) (7.66)
Here "i s thevector Lagrange mul ti pl i er of thequadrati c subprobl em. The val ues
for themu1tipliers arees bl i shedthroughthelinear equati ons
( g) - VAds) TVJ dE) =0
(7.67)
Thefactor i s a user-control l edp ' ameter todi scouragel argedesi gnchanges.
7. :3 DI RECT METHODS F OR CONSTRAI NED OPTl MI ZATI ON 305

The Restorati onphase is iterativelyappl i eduntil
( lCT() S 8
1
(7.68)
where EI is a smal l number.
At theconcl usi onof therestorati onphase econstrai nts arefeasibleandthenext
cycl eof Gradi ent-Restorati onph ec beappl i edagain. A summary of theabove
steps is capturedin eal gori thmgi venbel ow.
A ori thm: Sequenti al Gradi ent Restorati on A ' ri thm( SGRA) (A7.
Step]. Choose X
J
(must befeasible)
N. (no. of iterations).
8j ' S (for convergence and stoppi ng)
p = 1 (iterationcounter)
Step2. Execute Gradi ent Phase
Cal cul ateStepsi zeusi ngcubi c inte olation
Cal cul atexr
l
Step3. Execute Restorati onPhase
xP+J
Step4. Convergence for SGRA
If h
k
= O. for k = 1. 2. . . . 1;
If gj S 0 for j = 1. 2. . . . /11;
If KT condi ti ons aresatisfied
Then Converged Stop
Stoppi ngCriteria:
a X = XJ *I - X!'
I f aX
T
xS EI : Stop(d si gnnot changi ng)
If p = N.: Stop( maxi mumiterations reached)
Conti nue
p p+l
Go toStep2
Appl l cati on of t he SGRA 7. : Exampl e 7.1 is usedtotest theappl i cati onof
theSGRA. The mfileSec7 _ 3_ 4. mcontai ns thecode that appl i es theSGRA through
a combi nati onof symbol i c andnumeri c calculations. As in eprevi ous ex npl es it
iscustomi zedfor two variables. one equa1i tyand one i nequal i tyconstrai nt. Unl i kethe
GRG i mpl ementati onit i ncorporates eacti veconstrai nt strategy. It executes one
cycl eof the SGRA whi ch has a single iterationin Gradi ent phaseand several
iterations in eRestorati onphase. Inthegradi ent phasethecubi c i nterpol ati onof the
original techni queis pl acedby thequadrati c i nterpol ati onusedeverywhere inthe
book. The startingi nput is a feasibledesi gnvector. In egradi ent phasethemethod
reques a val ue for the stepsi ze to scan for the opt i mum usi ng quadrati c
int olation. Thi s is i mportant for successi vecycl es start at theopt i mumval ueof the
stepsi zefor theprevi ous cycl . The val ueof i n Equati on(7.67) is set at 0.5. Pri nti ng
is controlI ed roughthesemi col onand ereader is encouragedf ol l owdetails by
f l ovmg11.
In the folI owi ng theprogramis run wi e starting desi gn usedfor the GRG
method. 1 output f romthe Command wi ndowis pasted bel owwi th some bol d
annol ati ons 10drawauenti on.
>> Sec7 3 4
npu he f easi bl e desi gn chosen f or eval uat i on.
[ 1. 2 0. 74833] ( f easi bl e desi gn vect or i nput )
The desi gn vect or [ 1. 2000 0. 7483]
f unct i on and const r ai nt s( f h g) :
3. 6304e+000 - 2. 2111e- 006 - 2. 2000e- 001
***********************
Gr adi ent Phase
* * * * * * * * * * * * * * * * * * * * *
The Lagr ange mul t i pl i er s
- 1. 0812e+000
1 ( al pha used t o st ar t scanni ng pr ocess)
al pha f or quadr at i c i nt er pol at i on
o 1. 2500e- 001 2. 5000e 001
( al pha t hat br acket s t he mi ni mum)
f unct i on val ues f or quadr at i c i nt er pol at i on
1. 0142e+001 6. 6291e+000 1. 1300e+001
( cor r espondi ng f unct i on val ue)
opt i mumal pha: 1. 1615e- 001
3. 0077e+000 1. 3681e- 001 9. 8328e- 002
( bot h const r ai nt s vi ol at ed)
The per f or mance i ndi ces Q P
6. 3181e+000 1. 8718e- 002
(P measur es t he squar ed er r or i n t he const r ai nt s)
* * * * * * * * * * * * * * * * * * * * * *
Rest or at i on Phase
* * * * * * * * * * * * * * * * * * * * *
Number of Rest or at i on i t er at i ons: 21
/
!
l

3. 0000e+000 6. 712ge- 005 4. 6741e- 005
( Const r ai nt s sat i sf i ed - 9 act i ve)
The per f or mance i ndex P
6. 6911e- 009
The sol uti onhas converged oneiteration. The SGRA is of thesame cal i ber as th
GRG (some compari sons showit to be better). It is difficult to compare di fferent
al gori thms because of di fferenl codi ng strucl ures and nonstandard i mpl ementati on
details. Neverthel ess theSGRA has consi derabl emeri t. It is defi ni tel ybetter thanthe
GRG as far as Exampl e 7.1is concemed (Secti on7.3.3). Thi s compari son may not be
fair s ~ n e theacti veconstrai nt strategywas not usedfor theGRG even though it is part
of theGRG. Thi s demonstrates theneed for sucha strategyinall al gori thms atdeal
wi cons i nedopti mi zati on.
7. 4 ADDI TI ONAL EXAMPLES
Threeaddi ti onal exampl es arei ntroducedhere10ilI ustrateand expand the eof se
techni ques. Duri ng theearl ydevel opment of thesetechni qu rewas a si gni fi cant
emphasi s on effi ci ency robustness and keepi ngcomputati onal resources as l owas
possi bl e. Most of e techni ques were devel oped on mai nf rames usi ng FORTRAN.
Today Ihesearchfor gl obal sol ul i ons ( at arel argel yheuristic) coupl edwi thsi mpl e
cal cul ati ons rep led al most endl essl y performed on dkt op PCs wi i ncredi bl e
computi ng power I hroughsoftwru systems al provi dea wi de range of resource
has shi ftedthefocus toexperi mentati onand creativity. The author feels suchan effort
must be coupl ed wi th knowl edge and i nsi ght gai ned through a wi de sel ecti on of
appl i cati ons d understandi ng eprogress of numeri cal cal cul ati on. Inspiteof all
e expos e so far in the for many real desi gnprobl ems the sol uti on is not
automati c. D ling wi th nonl i near equati ons is a chal l enge and often surpri si ng.
Tryi ng 10 understand why the l echni que does not work is as much a l earni ng
expel" ncearouti nesol uti ontotheprobl em. Si ncel heseexampl es use themel hods
of ep vi ous secti on wi thdiffer nt functi ons all of thecode devel oped before is
revi sedwi thand theappropri atechanges and renamed. semodi fi cati ons qui re
that f ewlines be changed f romthe ori gi nal code. lis al l ows the user to make
i ncremental changes as needed. Thi s is not an efficient approach bUl capi tal i zes on the
unl i mi ted hard dri ve space avai l abl e today. Students are urged to understand the
al gori thm thefl owof thecode and eresult bei ngsought .
7.4.1 Exampl e 7. 2 agpol ePr obl em
Thi s is thesame probl emwhi ch was sol vedgraphi calI y inChapter 2 anal yti cal l y
in Chapter 4 and now consi dered for numeri cal sol uti on. In is subsecl i on the
scal ed versi on of I he probl emis sol ved. Fr omChapter 4 Exampl e 4.3 it is
rei ntroducedas
308 NUMERI CAL TECHNI QUESFORCONSTRAI NEDOPTl MI ZATI ON
Exampl e 2
Mi ni mi ze (7.69)
gl: 2.55+O.l 36l x
1
- 33.148( x1- 0 (7.70
g2: (0. 88+
- 0 (7.70b)
1 . 0868x
1
+ 0.3482 - (7.7Oc)
g4: x
2
- x
1
+0. 001
0. 02 x 1 1. 0; 0.02 x
2
1. 0
(7.70d)
(7.71)
I ns ht about Numer i cal Sol uti on: In Chapter 4 it was i nvesti gated e
soI uti ondoes not satisfytheKT condi ti ons because th sol uti onis not a regul ar poi nt.
It was ascertai ned t egradi ents of etwo acti veconstrai nts t determi ne e
sol uti on(810g3) arenearl yparallel inthenei ghborhoodofthe solution. How will this
affect thenumeri cal searchfor thesoI uti on?
Typi caJ ly what a numeri cal techni quedoes is pi ck a useful di recti onand seehow
best it canmove al ongit. Suppose thetechni quedeci des tomove al ongthegradi ent
of 83todevel opit intoanacti veconstrai nt . Thcre is obvi ousl y goi ngtobea probl em
inpushi nggl tobecome active. Pushi nga constrai nt to be acti ve is a basi c way to
reduce the obj ecti ve functi on in wel l -formul ated NLP probl em. Therefore most
al gori thms (unl ess ey are trai ned to compromi se in some si gni fi cant way) wilI
underachi eve by onl y pushi ngone ofthe constrai nts tobe acti veand ensuri ngtheother
is feasible.
ALM_ 7 _ 4_1. m: This m- fiI e uses theAL M method 10sol ve eprobl em. lnorder
topreservetheol dALM. mto flect thedi scussi oninSecti on7.2.2a copy of thefile
is made. Of un341. m Hf un341. m Gf un341. m and FALM341. mreflect e
functi ons f romExampl e 7.2. A1so ALM_7_4_1. mis adj usted avoi d handl i ng
equal i tycons aints dprovi desome pri nti ngsupport. 0 erwi seit is essenti al l ythe
same. Once agai n the output f romthe Command wi ndowis p ted bel owand
annotatedtodrawattenti ontosome issues. Onl y initial prompts wi thei nputs
theinitial iteration and theconsol i datedi nformati onafter thefinal i temti on copi ed
bel ow.
ALM_7_4_1 ( i nvoke pr ogr am)
I nput t he st ar t i ng desi gn vect or
Thi s i s mandat or y as t her e i s no def aul t vect or set up

7.4 AOOl nONAL EXAMPLES 309
The l engt h of your vect or i ndi cat es t he number of
unknowns (n)
Pl ease ent er i t now and hi t r et ur n f or exampl e [ 1
2 3 . . . ]
[ 0. 75 0. 55] ( i ni t i al desi gn vect or )
The i ni t i al desi gn vect or :
7. 5000e- 001 5. 5000e- 001
I nput t he mi ni mumval ues f or t he desi gn vect or .
These ar e i nput as a vect or . The val ues ar e
3 * st ar t val ue bel ow t he st ar t val ue unl ess i t i s
zer o.
I n t hat case i t i s - 5:
[ 0. 02 0. 02] ( i nput l ower bound not cur r ent l y used)
The mi ni mumval ues f or t he desi gn var i abl es ar e
2. 0000e- 002 2. 0000e- 002
I nput t he maxi mumval ues f or t he desi gn vect or .
These ar e i nput as a vect or . The def au1t va1ues ar e
3 * st ar t val ue above t he st ar t val ue.
f st ar t val ue i s 0 t hen i t i s +5:
[1 1] ( i nput upper bound - cur r ent l y not used)
The maxi mumval ues f or t he desi gn var i abl es ar e 1 1
Number of equal i t y const r ai nt s [0]
( def aul t val ue of 0 i s ok)
Number of i nequal i t y const r ai nt s [0] 4
I ni t i al val ues f or bet a [1 1 1 1]
( i nput number of i nequal i t y const r ai nt s and i ni t i al
bet a)
Desi gn Vect or ( X) 7. 5000e- 001 5. 5000e 001
Obj ect i ve f unct i on 2. 6000e- 001
( I n t he f ol l owi ng vi ol at ed const r ai nt s ar e shown i n
bol d)
I nequal i t y const r ai nt s - 5. 9180e+000 - 3. 5997e+001
5. 2383e+000 - 1. 9900e- 001
Squar e Er r or i n const r ai nt s( h g) 0 0
Lagr ange Mul t i pl i er s ( l ambda bet a) : 0 1 1 1 1
Penal t y Mul t i pl i er s ( r h r g) : 1 1
Desi gn Vect or ( X) 6. 4726e 001 6. 1013e- OOl
Obj ect i ve f unct i on: 4. 6695e- 002
Er r or i n const r ai nt s (h g) 0 2. 1366e 009
Lagr ange Mul t i pl i er s ( l ambda bet a) : 0 0 4. 8227e 002
o
Penal t y Mul t i pl i er s ( rh r g) : 1000 1000
The val ues f or x and f and 9 ar e
7. 5000e- 001 5. 5000e- 001 2. 6000e- 001 0
6. 5647e- 00l 6. 0173e- 001 6. 8871e- 002 0
6. 4726e- 001 6. 1017e- 001 4. 6638e- 002 1. 5525e- 006
6. 4726e- 001 6. 1013e 001 4. 6695e- 002 2. 1366e- 009
6. 4726e- 001 6. 1013e- 001 4. 6695e- 002 2. 1366e 009
( Not e desi gn and f unct i on ar e not changi ng hence
st opped)
The val ues f or bet a ar e
1. 0000e+000 0 0 0
1. 0000e+000 0 0 0
1. 0000e+000 1. 4062e- 002 3. B9B3e- 002 4. 8227e- 002
1. 0000e+000 8. 9252e 001 1. 7073e- 001 0
( Each col umn r epr esent s val ues of bet a f or an
i t er at i on - bet a f or const r ai nt s 12 and 4 ar e zer o and
i t can be seen bel ow t hey ar e f easi b1e but i nact i ve.
Onl y t he t hi r d const r ai nt i s act i ve)
I nequal i t y const r ai nt s 6. 7858e- 002 - 4. 9734e+OOO
4. 6223e- 005 - 3. 6136e- 002
( Not e t hi r d const r ai nt i s act i ve. The f i r st one i s
al most act i vel
The anal yti cal sol uti onfor is probl emis
X;=O 774; =0. 6445; I = 4.3577E - 02;
g; -O. O; g; =- 4.9837; g; =O. O; g; =- O. 319 - 02
Incompari son theAL M has done a decen1j obof geui ng cJ ose 10thesol uti onbefo
staIli ng. Some of is canbe traced10 di scussi onpri or 10runni ng eprogram.
7. 4. 2 Exampl e 7. BeamDesl gn
Thi s sec1i on r uires e use of e Opti mi zati on Tool box f romMATLAB. Mor e
speci fi caJl y it uses 1heQP functi onsol ve 1heQP subprobl emintheSQP me d.
The probl emis sol vedusi ngthedirect proced eilI ustral edthroughtheexampl e in
7.4 ADDI TI ONAl EXAMPLES 311

Secti on7.3.2. secti oncombi nes thesymbol i c capabi l i tyof MA theresou
avai l abl eintheOpti mi zati onTool box toi mpl ement anorigina1interpretationof theSQP.
exampl e is edesi gnof anI -beamfor useina parc ars ctura1probl em. is
a ditiona1 mhani caVcivil engi neeri ng strucra1 opti mi zati on probl em. It was
i ntroducedinChapter 1 for ilI us tion. It a1soappears inChapters 8 dI Ofor iIl us tion
of addi ti ona1issues. I n eexampl e here eI -b mwiI1 rrya wei ght at R r. le
n cl oad g d edesi vari abl esfor eprobl emaredefu:ted F i gure8.7(Chap
8). lefigureisnot reproducedhe but X I is thedepthof thebeam X2 is thefl angewi dth
X3 i sthew i ckn . dX4i s ef1ange tJuckns. Other parameters (g W. Ep y)
arenotr efi nedhe .
lemathemati ca1model is
Mi ni rni ze J (X) = pLAc
Subj ec1to: gl (X): WL
3
- 12EI 0.25 (de tionconstrai nt)
82(X): WLx
I
- 8/1y 0 (normaI stress constrai nt)
g3(X): WOc- 21x
3
: y:S; 0 (sh ss co t)
Some geometri c cons1rai nts torelatethevari ous desi gnvari abl es are
g4(X): XI - 3X2 0
8' (X): 2x2 - XI 0
g6(X): X3 - 1.5x4 0
g7(X): 0.5x4- X3 0
The si deconstrai nts on edesi gnvari abl es are
3 :S; x
1
20 2 15 0. 125:S; x
3
0.75 0.25:S; x
4
:S; 1. 25
The fol l owi ngrel ati ons defi ne ecross-seclI onal area(A
c
) moment of i ner( d
moment of areaabou1thecentroi d(Qc) interms of thedesi gnvari abl es:
Ac=2 X
4
+x
1
x
3
- 2x
3
x
4
I

xf ( - - 2x3)3

12 12
Qc=0. 5x
2
I - xJ + 0. 5x3(x1 - xJ 2
The desi gnparameters for theprobl emareW = 1.4x 2000 (1.4is factor of safety)
L = 96 E = 29 E+06 p = 0. 284y=36 E+03:=21 E+03.
EX343. m: Thi s is e mfile al wi l l i mpl ement the Sol ul i on l hrough the SQP.
Whi l e this is a tradi ti onaI desi gn probl em. it is aI so nol trivial . It is rare to fi nd a
publ i shed detai l ed procedu for sol uti on. The sol uti on is usuaI ly obtai ned
numeri cal l y. Inany event. anaI yticaI appl i cati onof the K T condi ti ons is usuaI ly not .
contempl ated because of 2
7
c ( f or e7 constrai nts) even if many of thosecases
are trivial . IncI usi on of the si de constrai nts wi l l provi de an addi ti onaI 8 l i near
cons (agrandto of 15constrai nts) of thef orm(iII us tedfor X) onl y)
gs(x): - x
1
+ 3 S 0
gI 2(X): x
l
- 20S0
The reader is strongl yr ecommended 10revi ew codeinthem- fiI e for ef ol l owi ng
reasons:
The power of symbol i c mani pul ati on is pl oi ted further usi ng vector
defi ni ti ons 10make the i mpl ementati on fairlyeasy. The symbol i c cal cul ati ons
areused togenemte edal afor theQP subprobl em.
Th symbol i c cal cul ati ons are effortl essl y i nl egral ed wi th numeri cal
computati ons (th sol uLi ontotheQP is numeri cal ).
The i mpl ementati on of SQPis s i ghtforwardand di rect (andsi mpl e).
'he sol uti on is si i fi cantl y better an the numeri cal sol uti on by MATL
own SQP program( Chap 10).
The programrequi res th starti ngguess for thedesi gnvector. The functi ons for the
exampl e ar hardcoded. The final sol uti onis pri ntedtotheCommand wi ndow:
Initial Guess:
Fi nal Desi gn:
Obj ecti ve:
n
u
n
u

'
a
n
u

n
u
J

n
U
4
z
J
'
i

n
u
n
U

4
n
u
r
o
o
n
u
z
u
t
J

n
u
U
0
6

m
5
AlI constrai nts ar satisfied:

Aci ve constrai nts: 1 (defl ecti on). 7. 9 (X2 l owerbound). 10(. l owerbo d).
11 (X4 l ower bound)
The reader is encouraged toexpl orewhy thesol uti onobtai nedhe is si gni fi cant1y
better theone usi ngtheSQP f romthetool box. grantedthat thei mpl ementati ons
arenot thesame. Some thought shoul dbe gi vento penormanceof quasi - Newton
(meri c) methods and their adaptati on0constrai ned l echni qu sif theHessi an is no
posi ti vedefi ni te whi ch inthis case it is.
7. 4. 3 Exampl e 7. 4- 0ptl mal Cont rol
This secti on i ntroduces another cI ass of probl ems C be addressed by desi gn
opti rni zati on techni ques. It is the basi c traj ecory determi nati on probl eminopti maI
control . Gi ven t wo poi nts ina pl ane fi nda pa at wi1I be tr versedbetween t hem
usi nga steeri ng v tor of constant magni tude. The pathhas to be aversedinuni t
ti me. The probl mc bedescri bedusi ngFi gure 7.11. If theprobl em nbe descri bed
in ephysi cal pl ane. thesl ri ngvector consi deredas vel oci ty. en theki nemati c
uati ons for a parti cI e movi ng on e pa can be descri bed by the fol l owi ng
di fferenti al equati ons (d i opti mi zati onprefers todeal wi tha1gebrai c functi ons).
whe X. Yl ocate eprobl eminphysi cal space:
=V cos 8(t); 0 S t S 1
=V sin8(1); 0 S t S 1
The initi aI and final condi ti ons are(t11esewere theval ues chosen for theexampl e)
(0) = 0; y(O) = 0
x(l ) = 3; y(l) = 2
Here. 8(t) is regarddas a control vari abl e. II is a conti nuous functi onof t between 0 S
t S 1 that will transfer thepac1ef rom initial poi nt 10thefinal poi nt . Incontrast.
desi gnopti mi zati on(i. e.. his book) determi nes scal ar val ues of thedesi gnvari abl es.
These types of probl ems bel ong to the area of opti mal control . Th are speci al
numeri cal techni ques al are avai l abl e 10 sol ve I hese categori es of probl ems. One
approach totheseprobl ems is tousedesi gnopti rni zati ontechni ques. This is ai dedby
usi nga parame ic representati onfor thecontrol functi on. Many di fferent par ne c
p sentati ons for the control have been used. such as cubi c spl i nes and Hermi te
pol ynorni al s. I n this sudy a Bezi er curvc is used to rep sent the control. The
di fferenti al equati ons ar ennumeri cal l y i nl egrated10es bl i shth j ecl ory.
y
x
FI gure7 Descri ptl onof Irajeclorynlrol. 1.
whi ch requi res br achi . mto retumthe state space defi ni ti on of probl em.
br achi . mal soi ncl udes theBezi er i nterpretati oncode.
Fi gure 7. 12descri bes eBezi er ve i ces of thecontrol functi onat thestanand
final iterations. The final conti nuous control is al sopl ottedon the figure. The start
curve is a s ai ght linethroughtheverti ces and is not shown. Fi gure 7. 13shows e
start traj ectory d efinal j ectory. It is cl rf rom efi gure einitial and e
final condi ti ons are sati sfi ed for the final trj ectory. Si nce e equati ons we
i ntegrated. thedi fferenti al constraI nts arethereforecompl etel y satisfied.
The final tr ectory shown is onl y feasi bl e it is one of many canp
betw nthet wo poi nts. Thi s opens thepossi bi l i ty at a further op ti onc be
perf ormed wi th spect to j ectory. Forexampl e. fi nd eshortest pathin eti me
provi ded. Not it is not e strai ght pa si nce for a gi ven vel oci ty magni tude the
desti nati onmay be achedsooner anp scri bed. The cassical probl eminopti mal
ntrol is e brachi sl ochrone probl em. lt is e mi ni mumti me for thep le 10
tr vel between t wo prescri bedpoi nts dynami cal l y (obeyi ngpri nci pl es of dynami cs).
The reader is cncouraged to expand techni que and e code to expl ore se
addi ti onai nteresti ngprobl ems.
315 7. 4 ADDI TI ONAl EXAMPLES NUMERI CAL TECHNI QUES F OR CONSTRAI NED OPTI MI ZATI ON
Sol utl on Techni que (adoptedin is secti on)
Use a Bezi er curvetoparametri zethecontrol functi on.
I ntegrate the differential equati ons forward usi ng a Runge- Kutta met hod
(ode45. m)
Use uncons i nedl echni que(f mi nunc. m) mi ni mi zer to i ve esquareof
eerrori n efinal condi tozero0 st squar error).
314
Feasl bl etroJectory
Thi s exampl e nds the M ATI . AO tool box to execute f mi nunc. m. Al tematel y. e
unconstrai nedmi ni mi zer can be among thoseprevi ousl y devel oped. is transl ates
inloan uncons i nedmi ni mumprobl em whi ch thereforeshoul dhave been handl ed
in Chapter 6. It is incuded here because e di fferenti al equati ons are actual l y
consi deredd renti al consl rai nts. The obj ecti vefuncti onthereforeis
f {X}=[ 'finnl-x(l )f + [Yfinal- y( l) J2
Ex744. mwill executetheexampl e and produce pl ots of thecontroI and thetrj tory.
It uses t wo standardMATLAO functi ons menti oned before(odc45. mand fmi nunc. m).
The Bezi er curve generati onr ui res coef f . m. Combl nat l on. m. Factorl al . m. le
obj ecti vefuncti onis constructedinobLopt cont . m. It qui stheuseof ode45. m.
M ze
The vertl ces and 0 (1)
z

llnal polnl

3
3.5 2
1.5
2.5

'

-
H

- - -
-
2
>- 1.5
0.5
3.5 3 2.5
Fl gu 7.13 Trajecloriestart nal .
2
x
1.5 0.5
- 0.5
0
o
0.5

a
'
e

d
y

o

a'
-...
0.9 O. 0.7 0. 6
Control funclion0(/)
0.5
tl me
0.4
Fl gu 7.12
0.3 0.2 0.1
REFERENCES
1. Fi acco A. V. and McConni ck G. P. Nonl i near Progral l l mi J l gSequenti al
Unconstrai l l edMi ni mi zoti ol l Techni ques Wi l ey NewYork 1968.
2. V rpl G. N. Numeri cal Opti mi zoti on Techl l i ques f or Engi neeri ng Desi gl l
McGr aw- HiII NewYork 1984.
3. ANSYS Software Ansys Inc. Pi ttsburgh PA.
4. Bertekas D. P. Constrai nedOpti mi zoti onand Lagrange Met hods Academi c Press New
York1982.
5. KeJl ey J. E. The Cutti ngPl an Me odJ oumal of SI AM Vol. 8 pp. 702- 7121960.
6. Gill P. E. Murray. W. andWri ght M. H. Practi cal Opti mi zoti on Academi c Press New
York1981.
7. Boo1 J. C. G. Quadrauc Programmi ng Sl udi es itl Mathemati cal and MQl l ageri al
Economi cs H. Theil ( d Vol. 2 North-Hol l and A msterd n1964.
8. Bi ggs M. C. Constr nedMi ni mi zati onUsi ngRecursi veQuadrauc Programmi ng: Some
Al temateSubprobl emFonnul auons Towards Gl obal Opt IIzoti onL. C. W. Di xonand
G. P. Szego (eds.) pp. 341- 349 North-Hol l and Amsterdwn 1975.
9. Powel l M. J. D. A Fast Al gori thmfor Nonl i near Consl rai nedOpti mi zauonCaI culalions
No. DAMPr P77I NA 2 Uni versi tyof Cambri dge Engl and 1977.
10. Han S. P. A G10bal l y Convergent Method for Nonl i near Programmi ng J oumal
Opti mi zoti onTl reoryand Appl i cati ons Vol. 22 p. 297 1977.
11. Arora. J. S.. I ntrnducti ol l l oOpl i mwn De. ri sn McGr nw- HilI NewYork 1989.
12. Branch M. A. andGrace A. Opti mi zoti otl Too 'oxUser' s Gui de MathWorks lnc. 1996.
13. Wol f e P. M ods of Nonl i near Programmi ng Recenl AdvQl l ces i/ l Mathemal i cal
Pr ogr ammi ng R. L. Graves andP. Wol f e (eds.) McGr nw- HiIl NewYork 1963.
14. Gubri el e G. A. DndRagsdel J K. M. leGcnerol i zedGradl cnt Method: A ReliableTool for
OpaI Design. ASME J oumal of Engi neeri ngand I ndJ " SeT BVol . 99 May 19.
15. Mi el eA Huang. H. Y. dHei demanJ.C. S uenti aI Gradi ent RestorauonAI gori thm
for the Mi ni mi zauon of Constrai ned Functi onsOr dinary and Conj ugate Gr adient
Versi ons J oumal of Opti mi zoti onTheo and Appl i cati ons Vol. 4 No. 41969.
16. Levy A. V. and Gomez S. Sequenti aI Gradi ent-Restorati on AI gori thmfor the
Opti mi zauonof a Nonl i nr Constrai nedFuncti on. J oumal 'heAstronauti cal Sci etl ces
Speci al l ssueon Numi c Me ds inOpti mi zauon Dedi catedtoAngel o Mi el e K. H.
Wel l (sp. ed.) Vol . XXX No. 2. 1982.
17. Rosen J. B. The Gradi ent Proj ecti on Method for Nonl i near Programmi ng Part D:
Nonl i near Constrai nts SI AMJ oumal of Appl i ed Mathemati cs Vol. 9 No. 4 1961.
PROBL EMS
7.1 Modi f y Sec7_2_1. J )l oLmto revi ewcontour val ues and pl ot automati cal l y
sel cctedcontours.
7.2 Modi f y Sec7 _2_1_cal c. mto automati cal l y cal cul atethe initial mul ti pl i ers
l oopover pena1ty mul ti pl i ers. and toautomati cal l y pi ck thedesi gnvector
satisfies thesi deconstrai nts.
PROBLEMS 317
~ (uses Opti mi zati on Tool box) Devel op eSLP by usi ngtheL P sol ver o mthe
Opti mi zati onTool box and appl y toExampl e 7. 1.
7. 4 Sol ve Exampl 7.1 by sol vi ng eQP probl emgraphi ca1ly.
7.5 Dcvel op code to fi nd a feasi bl e sol uti on f roma gi ven sl arti ng vector for
nslti nedprobl ems.
7. 6 Devel op a programtocal cul atethemul ti pl i ers for theNL P probl emat a gi ven
desi gn.
7.7 Devel op a programfor a constrai nedstepsi zecal cul ati onprocedure.
7. 8 (uses Opti mi zati onTool box) Use theQP programf romthetool box and devel op
your own SQP i mpl ementati on.
7.9 (uses Opti mi zati on Tool box) Use e SQP programf romMATLAB to sol ve
Exampl e 7.1.
7. 10 Devel op Sec7 _3_2. m to automati ca11y execute severa1 i terati ons to
convergence.
7. 11 I mpl ement a newversi onof SQP where theHessi an matri x is mai ntai ned as the
i denti tymatri x for a11iterations.
7. 12 Fi ni shsol vi ngExampl e 7.1 usi ngtheGRG me od.
7. 13 Sol ve Exampl e 7.1wi thZ = [X2] and Y = [x x)]T.
7. 14 Modi f y the GRG code to i ncl ude consi derati on of acti ve constrai nts and
comp eperf orm ce.
7. 15 Bui l dinK T condi ti oncal cul ati oni ntotheSGRA.
7. 16 Sol ve theBrachi stochrone opti mal control probl em.
8
DI SCRETE OPTI MI ZATI ON
Thi s chapter i mroduces some concepts methods and al gori thms associ ated wi
di screte opti zation. It is not possi bl e to compress su t of Di screte
opti mi zauon to a si ngl e chapter as the topi c itself is capabl e of spawni ng several
courses. Di screte opti mi zati on is very di fferent difficult di verse and conti nues to
devel op even today. These probl ems are l argel y combi natori al and are
compu ona1ly more ti me i ntensi ve correspondi ngconti nuous probl ems.
The probl ems addressedby thedi screteopti mi z onresearchcommuni ty aremai nl y
in e aof op eratios se chch a criz usual l yby linea rmo
opti mi zati o nmostl yincorporat.esno nl i ne arre la tio n IS. Fr oma real perspecti ve di screte
desi gn vaI tabl esMBf undamental i nengi needng opti mi zati on. h e beamdesIl m
p l m in Chap r 1 a p ical sol uti on shoul d i nvol v i denti fyi ng an
off-the-shel f' beamas erol l i ngrnill will probabl y make it prohi bi ti vel yexpensi ve
for a l i mi tedproducti onof uni quebeam was i denti fi ed thesol uti onto e
conti nuous opti mi zati onprobl em. Intheprobl emregardi ng enumber of di fferent
pl acement machi nes the sol uti on was expected to be i ntegers (di screte val ue).
rnilarl choi ce of di an rsl engths washers val ves men components ck
SlzeS G soonareusual l ygui dedby avoi di nghi ghprocurement costs associ ated
wi nons dard components unl ess cost recovery is possi bl e because of l arge
vol umes. Co n uti IOl I l
i tems to be compl etel y manufactured i n"hou s (no ofl lf com nenl is
necessary)
Practi cal engi ne ri ng desi gn quirl al some desi gn vari abl es bel ong ( 0 an
ordered set of val ues. -di screte vari abl es. Thi s makes it a di screte opti mi zati on
318
DI SCRETE OPTI MI ZATI ON 319
probl em. However there are very f ewengi neeri ng probl ems are currentl y
sol vedas a di screteopti mi zati onprobl em. Fi rst di screteopti mi zati onal gori thms
are difficull to appl y. Second they are ti me consumi ng. Thi rd most di screte
al gori thms and code address l i near mathemati cal model s. Di screteopti mi zati onin
engi neeri ng wil1 necessari l y i nvol ve adaptati ons f rom e currentl y avai l abl e
techni ques used by the deci si on-maki ng communi l y. Such adaptati ons ar
uncommon. It is al soraretofi nd y book on desi gn opti mi zati on addresses
or i ncl udes thesubj ect of nonl i near di screteopti mi zati on notwi thstandi ngthefact
at it is enormous inextent. A modest effort is bei ng made inthis book toacquai nt
thereader wi ththesubj ct area.
letypi ca1approach for i ncorporati ngdi scretedesi gnvari abl es inengi neeri ngis
tosol vethecorrespondi ngconti nuous opti mi zati onprobl emand adj ust theopti mal
desi gnto nearest di screteval ues ( is is sirnilar totheroundi ng process toarri ve
at ani nteger number). Accordi ng toFl etcher [1] thereisno guarante i sprocess
is correct or agood sol uti oncan be obtai ned is way. Very often is roundi ng
may result in i nfeasi bl e desi gns. Gi ven the al temati ve is to sol ve the di screte
opti rni zati on probl emitself a systemati c approach to is roundi ng process has
become acceptabl e. A roundi ng process is based on mai ntai ni nga feasi bl edesi gn
is avai l abl einReference 2.
'hischapter and thenext wi l l differ si gni fi cantl y o mtheprevi ous chapters inboth
content and organi zati on. Thei r pri mary focus is presenti ng newi deas and
content rather andevel opi nga techni queor assi sti ngincode dev l opment . Di screte
probl ems requi re si gni fi cantl y di fferent concepts e traditional conti nuous
mathemati cal model s of theearlier chapters. Si mpl e exampl es areusedtobri ngout
e di fference. For exampl e deri vati ves gradi ents and Hessi an do not appl y to
di screte probl ems. By extensi on search di recti ons and one-di mensi onal stepsi ze
computati on l ose their rel evance. Whi l e ere are sveral excel1ent references on
di screteopti mi zati on al most all of t hemdeal wi thl i near mathemati cal model s. There
are f ew if any about nonl i near di screte opti mi zati on appl i ed to engi neeri ng
probl ems.
Di screteopti rni zati onprobl ems (DP) arei mpl i edwhen thedesi gnvari abl es arenot
conti nuous. Extendi ng concept of thesi deconstrai nts thesedesi gnvari abl
teger val ueof 8 =>bi nary val ueof 1000
320 DI SCRETEOPTI MI ZATI ON
Therefore if edesi gnvari abl ex is res ctedtoi ntegers between 0 S; x S; 16 thenx
can be pl acedby fi ve0- 1 desi gnvari abl es [y.. Y2 Y3 Y4YsJ f romwhi ch x c b
assembl ed as
x = YI (2
4
) + Y2(2
3
) + Y3(2
2
) + Y4(2
1
) + YS(2
0
)
Such a transfoI I Dati on is not recommended if re is a l arge number of i nteger
vari abl es. Engi neeri ngdesi gnprobl ems can be expectedtocontai nbothconti nuous
and di scre vari abl es. These aretermed Mi xed Programmi ng ( MP) probl ems if e
mathemati cal model is line Inthis book thecl assi cati onis extended tononl i near
probl ems too.
Intheinterest of manageabi l i ty onl y threemethods epresentedin is chapter.
These methods are onl y representati ve and inno way address topi c of di screte
op ni zati onsufci entl ylet al onecompl etel y. The thi rdone has seenl i mi teduse but
wi thcompu tional resources not bi nga hurdl etoday it is defi ni tel yattracve. The
sel ecti on does not represent any pri ori ty though emethods are among those at
have been appl i edofteninengi neeri ng. The methods of thenext chapter have evol ved
through their initial appl i cati ontodi screteopti mi zati on. They coul dal sobel ong to
is chapter but have been kept sep ate because ey are the dri vi ng force in the
search for gl obal opt i mumtoday. The methods of is chapter are (l ) E austi ve
Enumerati on (2) Branch and Bound (partial or sel ecti ve enumerati on) and (3)
DI ami cProgrammi ng.
8.1 CONCEPTS I NDI SCRETE PROGRAMMI NG
In the openi ng di scussi on some defmi ti ons of di sc te programmi ng probl ems
regardi ng the nature of di screte opti mi zati on were in oduced. l n is secti on the
concepts aredetai l edusi nga si mpl e unconstrai nedmi ni mi zati onexampl e. As a result
e opti mal i ty issu do not in de on presenti ng the i mportant i deas in di screte
opti mi zati on i ncl udi ng the treatment of fi ndi ng a conti nuous sol uti on fol l owed by
adj usti ngthedi scretedesi gnvari abl es tonei ghbori ngdi screteval ues.
Exampl e 8.1 Mi ni mi ze obj ecti vefuncti ongi venbel ow wherexl is a conti nuous
vari abl eand X2 X3aredi scretevari abl es. X2must have a val uef romtheset [0.51.52.5
3.5J and x3must have a val ueu mtheset [0.220. 75 1. 732. 24 2.78J.
Mi ni mi zef (x
1
x
3
) = (x
1
- 2)2+ (x
1
- ~ 2 + (x
1
- X
3
)2+ (x
2
- x
3
)2 (8. 1)
The si deconstrai nts on thedesi gnvari abl es canbe set up as
x
1
E R (8.2a)
X
2
E .5 1. 5 2.5 3.5J E xu (8.2b)
.1 CONCEPTS INOI SCRETEPROGRAMMI NG 321
x
3
[0.22 0. 75 1. 73 2. 24 2.78] E X
3d
(8.2c)
The symbol E i denti fi es at thevari abl eon theleft can have one of theval ues on the
ri ght side. InEquati on(8.2a) R stands for a real val ue. Thi s represents a standarduse
of symbol E dis an effecti veway of expressi ngthei deathat thedi screteval ues
C onl y be sel ected f roma group of val ues. Tbi s is a mi xed programmi ng ( MP)
probl em.
8.1.1 Pr obl emRel axat l on
P
4
4
M
J
E is were a unconstrai nedprobl eminconti nuous vari abl es thesol uti oncoul dbe
obtai nedby setti ngV f = 0 and sol vi ng e eequati ons for eval usof x.. X2' and
X3Eq ti ons 8. 2b and 8.2c h to be overl ooked. Al tematel y for a conti nuous
probl em i nspecti onof theobj ecti vefuncti onyi el ds efol l owi ngsol uti on:
x; =2; x; =2; x; =2; I =O (8.3)
For theori gi nal probl em 'dXl is defi ned but not df/ dx2or df/ dx3' si nceX2 dX3
aredi screteval ues. Deri vati ves aredef med by taki ngthel i mi t of theratioof change
of obj ecti vefuncti onfto thechange intheval ueof thedesi gnvari abl e as thechange
inthevari abl eapproaches zero. The val ueof obj ecti vefuncti onfi s onl y defi ned
at sel ected combi nati on of X2X3 in Equati on (8.2) and is noL defi ned el sewhere.
Smal l l i nfi ni tesi mal changes in the di screte desi gn vari abl es and therefore in e
obj ecti ve functi on are not defi ned in Exampl e 8.1. Deri vati ves wi th respect to
di scretevari abl es do not eXI st. Thi s concl usi onisof m 01' si gni fi canceas it makes this
book' s previ ous body of wor k of I i mi ted use in epursui t of tbesol uti ontotheDP.
To recol l ect enecess yand suffi ci ent condi ti ons whi ch drove theal gori thms were
based on thegradi ents and thei r deri vati ves. Two areas intheprevi ous chapters escape
wi th a l i mi ted i mpact of is statement. beyar L P and zero-order methods for
numeri cal sol uti on to unconstrai ned nonl i near opti mi zati on. Both of t hempl ay a
si gni fi cant roleindi screteopti mi zati onprobl ems. It is thereforepossi bl etoconcl ude
at sol uti on to Exampl e 8.1 establ i shed in Equati on (8.3) is i ncorrect ( esi de
constrai nts areinvi ol ati on).
InDP thesol uti oninEquati on(8.3) represents thesol uti ontoa rel axati onprobl em.
Probl emrel axati oncan takeseveral forms. Mostl y it is appl i edto erel ax onor e
weakeni ng of the cons ai n or the obj ecti ve functi ons. There no expl i ci t
constrai nts inExampl e 8.1. I n is i nstance therel axati onrefers totheremoval of the
restrictionof di screl eness of the vari abl es. Thi s is i denti fi edas conti nuous rel axati on.
The rel edprobl emor rel axati onhas several advantages [3].
. If a constrai nt r ll axati onis i nfeasi bl e sois theprobl eml model it rel axes.
Constrai nt rel axati onexpands theset of feasi bl esol uti ons. The rel axedopti mal
val ue must i mprove or equal the best feasi bl e sol uti on 10 e ori gi nal
probl eml model.
322 DI SCRETE OPTI MI ZATI ON
The opti mal val ueof any rel axedmodel provi des a l ower bound on eopti mal
sol uti onif il is a mi ni mi zati onprobl em. Si mi l arl y il establ i shes an upper bound
for maxi mi zati on probl ems.
Many rel axati onmodel s produc opti mal sol uti ons at are easi l y rounded to
good feasi bl e sol uti ons of the ori gi nal probl eml model . lis appears to
di screteopti mi z onfor engi neeri ngdesi gnprobl ems.
The val ues establ i shedinEquati on(8.3) area sol uti onto conti nuolI srel axati onof
Exampl e 8. 1. It was easy toobtai n is sol uti onwhi l e a techni que for thesol uti onto
e di screte un i ned op zati on is still unknown e Kuhn-Tucker
condi ti ons areno l onger useful . lesol uti oninEquati on(8.3) is not acceptabl e- xi
and xj arenot el ements of thepermi ssi bl ese1(8.2). The val ue of I = 0 wi l l be betl er
best di scretesol uti on- l ower bound 0" thesol uti on10 ori gi nal probl em.
8. 1. 2 Dl screte C! ptl mal Sol ut l on
A standard approach 10 sol vi ng di scre1e opti mi zati on probl em parti cul arl y in
engi neeri ng is 10 use the conti nuous rel axati on of the mathemati cal model and
genemte a conti nuous sol uti on. For Exampl e 8.1this is gi ven inEquati on(8.3). Mor e
thanl i kel y edi scretevari abl es wi l l not bel ong tothep defi neddi scretesets. These
vari abl es arethenchanged (or rounded) to enearest di screteval ues. For probl ems
wi th constrai nts feasi bi l i ty is checked. The best feasi bl e sol uti on is l hen chosen.
Unl i ke conti nuous probl ems there a no necessary and suffi ci ent condi ti ons to
satisfy.
ln e case of Exampl e 8. 1 Equati ons (8.2) and (8.3) provi de the rel evant
i nformati ontoestabl i shthesol uti on. For conveni ence tJ l ey arereproduced here:
XJ R (8. 2a)
X
2
e [0.5 1.5 2.5 3.5] e X
2d
(8.2b)
X
3
[0.22 0. 75 1. 73 2. 24 2.78] E X
3d
(8. 2c)
(8.3) x ; = 2; X; = 2; x ; = 2; I = 0
l e conti nuous vari abl eXJ is chosen 10have a val ueof 2.0. Fi gu 8.1 i ndi cates four
sets of di sc eval ues for X2 and X3 around theconti nuous sol uti on. The least val ueof
theobj ecti vefuncti onat thesepoi nts wi l l be consi dered esol uti on. Eval uati ngthe
val ues of theobj ecti ve ncti onat thosepoi nts:
x ; = 2; X; = 1.5; X; = 1.73; 1 =0.3758 (8.4a)
x ; = 2; x ; = 1. 5; X; = 2.24; 1=0. 8552
(8.4b)
l '
8.1 CONCEPTS INDI SCRETE PROGRAMMI NG 323
3
'
1
+
'
1
1
1
t
'

-
e
'
i
r
i
-
-
'
'
'
1
7
i
b
o
a
-
-
t
J
o
-
-
T
A
t
l
l
1
l
r
-
-
-
-
a
w
U
-
o
0.5

_ L _ _
0.5 1.5 2 2.5

Dlscretevorloble~
Fl gure8.1 Dl scretenel ghborhood01conti nuous solution.
3.5
~ =2; X; = 2 xi = 1. 73; ] = 0. 9158
(8.4c)
x ~ = 2; x ; = 2.5; xi = 2.24; ] = 0.3752 (8.4d)
Fr omthis exerci se e 8t val ueof eobj ecti ve functi onis avai l abl einEquati on
(8. 4d) and woul d be regarded as the adj usted opt i mumsol uti on to the ori gi nal MP
probl em. lis is al sothecommon practi ceinengi neeri ngdesi gnopti mi zati on.
The best sol uti on however is obtai nedat
x;=1. 7433; X;=1. 5; x;=1. 73; ] =0. 1782 (8.5)
The di fferencebetween Equati ons (8.5) and (8.4a) onl y in evari abl eXI suggests
t addi ti onal conti nuous opti mi zati on needs to be perf ormed for each set of E
di screte val ues sel ected conf i rmi ng the rlier observati ons by Fl etcher. us
opti mi zati onshoul d be easi er as theorder of themathemati cal model wil be duced
by thenumber of di scretevari abl es si ncethey have beensi gned numeri cal val ues.
Thi s si mpl e exampl e commendsat every least a three-stepprocedw for probl ems
incudedi scretevari abl es:
Step 1: A conlI nuous rel axati onthat i denti fi es several sets of di scretevari abl es for
further expl or on.
324 OI SCRETE OPTI MI ZATI ON
Step 2: For each such set of di screte vari abl e combi nati on a conti nuous
opti mi zati onis perf ormed 10establ i sha newopt i mumva1ue of theconti nuous
vari abl es and thecorrespondi ngobj ecti ve functi on. If all of l hevari abl es are
di screte thenonl y thefuncti onand constraI nts need tobe eva1uatedat eachof
thesel of vari abl es.
Step3: A si mpl e comp i sonof theabove sol uti ons/va1ues inStep2 toi denti fy e
opt i mumsol uti onto edi screteprobl em.
Thi s unconstrai nedopti mi zati on exampl e in three vari abl es has demonstrated
that di screte opti mi zati on requi res a lot of wor k compared to conti nuous
opti mi zati on. Thi s wor k expands si gni fi cantl yif thenumber of vari abl es i ncreases
or if the mathemati cal model is enhanced through the i ncl usi on of constrai nts.
Another essenti al f l re in the above expl orati on is no newmathemati ca1
condi ti ons were necessary for stabl i shment of the di screte sol uti on beyond a
si mpl e compari son of the obj ecl i ve functi on. The natureof di screte vari abl es and
di screl e functi ons precl udcs any sophi sti catcd mathemati cal condi ti ons
establ i shed by deri vati ves of the functi ons i nvol ved in tbe model. Trappi ng and
branchi ngbased on compari son of val ues are the mai nstay of di screte al gori thms.
There techni ques arecl assi fi edas heuri sti c methods. Thi s encourages uni que and
personal i mpl ementati ons of the search for di screte opti mi zati on that can h
tai l oredfor a cl ass of probl ems.
8. 2 DI SCRETE OPTl MI ZATI ON TECHNI QUES
There arethreedi screteopti mi zati ontechni ques inthis secti on. The first one is
Exhausti ve Enumerati on. Thi s i nvol ves i denti fyi ng tbe sol uti on to e
mathemati cal model for all possi bl ecombi nati ons of thedi scretevari abl es. Thi s
is suggesti ve of the zero-order numeri cal techni ques of Chapter 6. Those
met hods l ackcdsophi sti cati onas they i nvol ved onl y eval uati onof thefuncti ons
at a phenomenal number of poi nts. They were abl e to take advantage of th
pl enti ful computer resources avai l abl e today. I magi ne operati ng in the
peer-to-peer computi ng envi ronm nt afforded by MP3" or Gnutel l a" and
sol vi ngtheprobl emusi ngall of thePCs intheworl d. The second met hod is the
Branch and Bound method. Thi s is based on parti al enumerati on where onl y part
of thecombi nati ons areexpl ored. The remai ni ng arepruned f romconsi derati on
because they wi11not determi ne thesol uti on. Thi s is currentl y themost popul ar
met hod for di screte opti mi zati on for engi neeri ng desi gn. The last met hod is
Dynami c programmi ng an el egant approach to opti mi zati on probl ems but
whi ch di d not gai n favor because it i nvol ved si gni fi cantl y l arger amount s of
computati on thancompeti ti ve met hods even for probl ems of reasonabl e size. It
is restri cted to probl ems thal requi re a sequenti al sel ecti on of the desi gn
vari abl es. Today such resourcI i mi tati ons are di sappeari ng in the worl d of
powerf ul PCs and hence l hemet hod deservcs lobc revi si ted.
l
;;:;11
8 2 OI SCRETE OPTI MI ZATI ONTECHNI QUES 325

St andard Format - Dl scret e Optl m/ zatl on: The author is not aWllI of a
standardformat for di screteopti mi zati onprobl em. It subsumcs format of l he
copondi ng conti nuous rel axati onprobl cm. Inthis book Ihc fol1owi ng format fol'
themi xed opti mi zati onprobl emis used.
Mi ni mi ze f (X Y ) ]n<; [Y]n
d
Subj cct to: h( X Y) = [0]; [hJI
g( X Y) [0]; [g]m
J; S Xj S.i;'; ;=12.. . n
c
(8.6)
(8.7)
(8.8)
(8.9)
Yi E Yd ; [ Yd]p ; i = 1 2. . . nd (8. 10)
X represents l he set of nc conti nuous vari abl es. Y rep senl s the set of nd di scrcte
vari abl es. f is eobj ecti vefuncti on. h is l heset of 1 equal i l yconstraI nts. g is l hesel
of m i nequal i ty constrai nts. Expressi on(8.9) rep :sents esi decons ints on each
conti nuous vari abl e. Expressi on(8.10) expresses esi deconstrai nts on thedi screte
vari abl es. Each di screl evari abl eYi must bel ongtoa preestabl i shedset of Pi di screl e
val ues Yt/I' If nd = 0 nil is a conti nuous opti mi zati onprobl em. If 11c = 0 cnit is
a di screteopti mi zati onprobl m. If botharep sentthenit is a Mi xed Probl em.
Cont l nuous Rel axatl on: The conti nuous relax on of lhe mi xed opti mi zati on
probl cm(8.6 (8.10) is identi ca110probl cm(8 (8.10) wi thY as a conti nuous vari abl e
t . lCdi scre ns nl (8.10) is repl ac by a conti nuous si de cons int for e
di scretevari abl es of thef ormof (8.9). is not explicitlydevel opedsi nce(8.6) (8.9)
and a modi fi ed (8.10) is fairly representati ve of econti nuous rel axedmathemati cal
model .
Reduced Model : The ducedmodel is i mportant insubsequent di scussi ons. It is
emodel sol vedafter a part of theset of di scretevari abl es areset at some al owabl e
discret va1ues. Thi s removes l hosevari abl es f rom eprobl emas ir val ushave
been set. The mathemati cal model is thendefi nedby theremai ni ng di scretevari abl es
wel e ori gi nal conl i nuous vari abl es. If Z( n
z
) are e remai ni ng di screte
vari abl es need10be sol ved en
Mi ni mi ze l ( X Z) [X]" ; [Z] n (8.11)
Suect 10: ( XZ) = [0]; [11]/ (8.12)
g( X z) [0]; [i ]m (8.13)
J; S Xj S xj'; i = 1 2 . . . nc (8.14)
Zi e z.t; [z.t): (8.15)
326 DI SCRETE OPTI Ml ZATl ON
Tobl e 8.10 Opmal Vol ue of Coououous Vori abl ex. for Di screteCombl oauoo
f Xl oodx3
X21x) 0. 2200 0. 7500 1. 7300 2.2 2.78
0.5 0. 9067 1. 0833 1. 4100 1. 5800 1. 76
1. 5 1. 2 O 1. 4167 1. 7433 1. 9133 2. 0933
2.5 1. 5733 1. 7500 2. 0767 2. 2467 2. 4267
3.5 1. 9067 2. 0833 2.4100 2. 5800 2. 7600
Not e t in the above 4 is not a new set . It co ponds to a reduced Y d.. In
Exhausti ve Enumerati on general l y it is empt yset . AI I of thedi scretevari abl es
set tosome al l owabl e val ue.
8. 2. 1 Exhaust i ve Enumer at i on
Thi s is esi mpl st of edi screteopti mi zati ontechni quS. It eval uates an opt i mum
sol uti onfor all combi nati ons of the di screte vari abl es. The best sol uti on is obtai ned
by scanni ng thelist of feasi bl esol uti ons in eabove i nvesti gati onfor themi ni mum
val ue. The total number of eval uati ons is
/ 1. = IIp/ (8.16)
If ei thr nd or Pi (or both) is l arge n is represents a lot of work. lt al sorepresents
an exponenti al growt h inth cal cul ati ons wi th number of di screte vari abl es. I na
mi xed op zati onprobl em is woul d i nvol ve nc conti nuous opt i mumsol uti onof
thereduced model. If themathemati cal model and its mput er cal cul ati ons are sy
to i mpl ement this is not a seri ous burden. I f mathemati cal model qui res
extensi ve cal cul ati ons for ex npl e ni te el ement or finite di fference
cal cul ati ons- then some concemmay ari se. I nsuch casesl i mi ti ngtheel ements of the
di scr tevari abl es tosmal l er c1usters may rep sent a useful approach.
Exampl e 8. 1: be exhausti veenumerti onof Exampl e 8.1isavai l abl einEx8_1. m.
1
Tabl e 8.1a shows opt i mumval ue of theconti nuous vari abl ex I for every a110wabl e
combi nati on of di scretevari abl es X2and x ) Tabl e8.1b shows correspondi ng
val ue of theobj ecti vefuncti on for each of the cases. Si nce is is an unconstrai ned
probl em no di scussi on of feasi bi l i ty is i nvol ved. There are 20 val ues and the best
sol uti onfor Ex npl e8.1 is
~ = l. i 433; X; = 1. 5; x ; = 1. 73; r = 0. 1782 (8.5)
There is some good news though. Fi rst each of the 20 cases is essenti al l y a
s;ngle- lI ari abl eop zati onprobl em. The number of desi gnvari abl es in ori gi nal
IFil es 10 bedowl l l oadedfromIheweb a indicalcdbyboldfuce S S iflype.
8.2 OI SCRETE OPTI MI ZATI ONTECHNI QUES 327
Tobl e 8. 1b Opumol Val ue of Obj ecuve Fuocuoo for Dl screl eCombi ooti on of Xz aod X3
x/X3
0. 2200 0. 7500 1. 7300 2. 2400 2. 7800
0.5 1. 9107 1. 3542 2. 7915 4. 8060 7. 8840
1. 5 3. 3240 1. 3542 0. 1782 0. 8327 2.4707
2.5 8. 0707 4. 6875 0. 8982 0. 1927 0.3907
3.5 16. 1507 11. 3542 4. 9515 2. 8860 1. 6440
model (3) is reduced by e number of di screte vari abl es (2). Model reducti on is
i nvol ved inenumerati on techni ques.
Al gori thm: E. ' austl veEnumerat i on (A8. 1)
Step 1. ! = i nf X = [0 0 .. 0]
For every a1l owabl e combi nati on of (Y.. Y2 Y"d) = ( Yb)
Sol ve Opti mi zati on Probl em(8. 11)(8. 15) (Sol uti onX)
I f h( XY
b
) = [0] and
I f g( X. Y
b
} [0] and
If j ( XY
b
) <!
Thenf ft. X Yb}
XX.
Y Yb
End If
End If
EndIf
EndFor
Exampl e 8. 2iII ustrates useof al gori thmA8. 1 wi th feasi bi l i ty requi rements.
Exmnpl e 8. 2 The Aerodesi gn Cl ub has i ni ti ated fund-rai si ng activiti by sel l i ng
pi zza duri ng l unch inthe student acti vi ty center. Over the y rs ey have col l ected
ta to i denti fy a mathemati cal model that wi l l r uce eir cost whi ch di rectl y
tr sla to hi gher profi ts. Onl y t wo types of pi zza wi1I sel l - pepperoni (x.) d
cheese (x21. The l ocal pi zza chai nwi1I del i ver a maxi mumof 26 pi zz . Factori ngi nto
account wastage and the ti me avai l abl etomake thesal e ey arri veat thefol1owi ng
mathemati cal model :
Mi ni rni ze f(x.. X2) = (XI - 15)2+ (X2- 15)2
Subj ect to: 8. (Xh X2): XI + X2S 26
g2(X. x21: x. + 2. 5x2S 37
(8.17)
(8. 18a)
(8. I 8b)
328 DI SCRETEOPTI Ml ZAnON
XI ~ 0; x2 O. and i nteger (8.18c)
The graphi cal sol uti on for conti nuous rel axal i on is avai l abl e in Fi gure 8.2and the
anal yti cal Sol ul i on (not i ncl uded) i denti fi es that 12. 86 pepperoni pi zzas and 9. 65
cheese pi zzas will provi de for themi ni rnumcost of operati on.
Ex8_2. mis e m-fi l e at wi l l perf ormthe exhausti ve enumerati on of the
probl em. The di scretesol uti onis
X; = 12; xi = 10; r = 34; g2is acti ve
(8.19)
Thi s confi rms tnei ghbori ngdi screteval ues around a di screl esol uti onworks wel l .
Programmi ng exhausl i veenumerati on is strai ghtforwardand parti cul arl y easy wi th
MATLAB. Gi ven processi ngspeed l argememor y avai l abi l i tyon edesktop. sy
programmi ng throughsoftware likeMATI ..AB. e ti veenumerati on is probabl y a
very good i deal oday. Whal recommends it even further is thesol uti onis a gl obal
ti mum. FlI rthermore. unl i ketheearl i er chapt rs where th concepts of deri vati ves.
matri ces. l i near i ndependence. and soon were essenti a1tounderstandand i mpl ement
thetechni ques of opti mi zati on l ack of sucb knowl edge is no di sadvan getosol vi ng
a di screte opti mi zati on probl emusi ng exbausti ve enumerati on. Tbere is onl y one
essenti aI skill Is1ati ng mathemati cal model i nto programcode. Wi t h E
experi ence provi dedin is book. this is defi ni tel y not an issue.
40
35
g ~ O
. .
- ....
6 i e - 7
5
10 15 20 25 30 35 40
pepperonl pl zza
Fl gure8. 2 G phical solutlonExampl e 8.2.
;
8.2 DI SCRETEOPTI MI ZATI ONTECHNI QUES 329
esol uti oninEquati on(8.19) canbe obtai nedwi thconti nuous rel onof E
probl em. In this case the di screte sol uti on has to be recovered by adj usti ng the
conti nuous sol uti ontonei ghbori ngdi screteval ues.
Exhaust#ve Enumer at l on for Cont l nuous Optl ml zatl on Probl ems: Thi s title
must be quaI ified f urther- i n lieu of gradi ent-basd numeri caI techni ques. The
avai l abi l i tyof seri ous desktop computi ng power. thepromi se of gl obal opti mi zati on.
and easeof i mpl ementati on suggesl a mul ti stageexhausti ve enumerati on strategy
for conti nuous opti mi ti onprobl ems. Exhausti ve enumerati on after all is a search
strategy. si mi l ar 10 the ones used before. The search for sol uti on is aki n 10 e
zero-order mel hods empl oyed for unconstrai nedopti mi zati on(Chapter 6).
The hi gher stages in mul ti stage enumerati on i nvol ve enumerati ng on decreased
i ntervaI s centered around e sol uti on f rom previ ous stage. The vaI ues of the
desi gn vari abl es used for enumerati on wi l l i ncorporate progressi vel y i mproved
tol erance. The strategy must takei nto account thepossi bi l i ty that theiniti aI i nterval
mi ghl overl ook gl obaI opti mal sol uti ons tllat do not appear attracti ve ina parti cul ar
stage. Hence. mor e than one mi ni mumis tracked in each stage. Each of these
mi ni mumcan be consi dered enl l merati oncluster. Each such c1uster is mul ti staged
until a prescri bedtol eranceis achi eved. Inengi n ri ng. m ufacturi ngtol eranceof the
order of 0. 0001 is achi evabl e today. Tol erances bel ow is l i mi t probabl y need tobe
j usti fi edunl ess eareais mi croel ectroni cs. Fr oma programmi ng perspecti veit jllst
adds an extra l oop to the i mpl ementa1i on. le final sol uti on obtai ned represents a
conti nuous sol uti onwi thi nthefinal l ol erancei mpl emented by thetechni que.
Such an exerci seis l efl 10thestudent . As wi thall i mpl ementati on themeri t of lhis
approach lies inthetractabili1y of themathemati cal model and theneed for a gl obal
opt i mumsol uti on. Si nce themet hod is l argel yheuri sti c thereis no guarantee that the
sol uti onis actuaI lya gl obal opt i mum(of course assumi ng thefuncti ons i nvol vedare
not convex).
8. 2. 2 Br anch and Bound
Exhausti ve or compl ete enumerati on is possi bl e onl y for a l i mi ted set of vari abl es
because of theexponenti aI growth of effort qui dtoexami ne aI l possi bl esol uti ons.
Usi ng parti al (or i ncompl eteor selective) enumeral i on toarri veal thesol uti ontothe
di screl e opti mi zati on probl emwoul d provi de tlt necessary advantage inhandl i ng
l arge probl ems. Partial enumerati on works by maki ng deci si ons about groups of
sol uti ons rather every si ngl one. The 8ranch and 80und ( 88) al gori thmis one
of thesuccess I al gori tl1ms touse sel ecti ve/partial enumerati on. lt does soby usi ng
rel axati onmodel s i nsteadof theori gi nal compl e1e di screl ema1hemati caI model . The
8 B al gori thmuses a treestructure10represent cqmpl eti ons of parti al sol uti ons. The
'ees uctureuses nodes and edges/li nks 10represenl l hetrail of partial sol uti ons. Thi s
is call fatl romul g ep tial sol uti on. The termi nol ogy is borrowed f romoperati on
rese h literature [3] and is expl ai nedbel owwi resp to Exampl e 8.1 whi ch is
reproduced herefor conveni ence:
Mi ni mi ze f(Xl X2X3) = (XI - 2)2+ (XI - X2)2+ (XI - X3)2+ (X2- X))2(8.1)
XI E R (8.2a)
X2[0. 5 1. 5 2.5 3.5] E X
2d
(8.2b)
X3E [0.22 0.75 l. 73 2.24 2.78] X
3d
(8.2c)
Partl al Sol utl on: A sol uti oninwhi ch some discretedesi gnvari abl es arefi xedand
o ers arej 'eeor undetenni ned. Vari ab1es that arenot fi xedarerepresentedby e
symbo1 :" For exampl e
x = u : 0.5 (8.20)
rep sents theopti mi zati onprob1emwi X2= 0.5and XI and X3tobedetermi ned. The
pa al sol uti ons ara1sotermedthenodes of thetree. lefreevari abl es areusual l y
sol vedusi nga conti nuous rel onof themodel .
Compl er / ons (0' PaTt Sol utl on): Ench partial sol uti onor nod cangivt: rist: lu
addi ti onnl partia1 sol uti ons/nodes di rectI y under it. Thi s is called bral l chi ng or
expansi oTl of lhe node. In this newset of nodes another one of frec discrelc
vari abl cs is fi xedand eremai nder slill conti nuetobefree. For exampl e under
node representedby thepartial sol uti oninEquati on(8.20) fiveaddi ti ona1nodes are
ssible:
X = [ f 0. 5 0.22]; X = [ f 0.5 0.75]; X = [ f 0. 5 1. 73];
X = [ f 0.5 2.241; X = [ f 0. 5 2.781
Thesc compl eti ons branchout andhcnce a treelikc structure. There is a hi erarchi cal
structuri ngof thenodes. Eachbranchrepresents 1eads toa tier inwhi ch some vari abl e
is hel dat its prescri beddiscretevalue.
Fat homi ng t he Parti al Sol utl on: Expandi ng thenode all theway till theend is
termedf atl wmi ng a node. Duri ngthe fathomi ngoperati on it is possi bl e10idenlify
thosepartial sol uti ons at a node that do not needtobe i nvesti gatedfurther. Onl y the
best sol uti onat e node is branched or compl eted. Thosepartial solu t io nsll I no odes at
will not b ei nvesti gatedfurther arepruned 0 r tenni natedi n elI C.
Act l ve Partl al Sol uti on: A node thal has not beentenni natedor prunedand has
nol bcencompl eted.
EdgeslLl nks: Thes ' enode-tonodeconnecti ons ina . They identifytheval ue
_ r _ ~ ..n_ L '....... : . . . . . . . . J
8.2 D1SCRETE OPTI MI ZATI ONTECHNI QUES 331
Root: Th BB al gori thmsearches a10ng a II C. lt starl S al the root where a11 the
23t :31? 1 J : :;;:??J22::
preferredtoj umpi ng across nodes in esame erarchy.
I ncumbent sof tJ Mont The i ncumbent solutionisthsbGSt feasiblesolutionMown SO
far wi tha11 discretevariables assumi ngdiscretevalues. When lh BBsearchfinal1ystops
Ehal i nCUREbBEEt sol uti oni fomexi stsisa zi obal opti mumsoi tHi on-hthefol l owi ng
thei ncumbentsol uti onis denotedbyr- - 13mi ncumbaudESi gn vmabl es X...
Exampl e 8. 1- Usl ng Br anch and Bound: Here e BB search is appl i ed
Exampl e 8. 1. BBEX8J J n cal cul ates al l of thEdataneeded for genEraI i ng
TheBB itse1f is devel opedinFi gu s8.3and 8.4.
Start: Node O. Al l of vari ab1esaref C. The sol uti ontotheconti nuous rel axati on
ofthe obj tivefuncti on(sol vedas a conti nuous probl em) is identifiedat the
node.
Nods1-4: Thi s is th first tier of theBB tre Thedi scretevari abl eX2 is assi gned
Eachof thgpenni sBi bl edi screteval ues at thenudcs inthisti er-Thi s ESi ndi cated
X=[ fffJ
X* = [1.741. 5 1.73]
p =0.178
p* = 0.178l ncumbenl solutlon
XoI". = [1.74 1. 5 1.73J
Fl aure8.3 BranchandboundExampl e8.1.
332 Dl SCRETE OPnMI ZATI ON

X" = [1.57 2.5 0.22]
P=8. 07
X" = [1.752. 5 0.751
P=4. 68 xe=I 107zs1.731 x' =I 2. 26zaz. Ml
P = 0.898 P = 0.192
F1gure 8.4 Branchandb undExampl e8.1-fathomi ngNode 3.
on the edges between the nodes in Fi gure 8.3. The remai ni ng v i abl es B
fr I conti nuous. The opt i mumsol uti onat thevari ous nodes is di spl ayedon E
tree. The mi ni mumsol uti onis theone at will be f athomed at thenext step.
Here Nodes 2 and 3 - e candi dates for expandi ng thebranches. Nodes 1 and 4
will not be f athomed (unl css their opt i mumval ueis better anIhei ncumbent
sol uti on) ey will be termi nated or pruned e obj ecti ve functi on has a
hi gher val uc. l e conti nuous rel axati onensures a l ower bound on theopt i mum
val ueof theobj ecti veat thenode. Node 2 is takenup for f athomi ng based on
node orderi ng
Compl eti ons of Node 2: Thi s expands Node 2 by setti ng edi scretevari abl eX3 10
its permi ssi bl eset of di screteval ues. It creates branches Nodes 5-9. lt is al so
thesecond tier of thetree. The sol uti onis i ndi catedat evari ous nodes. Inall
of themthe t wo di sc te vari abl es areset 10i ndi cal edval ues whi l e Xl is free.
whi ch is a conti nuous vari abl eby defi ni ti on. Si nce all of thedi screl evari abl es
bave di screte val ues this node is al so fathomed. Node 7 i ndi cates the best
sol uti on. Si nce all of e di sc te vari abl es ar assi gned di sc le val ues E
sol uti onat lhisnode is thecurrent i ncumbent sol uti onTheremai ni ng nodes will
be pruncd. The currenl i ncumbent sol uti on (f. = 0. 1782) is hi gher
sol uti onal node 3 (f = 0.15). Thi s requi res Nod 310be fathomed.
Compl eti ons of Node 3: Fi gure8.4iIl ustrates thebranchi ngat Node 3. The process
is si mi l ar 10compl eti ons of Node 2 above. The best sol uti onis at Node 13wi th
a val ueof f = 0. 1927. This is l argerthantheCurrent i ncumbent sol uti on. Al I of
enodes can be l ermi nal ed.
No further nodes areavai l abl e for expansi on. The current i ncumbent sol uti onis
al so thegl obal opti mal solution. Thi s is thesol uti onto eopti mi zati onprobl em. In
8.2 Dl SCRETEOP nMI ZATl ONTECHNI QUES 333
is exampl e 15 partial sol uti ons were n essary. Each partial sol uti on can be
consi dered an enumerati on. In exhausti ve enumerati on 20 numerati ons were
requi red for Exampl e 8.1. There is a nel gai n of 5 enumeral i ons. Th BB search
redu sthenumber of total enumerati ons needd 10sol ve eprobl em.
Al gorl thm: Br anch and Bound 8.2): For even a smal l set of di screte
vari abl es actual l y drawi ng th BBtreestrucl ureis most l i kel y a difficult enterpri se.
The 11' ee structure merel y i denl i fi es the search process. Exampl e 8.1 di d not have
constrai nts so feasibility was nol exami ned. Wi t h constrai nts present choosi ng the
p tial sol uti ontocompl ete based onl y on mi ni mumval ue of theobj ecti vefuncti onis
not a j usti fi abl e s tegy. In this case the besl feasi bl esol uti on is e candi date for
compl eti on. Al so an acti ve feasi bl epartial sol uti on has a better val ue
current i ncumbent sol uti on is al so a candi date for compl eti on. The fol l owi ng
al gori thmcapres essenceof theBranch and Bound al gori thm.
StepO. Establ i shtheroot node wi thall di scretev i abl es .
lnitializesol uti oni ndex s toO.
. .
Sel current i ncumbent soI uti on tof -- = "" X- - = [] (mi ni rni zati on
probl em).
Stcp 1. If nny acti vefeasi bl eparua1sol uti ons remai n ( is beuer thanthecurrent
i ncumbenl sol uti on) sel ecl one as X<J) and go toStep2.
If none exi sts and thereexi sts an i ncumbent sol uti onil is opti ma1.
If none exi sts and the is no i ncumbent sol uti onthemodel is i nfeasi bl e.
Step 2. Obtai n e compl eti on of the partial sol uti on Xusi ng a conti nuous
rel axati onof theori gi nal model.
Step3. If there e no feasi bl ecompl eti ons of partial sol uti onX termi nate
X<I) i ncrement s s + 1 and retumtoStep1.
Step4. If thebest feasi bl ecompl eti on of partial sol uti onX<I) cannot i mprove the
i ncumbent sol uti on thenterrni nateX i ncrement s s + 1 and retumtoStep
Step 5. If e best feasi bl e compl eti on is better the i ncumbent update the
i ncumbent sol uti on
r. !<I) x.. x
termi nateX<s> i ncrement s s + 1 and retumtoStep 1.
Exampl e 8. 2 The Branch and Bound al gori thmis appl i ed to Exampl e 8.2. The
S h is appl i ed through the BB . le mathemati ca1 model for Exampl e 8.2
whereXl andX2 arei ntegers is
Mi ni mi ze !(x" x v = (Xl - 15)2+ (x2- 15)2 (8.17)
334 DI SCAETEOPTI MI ZATI ON
Subj ect 10: 81(X" X2) : X( + x2 :!>26
82(XI X2) : X( + 2.5x2 :!>37
XI 0; X2 2! 0
The sol uti onis
(8.18a)
(8.18b)
(8.18c)
X;=12; X;=10; f =34; g2is acti ve (8.19)
i z : ; 2 2 J 3 1 f e? L i t z 522e i f z : : i : : z z t s z
fol l owi ngdevel opment
X
1
E [10 Il 12 13]
E [9 10 11 ]2)
Usl ng Br anch and Bound: eBB eis shown inFi gure8.5. The datafor e
C cuJat through BBEx 2. mwhere theconti nuous l ation is sol v
anal yti caIl y usi ngthc K T conditions. 'he treeis brnnchedns fol l ows:
XCo:::[139)
1'* =40
X. = [12. 86 9.951
1'* :::33.13
1nfeaslble
1nf slble
Rgure 8.5 BranchandboundIr for Example8.2.
Infslble
8.2 DI SCAETEOPTI MI ZATI ONTECHNI QUES 335
NodeO: is theroot node. Al l thevari abl es efr. econti nuous rel axati on
of theprobl emis sol ved. The sol uti onis i ndi catedon tree. Currentl ythis is
theonl y avai l abl eactivepartial sol uti onand hence needs tobecompl eted.
Nodes ] - 4: epartial sol uti onat Node 0 is compl etedusi ngvari abl eXI f romthe
reduced discrete set identified above. Thi s l eads to four branches and the
opti mal val ues ar i ndi catedon the . The best val ues are i ndi catedat e
nodes. For i sprobl em. y arealsofeasible. There enowfour activepartial
solutions.
Node 3: The so]uti onat Node 3 is alsoa feasiblesol uti onof theoriginal discrete
opti mi zati onprobl emsi ncethedesi gnvari abl es aredi screl eand bel ongto e
permi ssi bl ediscreteset. Node 3 i sfatJ l Omed. is provi des thefirst i ncumbent
solution.
Th rearethreeavai l abl eacti vefeasiblepartial sol uti ons ( Nodes 12 and 4)
for compl eti on. Si ncethesol uti ons at enodes areall feasible onl y thebest
activefeasiblepartial sol uti onis pi ckedup for compl eti on- Node 4. Nodes 1
and2 canbetermi nated.
Compl eti on Node 4: Node 4 has a feasible sol uti on better e current
i ncumbentsol uti on. l ti s compl etedby branchi ng10Nodes 5 6 78. Onl y Node
5 has a feasib]esolution. Tbe besl sol uti onal Node 5 isnot better thanthecurrent
i ncumbent solution. Nodes 5 6 7 8 canbe termi nnted.
There areno more acti vepa al sol uti ons and ehas becnfathomed. The
current i ncumbent sol uti onis theopti mal solution.
lis compl etes the BB search appl i ed to Exampl e 8.2. The ducti on in
enumerati on compared to exhausti veenumerati on is difficult 10comment on for
two reasons.
1. The del i beratechoi cetorestrlct ese1of discreteval ues used10branchthe
seemed arbitrary. Whi l e is was done for conveni ence a seri ous justification
n be advanced based on the conti nuous sol uti on at Node O. The reduced
discrete set is chosen to i ncl ude a l i mi 1ed nei ghborhood/ wi ndowof the
conti nuous 1 on.
2. The exhausti veenumerati on basedon disc teval ues was easi ertoeval uatefor
a purediscreteprobl em. Appl yi ng KT condi ti ons toobtai nthepossi b]ep tial
sol uti ons to econti nuous rel axati on and choosi ngtheacceptabl esol uti onis
an i nvol ved exercise. Usi ng numeri cal l echni ques Ii ke SQP 10 sol ve the
probl em whi ch wil1 mosl Ii kel y be done in practi ce has its own probl ems.
Dependence on good startingval ues wi noguaranteeof convergence requi res
user i nl erventi on 10identify eparticular solution. It woul d requir alot of
effort 10automate thepro 5Ssoas 10appl y eBranch and Bound al gori thm
10general nonl i near mi xed opti mi zati onprobl ems. Li near mi xed opti mi zati on
probl ems do not possess is limitation at least to thesame extent.
In is exampl e eacti ve partial sol uti ons at the root and in first tier were all
feasible. What woul d be thestrategyif theroot nod or other nodes have ani nfeasi bl e
acti ve partial sol uti on? In case e88 searchprocess must compl ete all acti ve
partial sol uti ons and not onl y feasibleacti vepartial sol uti ons. That is a modi fi cati on
at canbe easilymade tothe8ranch and 80und Al gori thmabove.
Ot her Sear ch Techni ques: l e two search techni ques in is secti on n nel y
exhausti ve enumer tionand branch and bound provi de an addi ti onal feature of
di scoveri nggl obal opt i mumsolutions. 'hey al sobel ongtothec1ass of methods used
to address gl obal opt i mum. be sy availability of l arge and faster computi ng
resources has brought renewed emphasi s to gl obal l y opti mal desi gns. Si muUl ted
anneal i ng and geneti c al gori thms ar among the l eadi ng candi dates for gl obal
opti mi tion. ley ecoveredinthenext chapter and canal sobe characteri zedas
enumer i on chni queslike ones in is section. 80th of themcanand areused
for discreteopti mi zati on. They 'enot di scussedinthis chapter. The appl i cati onof
most of thesetechni ques is still heuristic and oftendepends on experi encebasedon
numeri cal experi ments among c1asss of probl ems. Some appl i cati ons are probl em
specificand requi reuser i nterventi onand l earni ng. Standardi mpl ementati ons in ese
methods. especi al l yfor nonl i near probl ems arenol yet thenorm.
8.2.3 Dynami c Pr ogr amml ng
Di scretedynami c programmi ng (DDP) is anexci ti ngtechni que for handl i nga special
cass of probl ems. Exampl es 8.1 d8.2arenot direct members of this cass. chard
8el l man[4) was responsi bl efor ti n duci ng econcept andthealgori m. It is an
op zi ngprodure basedon Bel l mansprincipleof optimality. The principleis based
on a sequen of deci si ons on partial solutions. so at when ccrtainvariables have been
determi ned. theremai ni ngvariables establishanopti mumcompl eti onof theprobl em[5).
Ano er expressi onof this ncipleis ob i nedfromReference6. An opti mal sequen
of deci si ons h epropertythat at ec rent timc. whatever thc initial stateand e
previ ous decisions. E mai ngdeci si ons must be anopti mal set wi thregardto estate
m ngfromthefirst d sion. DDP isabout a sequence of deci si ons (ofteninti me) and
is someti mes term asequenti al dsi onprobl em. Probl ems I i keExampl es 8.1 d8.2
do not directlyfit iscl sifi tionbut they nb sformedtofit isr ui rement (5).
Very often. DDP probl erns canbe descri bedas opti mal pa determi nati onprobl em.
Such P hprobl ems aresol vedusi ngdi rectedgraphs (di graphs). leseprobl erns may
actual1y have no connecti ontophysi J paths or distances. Li ke the88 tree ycnabl e
better appreci ati onl appl i cati onof theal gori thmi c produre andarenot sictly quired.
DDPr u sdefi tionof s. tes andstages. andonl y theformer is di scussedhe. le
equati onsed toes blishtheprinciple of opti mal i ty are ))ed Ul ctiollal equati ons.
Exampl e 8.3will be usedtodefi ne enewterms.
Exampl e 8.3 An establ i sheduni versi tyis inl stedindevel opi nga newCo))gefor
I nformati on Technol ogy duri ng the currenl ye . Tabl e 8.2 ilI ustrates data
2 DI SCRETEOPTI MI ZATI ONTECHNI QUES 337
Tabl e8.2 Facul tyH1ri ngPollcyData for Exampl e 8.3
l tem Fall Wi nter Spri ng Summer
R uiredFaculty 5 10 8 2
Recrui entCost 10 10 10 10
Uni t Facu1tyCost 2 2
Uni tRe ni ngCost 2 2 2 2
necessary tocal cul al ethecost of recrui ti ngfacul tyfor is enterpri sefor thefirst ye
of oper n. The col l egewill operateduri ngfour quarters (FaU. Wi nter. Sp g. d
Summer) i ndi catedby ecol umns. Deci si ons areassoci atedwi ththebegi nni ngof the
quarters. The first rowi ndi cates thenewfacultyrequi redtoi mpl ement thecours at
will be offered. The secondrowis erecrui tment cost for newfacul tyeachquarter.
whi ch is si gni fi cant because of ecompeti ti vedmand for qual i fi edpeopl e in this
. le irdrowis eunit cost of newfacul ty interms of thestandardinstitut
facul ty cost . Course speciali on and availability makes this cost different in
different quarters. The last rowis unit cosl for retai ni ngextrafacul tyhi redin E
earlier quarter (tomi ni mi ze cost over theyear). What shoul dbe thehi ri ngpol i cyfor
uni versi l ylomi ni mi ze cosl ?
lis exampl e is c1early il1ustrative of sequenti al deci si on maki ng. Deci si ons are
requ dtobe made every qu er. For exampl e it ispossi bl e10 cruit all neededfaculty
duri ng efirst quarter andpay B ni ngcost. or it may cost less tohireduri ng ch
quarters andhireagai nin ef o . leprobl emwill be
sol vedusi ng opti maUshortest pathmethod even oughthe arenorou i nvol vedi n
. eprobl em. bebest ways tounderstand eshortestp hal gor mistodrawa di rected
graph(di graph) betw nthenodesl states intheprobl em.
S tes: States present poi ntsl condi ti ons in DP where deci si ons have to be
consi dered. Thesearesometi mes 'edtoas pol i cy deci si ons or just pol i ci . This
has a s uenti aJ connecti on as current deci si ons cannot be nsi de d UD e
previ ous deci si ons have been made. Cl earl y in Exampl e 8. 3 present e
begi nni ngof eachquarter. lenumber of states is usual l yone more thenumber
of dci si ons toi ncorporatetheeffect of thelast pol i cy/deci si on. Nodes inthedi graph
rep sent states intheDP probl em. The di graphrepresents a connecti onbetween each
pair of states at is permi ssi bl e. Thi s canbe made c1earer by stating t if a node is
goi ng to be affected by a deci si on at any previ ous nod. then there is a direct
connecti on(I inelarc) between enode. The cost of theconnecti onis i ndi catedon E
connecti onitsel f. An arrowcanbe usedtoi ndi catethedi recti onof traverseal ong is
linelarc segment. TheseIi ns/ arcs ereforerepresent decisiofls. Once thedi graphfor
theDP probl emis available. theopt i mumsol uti oncorrespofl ds to theshortest path
f r omIhebegi nni l l g10 theendi ng stateof the di graph [3).
338 DI SC ETEOPTI MI ZATI ON
99
74
Fl gure8.6 Di graphfor Exampl e 6.3.
Fi gur 8.6is di graphfor thehi ri ngpol i cy. There arefi vestates wi ththefirst
four representi ng equ wherethedeci si ons must be made. Node 1 P sents
theFall quarter and 50on. Al ong thearcs on edi graphis th C05t of thedeci 5i onor
pol i cy at wiJI ori gi nateat thecurrent node and meet a11 of therequi rements at e
l r node. Inorder 10us thei nformati oninthedi graphefficl i vel yand 10devel op
Ihemathemati cal model thequarters areassi gnedi ndi ces f rom1 to4. The fol l owi ng
abbrevi ati ons areused: requi redfacul l Y RF; crui tment COSI RC; uni t facul l y cost
FC; uni l rel ai ni ngcost - hol di ngcosl HC. The pol i cy deci si ons or cost between the
start node i and theend nodej is C(ij). For exampl e. C( l 4) rep sents epol i cy
theFal l q rterwhi ch will meet estaffi ngreq rements for eWi nl erand
Spri ngquarters a150thus payi ng eappropri ateretai ni ngpena1ties:
C(l 4) = RC( l) + [ RF(l) + RF( 2) + RF( 3) ) . FC(l )
+ ([RF(2) + RF(3)) HC( l) + [RF(3)] HC( 2) )
Ex8_3. mis them-fi l e at generates thedatafor thedi graphfor Exarnpl e 8.3 d
i denti fi es e50l uti onby sol vi ngthefuncl i onal eq l i ons and backtracki ng epa
to i denti fy the opti ma1 squence. Note at for differ nt probl ems on1y the cost
cal cul ati onwiJI differ. The functi onal equal i ons and backtracki ng el ements of the
DDP met hod and shoul dbe thesame.
Functi onal Equatl on for Dynaml c Pr ogr amml ng: The functi ona1equati onis
theessenti al part of theDDP a1gori thmor method. It empl oys a recursi vepro du
10i denti fythebesl sol uti onal eachstate. Lel Val ue (i) be th best val ueof thedeci si on
al node i. l aki ngi nl oaccount a11 possi bl earcs/ paths thal l eadtonode i. Let Node (i)
be thenode l edtothebest pol i cyat ecurrent state. The recursi onrel ati ons for
IJ states canbe sel up as fol l ows:
8.2 DI SCRETEOPTI MI ZATI ONTECHNI QUES 339
Val ue (1) = 0
Val ue (k) = mi n (Val ue (+C(/ k); 1 S 1 < k} k = 2 . . . 11+ 1
Forexampl inExampl e 8.3
VaLue (4) = mi n (Val ue (1) + C(l 4); Val ue (2) + C(2 4); Val ue (3) + C( 3 4)}
Val ue (4) = mi n ([0+85]; [15+ 62]; [45+ 26]) = 71
Node (4) = 3 (whi ch establ i shdthemi ni mumval ue)
Theseca1cul ati ons areembedded inEx8_3. m. The output f romtheprogramIS:
E xampl e 8. 3-Dynami c pr ogr ammi ng
Cost bet ween pai r s of nodes/ st at es C (i j )
o 15 45 85 99
o 0 30 62 74
o 0 0 26 34
o 0 0 0 12
0 0 0 0 0
odes - Val ues ( i )
opt i mal pat h val ues at v r ~ o
o 15 45 71 79
Best pr evi ous node o r each cur r ent node - Node( i )
0 1 2 3 3
The Val ue (4) is 71 alldNode (4) is 3. The opt i mumval ue for theprobl emis Val ue
(5) whi ch is 1Ua11079.
Shortest Path: The shortest path is obtai nedby backtracki ng f romtlle end stat
i ngtheNode (i )i nf onnati onwhi ch is ther owof dataabove.
At NodB5t hEbest nodEf r omwhi ch toreachNode 5i s Node3
o 2 3 3
AI Nod 3thebest node f romwhi ch toreachNode 3 is Node 2
o 2 3 3
At Node 2 thebest node f romwhi ch toreachNode 2 is Node 1 (this is estartmg
node)
o 1 2 3 3
Thi s sol uti onis i ndi catedby ethi cksoIi dlineinFi gure 8.6.
ne ac aI h li iri tnI gp ol i c cya .cco orld ngto issol u tio nistohi ir e sttherqu iI1 edf: acuIty
for FaJ l qu ter the FaJ l qu ter. Do thesame for the Wi nter quarter. For the
Spri ngquarter however hi for boththeSpri ngand Summer quarters.
Al ternate Pat hs: There is an al ternate sol uti on for thc same opti mal val ue. The
output f romEx8_3. mi ncl udes is i nformati onthough it was not i ncl udedabove. In
eoutput inthecommand wi ndow each r owcontai ns addi ti onal nodes at wi l l yi el d
same opti mal vaJ ue. l eonl yen y is 1 inthi rdr owand second col urnn. Thi s suggests
that e thi I1dnode Node (3) can o have the val ue of l - t hat is Node 1 al so
provi des thesame vaJ ue toreachNode 3. This wi l l cause eNode (i) i nformati onto
change as
3 3
The shortest pa C enbe i nterpretedas fol l ows:
At Node 5 thebest node f romwhi ch toreachNode 5 is Node 3
3 3
At Node 3 thebest node f romwhi ch toreachNode 3 is Node 1
o 1 3 3
I nFi gure 8.6this is i ndi catedby thethi ckdashed arcs. The hi ri ngpol i cy now woul d
be tohi refor FaU and Wi nter inFal1. Then hi refor Spri ngand Summer inSpri ng.
Dynaml c Pr o r amml ng a
The fol1owi ng al gori thmis i denti fi cdi n-Refcrencc 3 for pa s that do not cycl e. The
ori gi nal 8el1man al gori thmincJ udes an i terati ve f i nement that al l ows for cycl i c
pa s. I n Exampl e 8.3 ere are no cycl i c pa s. Thi s al gori thmis i mpl emented in
Ex8_3. m.
Step 0; Number nodes insequenti al order. Arcs ar defi nedfor j > i. Set thestart
node S opti mal val ue as
Val ue (s) =0
Step 1: Termi nate if all Val ue (k) have been cal cul ated. If p represents the last
unprocessdnode. then
Vallle(P) = mi n{ Va/ ue (i) + C (i. p): if arc (i p) exi sts 1 S i < p}
Node (p) = thenumber of Node i that achi evedthemi ni mumabove
Return toStep 1
8. 3 AOOI TI ONAL EXAMPLES 341
Thi s was a si mpl e exampl e of DDP. It is i mportant to note at sol uti oninfinal
l ue ( incJ udes on1y the best sol uti on at all states/nodes. Li ke other techni ques
the ' c several di fferent categori es of probl ems addressedby DDP. incJ udi ng speci al
di screteprobl ems liketheKnapsack probl ems. It is al so possi bl to bri ngExampl es
8.1 and 8.2under thef ramework of DDP. though toi mpart a sequenti naturl inthe
process of sel ecti ngdesi gnvari abl es inthis casemay i nvol vea to transformati onof
eprobl em. Even thenit may not be effi ci ent tosol veit usi ngDDP. Inpracti ce DDP
has al ways becn regardedas computati onal l y compl ex and prohi bi ti ve. It is sel dom
used for probl ems wi a l arge number of states or if the esti mati on of cost is
i ntensi ve. Another cri ti ci smof DDP is at it ca es out a lot of redundant
computati on for conti nuous probl ems. Today. these are less of an i ssue in the
past. For stochasti c and di screteprobl ems DDP may sti11be an effecti vetool . 8el1man
has al soi ndi catedthat it c beuscd tosol vea set of unstabl esystems of equati ons.
DDP al so compu s desi gn sensi ti vi ty as part of its sol uti on. This secti on was
pri m y meant to i ntroduceDDP as a useful tool for some categori es of probl ems.
Readers arc encouraged toconsul t ereferences for mor e i nformati onon esuect .
Inthis secti on. t wo addi ti onal exampl es arepresented. The first one is an engi neeri ng
cx npl e f rommechani cal desi gn und illustrates how the di sc te versi on of a
four-vari abl eprobl emvery effecti vel y reduces toa si ngl e-vari abl eprobl em atcan
be easi l y ad sed by enumemti on. An unbel i evabl e reducti on in compl exi ty
yi el di ng gl obal opti mi zati on is obtai ned by el ementary cal cul ati ons. It defi ni tel y
i nvi tes scruti ny regaroi ng Eort in sol vi ng conti nuous nonl i near opti mi zati on
probl ems. The second one is eappl i cati onof thebmnch and bound searchtoa 0- 1
i ntcgcr pr ogmmmi ng probl em. Much of theappl i cati onof di screteopti mi zati onis in
e 'eaof deci si onma ng. and deci si ons likefor or agai nst. yes or no arepossi bl c
usi nga 0- 1 l i near i nteger programmi ng model.
8. 3. 1 Exampl e 8. I BeamDesl gn
hi s cxampl e is a structura1desi gnprobl em. Infac it is another vari ati onl appl i cati on
of the I - bcamstructure seen rlier. The conti nuous probl cmwas devel oped and
sol vedas part of a student proj ect . In is exampl e ebeamis used as a hori zontal
member ina fi xedE" seri es gan cmne. The hoi st wi th a capaci tyof W = 2000 Ib.
is at thecenter of thebe 1. The obj ecti vei sdesi gn a mi ni mummass be nthat wi l l
not fai1under bendi ng. shcar and spcci fi eddetl ecti on. The l engthL of the beamis
speci fi cd as 96 in. Steel was choscn for thc materi al of e be n. Thc modul us of
elasticity of steel is E = 29 E+06 psi . The wei ght densi ty of s I is P = 0.284Ib/in
3

The yi cJ d s ng in onis 'y= 36 E+03 psi. The yi el dstreng inshear is y=
21 E+03 psi . esp ifiedal l owabl e detl ti ODis 0. 25in. A factor of safetyof 1.4on
thel oadis assumed. Fi gure 8.7iII us tes thecen c l oadi ngon thebeamas wel l as
thedesi gnvari abl es. (A11 cal cul ati ons areini nches and pounds. )
/
/
.:....:<<'1
x
/
/
/
DI SCRETEOpnMI ZAnON
L
342
The Dl screteProbl em: The di screte probl emshares the same mathemati cal
model except the val ues for thedesi gn vari abl s wi l l bel ong toa set of di screte
val ues. It is possi bl e to useith compl eteenumerati on or the BB search to
sol ve probl em. Bef ore proceedi ng further in this di recti on it is useful to
recogni ze the reason for di screte opti mi zati on is to choose an off-the-shel f
I - beamwhi ch wi l l keep thecost and producti onti me down. Several mi l l s provi de
i nformati on on standardrol l i ngstock ey manufacture. Ref erence [7] is used to
col l ect data for this exampl e. lf a standard S- beamis needed e reference lists
several beams wi th acal val ues for thedesi gn vari abl es XI X2X3X4Choosi ng
an I - beamf romthelist al soi denti fi es all of thedesi gnvari abl es. Thi s becomes a
si ngl e-vari abl e probl em the vari abl e bei ng the parti cul ar i temf romthe list of
beams. I nthis case thereareno geometri cal constrai nts necessary and nei ther are
esi deconstrai nts. A compl ete enumerati on can be perf ormed on thecandi dal e
beams f rom stocklist and thebest beamcan easi l ybe i denti fi ed. Compar e this
enumerati on to sol vi ngthe nonl i near mathemati cal model wi thscven functi onal
constrai nts. When such an opportuni ty ari ses expl oi ti ngthedi scretc opti mi zati on
probl cmis hi ghl y rccommendcd. On theother hand consi der a di fferent probl em
where a pl asti c I - beamwi1I be mol ded in the manuf acturi ng faci1i ty. Hcre the
conti nuous probl emneeds tobe sol vedfor themi ni mumcost.
Ex8_ 4. mwiIl perf orma compl ete enumerati on of this exampl e. It i ncl udes a
truncated stock list f romwhi ch thefmal sol uti on is obtai ned. The programmi ng is
strai ghtforward. The output intheCommand wi ndowis
X: =O 1" =48. 239 x; =0. 1563; X; =2. ; =5 28;
Opt i mumBeami s : W6X9
Mass of t he Beam 71. 543 (l b)
Dept h Wi dt h Web Thi ckness Fl ange Thi ckness
5. 900 3. 940 0. 170 0. 215
Const r ai nt s: Def l ect i on Nor mal
( <= 0)
- 2422048221. 600
Sol ut i on f ound
( i nches)
St r ess Shear St r ess
- 86086. 261
Note that esol uti onis si gni fi cantl y hi gher antheone obtai nedintheconti nuous
rel axati on once agai n illus ting the conti nuous rel axati on provi des a l ower
bound on thesol uti on.
v
-
M
Y
'til
ZX4
(normal slress)
1
1
1
1
(defl tion)
F
The mathemati cal model for this probl emcan be exp ssedas:
Mi ni mi ze
Subj ect to:
Desi gnvariables: Exampl e8.4.
f. i . X) =pLA .
gl (X): WU - 0
g2(X): Wl x
l
-
g3(X): W - 21X3yS0
Fi gure8. 7
(shear s 5S)
Some geomctri c constrai ms torelatc thevari ous desi gnvari abl es ar
g4(X): xl - 3X2::;; 0
g5(X): 2x
2
- xl ::;; 0
g6(X): x3- 1.5x4 0
g7(X): 0. 5X4- x3 0
The si deconstrai nts on edesi gnvari abl es are
0. 25::;X
4
::; 1.25 0.125 ::;x
3
S 0. 75
2::;x
2
::;15 3::;x
1
S20
- 2468680. 570
The fol l owi ngrel ati ons def me ecross-secti onal a a(Ac) moment of inertia and
first moment of areaabout thecen id(Qc):
Ac =2x
r 4
+X1X) - 2X
4
1 = _ (x
2
- X
4
)(X
1
- 2x3)3
= - - - - =- - - ' - - "' - - - - - - : :
12 12
Zer neI nteger Pr ogr amml ng
Many L P programs/model s have speci al features whi ch areusual l yexpl oi tedtbrough
speci al al gori thms. An i mportant cl ass of LP probl ems requi re evari abl es be bi nary
that is ycan have one of two val ues. Theseprobl ems can be set up so val ues
8.3.2
Qc
Thc sol uti ontotheconti nuous rel axati onmodel is
:
DI SC ETEOPTI MI ZATl ON 344
Capi 1Budgeti ngExampl e Tabl e8.3
VaJ ue Year3 Year Yearl Group Item
A
u
q
J
A
U
Z
J
h
u
e
J

&
3
'
a
n
u
n
u
-
E
n
u
'
A
J
U
-
E
3
n
U
4
4
3
2
1
2
5
0

Systems
Mi crosystems
Software
Model i ng
M Is
Photoni cs
a
T
Z
J
Z
O
3
X" X6[ Oor 1]
-
4
0

x

+

a
x

+
3

3
Z
+
5
b
+
+

8 10 Budget
i = 1 2. . . m
u g

--J
X

a

n
T
'
M
M
F

Mi ni mi ze
Subj ect to:
Note is is not a mi ni mi zati onprobl emas requi dby thestandardf ormat .
A ori thms: .erearesevera1al go ms . 1have beendevel opedfor sol vi ngZI P
model s. They all usei mpl i ci tJ selectiveenumerati on roughBranch andBound s h
together wi th conti nuous relaxation. However the bi nary nature of e variables is
expl oi ted when compl eti ng the partial sol uti ons or versi ng the trl . le Bal as
Zero- One Addi ti veAl gori t hml Methodis one of commend i ques for s
c1ass of probl ems [6]. The al gori thmfrom samereferenceis inc1udedbel ow.
Xj = 0 or 1 j = 1 2 . . . n
Except for the bi nary sid constrai nts on e desi gn vari abl es is represents e
mathemati cal model for an LP probl em(the standard model [1 res equal i ty
cons intsonly). Most al gori thms assume M positive. If any cj is negati ve then
1 is sllbstitlltedfor xJ in eprobl em.
Several varieties of opti mi zati onJ deci si onprobl ems i nvol vebi naryvari abl es. They
inc1ude theknapsack (apure I LP wi tha si ngl emai n constrai nt) capital bl l dgeti ng(a
mul ti di mensi onal knapsack probl em) assembl y line bal anci ng matchi ng set
coveri ngandfacilityl ocati on and soon. Some exampl es aredetai l edbel ow.
for e vari abl es are zero or one- - defi ni ng a Zero- One Integer Programmi ng
Probl em(ZIP). The general f ormof a ZI P is
f(X): :2
}=1
Exampl e 0 ' a Knapsack Probl em: Mi ni mi ze e nl l mber of penni es (X1)
ni ckel s (x21 di mes (X3) and quarters (X4) toprovi decorrect change for b cents:
X1+x2 +x3 +X4
X1 + 5X2+ I Ox3+ 25x4 = b
'emtegers
Mi ni mi ze
Sl l bj ect:
X" X2X3X4
In eabove probl em theunknown vari abl es arenot bi nary vari abl es but ei ntegers
none eless.
A orl m: Bal as Zero- One Addi ti ve (A8. 4)
The enumerati onis ackedby two vectors U aitdX. Both of l engthn. lf a partial
sol uti onconsi sts of 1. . sassi gnedinorder theval ues of X correspondi ng
to eassi gnedvari abl sareOor 1. 'he remai ni ngv iabl e 'eeand have thevallle
-1. The componenof U are
I i f k =J a nditscom pIlemn Ithasno t bcnco n ere
II l l k=:k = 1 or 0 and its pJ em h eenconsi d d
o I i f k>s
Step 1. If b :2: 0 opti mal sol onis X = [0]. Otherwi s U= [0] X = [ J] .f. = 0 0
X. = X (i ncumbent sol uti on)
Step2. Calclllate
Exampl e 0 ' Capi tal Budgeti ng: le Col l ege of Engi neeri ng is consi deri ng
devel opi ng a uni que Ph. D. prograrn in Mi crosystems in the next thrl years. Thi s
expansi oncallsfor si gni fi cant i nvestment inresources and personnel. Si x i ndependent
focus areas have been i denti fi edfor consi derati on a11of whi ch c not be devel oped.
Tabl e 8.3pI 1CSents theyearl ycost associ atedwi thcreati ngthevari ous grOllpSover the
ey rs. lelast linein etablerepresents theavai l abl ebl l dget (inmi l l i ons of
dollars) per year allotted to this devel opment. le last col umn is e vallle (in
$100000) thegroupis expectedtogenerateinthenext 5 years.
The capital budgeti ngprobl emllses sixbi naryvari abl es ( Xh . X6) one for each
i temin etable. The ZI P probl emcanbe fonul atedas
J is theset of assi gnedvari abl es
y=bj - : 2 a
20X1 + 3X2 + 30X3 + 5X4 + I Oxs + 5X6
3X1 + 2x2 +x3 + 2x4 + 5xs S 10
Maxi mi ze
Subj ect to:
346 OI SCRETE OPTI M ATI ON
y = mi n Yi' i = 1 2 . . m
I =L
C
fj
I f y O d 1<1' then I =J and X=X Go toStep6
0 erwi seconti nue
Step3. Createa subset T of - eevari abl es defi nedas
T=i j : f + <1aij < 0 for i such at y/ <O)
1f T = o (empty set) thenGo toStep6 EI seconti nue
Step4. Infeasi bi l i tytest:
If i suchthat Yi < 0 and
Yi - L mi n(O aij) < 0
j GT
Go toStep6 EI seconti nue
Step5. BaI as branchi ngtest: For each freevari abl e createtheset
=(i: Yj - aij < 0)
If aI I sets Mj areempt y Go toStep6. Otherwi se CaI cul atefor ee
vari abl e
= -ai j )
i eM}
where =0 i ftheset is empty. Add tothecurrent parti aI sol uti onthe
V i abl eXj at maxi mi zes GotoStep2
Step 6. Modi U by setti ng the si gn of theri gh ost posi ti ve component . AlI
el ements totheri ght areO. Ret umtoStep2
If there are no posi ti ve el ements in U e enumerati on is compl ete. The
; _ _ . dI
opti maI sol uti onis inthei ncumbent vector X-. The obj ecti veis in.r. Jf.r is
thereis no feasi bl esol uti on.
Bal as. mis em-fi l e i mpl ements theBaI as al gori thm. It was transl atedf roma
PascaI programavai l abl einReference 6. It is a programand prompts theuser for the
probl emi nf ormati on- n mA b c. letest probl emgi veninReference 6 is i ncl uded
bel ow.
Exampk8. 5
Mi ni mi ze f (X): l Ox. + 14x2+ 2l x3+42x
4
A
l ~
~ ~
8. 3 AOOl Tl ONAL EXAMPLES 347
T
i
-
-
v
M
W
/
P
Subj ectto: - 8x 1 - l l X2- 9X3- 18x4S - 12
- 2x. - 2x
2
7x3- 14x4S- 14
-9x. - 6x2 - 3X3 - 6x4 s- l O
X.. X2' X3. X4E [0 1)
M
m
r
d
l
i
5
U
H
h
Theoutput omBal as. m:
Sol ut i on exi st s

- - - - - - -
The var i ab1es ar e:
1 0 0 1
The obj ect i ve f unct i on 52. 00
const r ai nt s - Va1ue and Ri ght hand si de
- 26 - 12
- 16 - 14
- 15 - 10
This corresponds tothesol uti oni denti fi edinReference 6.
Compl et e Enumer at l on: The Bal as aI gori thmperforms ~ ~ enumerati on. The
compl ete or exhausti veenumerati on (n.) for n bi nary vari abl es is gi venby therel ati on
ne=2n
Typi cal l yerIteEprisedeci si onmaki ng i nvol ves several hundred vari abl es and l i near
model s. On the0 erh dengi n ri ngprobl ems characteri zedby a nonl i near model
may have about 20 bi nary vari abl es mqui ri ng ne=l O48576enumBEaI i ons- Thi s i s-no
bi ' deal wi th desktop resources today. KEep in mi nd compl te enumeratl on
avoi ds any rel axati onsol uti on(purel ydi screteprobl ems onl y).
3
i
-
i
j
i
i

CaI l Si mpl ex. m: is a functi onm-fi l ethat c beused tosol veL P programs. It
c m beusdt oobt ai n sol uti ons toLP ml axati onmodel s duri ngtheBB SEarch.I t is
basedon theaI gori thmavai l abl einReference 6. It is transl atedf roma Pascal program.
lemathemati caI model is
Mi ni mi ze c
T
x
Subj ect to: [A] ] = [b]
x ~ o
Sl ack vari abl es are incJ uded in ecoeffi ci ent matri x [A]. Greater or equal to
constr nts c be set up wi th negati ve sl ack vari abl es as the programwill
automati cal l y execute a dual -phase si mpl ex method. Another al temati veis to e
L P programs provi ded by MATLAB. The functi on stal ement for usi ng Cal l
Sl mpl ex. mis
f unct i on [ xsol f] Cal l Si mpl ex( ABCMN)
To use thefuncti on. A. B. C. M. N have tobe defi ned. M is numbcr of constrai nts
and N is thenumber of desi gnv iables.
REFERENCES
1. Fletcher. R. Pracl i cal Mel hods ol Op' i mi zal i on Wi l ey NewYork 1987.
2. Papa nbrosP. Y. and Wi l de J. D. P l Ciples 01 p l i ~ Desi gn- Model i ng and
ConUlalion. Cambri dge Uni versi tyPress New York 1988.
3. Rardi n R. L. Opl i mi l al i oninOperal i ons Researcl J Prcnl i ce-Hal l Engl ewood CI fs
NJ 1998.
4. Bel l man R. Dynami c Progranun gPri ncetonUni versiLy Press Pri nceton NJ 1957.
5. Whi te D. J. Dynami c Programmi ng Hol den- Day SanFranci sco 1969.
6. Sysl o M. M. Deo N. and Kowal i k J. S. Dl screl e Opl i mi zati on Al gorl l hms
nti ce-Hal l Engl ewood Cliffs NJ . 1983.
7. RyersonSl ockSleel Lisl. J osephT. Ryerson Inc. 1995.
PROBL EMS
8.1 Transl ate Exampl e 1.1 i nto a di screte opti mi zati on probl emand sol ve by
exhausti ve enumerati on.
8.2 Transl ate Exampl e 1. 2 10 a di screte opti mi zati on probl emand sol ve by
8.3 Transl ate Exampl e 1. 3 i nto a di screte opti rni zati on probl emand sol ve by
8. 4 I mpl ement a mul ti stage exhausti ve enumerati on for thc conti nuous
opti mi ti onof Exampl e 1.1.
8.5 Transl a Exampl e1.1i ntoa di screteopti rni zati onprobl emand sol veby branch
dbound method.
8. 6 Transl ateExampl e 1.2i ntoa di screteopti mi ti onprobl emand sol veby branch
and bound method.
8.7 Transl ateExampl e 1.3i ntoa di screteopti mi zati onprobl emand sol veby branch
and bound method.
8.8 I denti fyappropri atestk avai l abi l i tyfor thefl agpol eprobl eminChapter 4 and
sol vethecorrespondi ng di screteopti mi zati onprobl em.
8. 9 Sol ve Exampl e 1.3usi ng stocki nformati onavai l abl einEx8_ 4. m.
PROBLEMS 349
8. 10 You aretheowner of an automobi l e deal ershi pfor Brand X cars. Set up a DP
for monthl y i nventory control based on cars expected 10be sold. hi ghorderi ng
costs. uni t costs for thecar. and i nventory costs for excess cars. Sol ve it usi ng
thecode inEx8_3. m.
8. 11 Sol ve thc fol l owi ngusi ngBal as al gori thmusi ng compl etc cnumerati on
Mi ni rni ze f ( X) = XI + X2 + X3
Subj ect 3 x I + 4X2 + 5x3 8
2x
1
+ 5x2+ 3X3 3
X. X2. X3 [ 0 1]
8. 12 Progr 1an exhausti veenumerati on techni quc for zero- one I P usi ngMATLAB.
Sol ve Probl em8.10.
8. 13 Is it possi bl etosol vethecapi tal budgcti ng ex npl e? If so what is thesol uti on
usi ngcompl ete enumerati on?
8. 14 Sol ve capi tal budgeti ng exampl e usi ngBal as program.
8. 15 Create an exampl e whi ch i nvol ves maki ng bi nary choi ccs regardi ng your
parti ci pati on incourse and extracurri cul ar activitics 10 maxi mi ze your grades.
Sol ve it .
9
GL OBAL OPTI MI ZATI ON
Thi s chapter i ntroduces two of themost popul ar techni ques for fi ndi ngthebest val ue
for the0ecti vefuncti on al so referredtoas thegI obal opti mumsolution. They are
Si mul ated Anneal i ng (SA) and the Geneti c Al gori thm(GA). The names are not
acci denta1si ncethei deas behi ndthemarederi ved o mnatural processes. SA is based
on cool i ng of metal s to ob in defi ned crysta1line structures based on mi ni mum
potenti a1energy. GA is basedon thecombi nati onand recombi nati onof egenes in
thebi ol ogi ca1 systeml eadi ng to i mproved DNA sequences or speci es selection. In
actual appl i cati on they ar more like a conti nuati on of Chapter 8 (Di screte
Opti mi zati on) because these methods can be used for conti nuous and di screte
categori es of theopti mi zati onprobl ems. They sharean i mportant characteri sti c wi th
other opti mi zati on techni ques in ey are pri m ily search techni ques-they
identify e opt i mumby searchi ng the desi gn space for the solution. Unl i ke e
gradi ent-basedmethods thesetechni ques arel argel yheuristic genera11yi nvol vel arge
ounts of comput onand i nvol vesome statistical rea50ni ng. Gl oba1opti mi zati on
is bei ngpursuedseri ousl ytoday at cti nga si gni fi cant amount of newresearchto e
areaof appl i edopti mi zati onafter a l onghiatus. A si gni fi cant drawback tomany of
thesetechni quesi s eyrequi reempi ri cal tuni ngbasedon theclass of probl ems
bei ng i nvesti gated. There is also no sy way to detecmi ne techni quel
probl em-sensi ti veparmeters toi mpl ement automati copti mi zati ontechni que.
The gl obal opti mumcannot reall y be characteri zedbei ngdifferent f romthel ocal
opti mum. The latter is referencedwhen onl y a part of thedesi gnspace(as op sedto
ecompl e space) is vi ewed inthenei ghborhoodof a parti cul ar desi gnpoi nt . The
standard Kuhn- Tucker condi ti ons establ i shed and used in earlier chapters (for
conti nuous probl ems) wi l l i denti fy e loca1 opt i mumonl y. Thi s is a1so more
emphati c innumeri ca1i nvesti gati ons of conti nuous probl ems where theal gori thms are
expectedtomove towardtheopmumval ues cl osetowhere th:yarestarted(i nitial
350
9.1 PAOBLEMDEFI NI TI ON 351
Whi leit makes sensetol ook for thegl oba1opti mum i5no method to
LEn6ff v J char
ultt
c in uo1u 1sc. ' 01 mv. Fex; Em roo bl ems gu aran aE l Iso lu Iti is di iffi fic
c
toa E tt ha tSl a d representson l yaminis c u
r : P i o np b l em s not n ces s ly c o n v Therefore i nt helit t e r
bothofthesemethods areexp ct ed sol uti ons c10se opti mum
sol uti on rather thegl oba1sol uti on emsel ves-' I11i sisnotto suggestthat i nvesti gati on
of thegl ob ol uuOni s a wa of eif thepossibilityof one notb lished
; 222221;ti::;fr;;LP222 JZ;21
0pti IIl i zaGon techni ques are si mpl e to i mpl ement whi l e mquut ng exhausti VE
cal cul ati ons.Thi s canpotenudl y soak up all of eidleti m onthepersona1desktop
computi ng resource le ngit medi atea acti veness.
23e techni ques and their ascussI on axeVEry basI Cfol l owi ng E emeespoused
in apter 8. e der is encouraged to it j ournal publ i cati ons moni tor
ma aU 1IEzzE odf hniq us. l n is c hapte r st s o m e i s s sues s r e g i ng g 10

u o n ' ep nt f011 1
'h fol l owi ngdcvcl opmentor-gl obal opti mi zadonisbas do n unconstrai nedconti nuous
op zation. Sol vi ngcons i nEdpmbl emsdGemotappeartohaveevol vedt oacommon
tec d is not illus .ted in this book. Note at cons i ned probl ems can be
convertedto anuncons i nedprobl emusi ng a penal ty formul ati on(Il ap 7) le
probl emcanbe defi nedas
Mi ni mi ze f [X); [X)n
Subj ectto:
(9.1)
(9.2)
sprobl emwere (onl y one um) eop oncoul d ly be
tabl i shedby any of theai f Opriat techni quesusedinthebk si ncetheloca1mi ni mum
is also egl obal mi ni mum. To vesti ga thegl obal mi n nit is thereforeexpected
ul dbem yl o ninirr 15several of nhavi ngd rent va1 r
objective. least among themisthen egl obal solution. Loca1mi ni mums in ese
of conti nuous probl ems are ose sa fy eKT condi ti ons dwill be numeri cal l y
determi j f estarting s w in enei bo oodof p mi ni mum. Thi s
definitionmay be appl i edtodiscreeop zati onprobl ems that us conti nuousrelaxation
o b in esoMUon- For di scmteprobl ems usi ngcompl e enumerati onthe gl obal
ooti mi muon Shoul dhdi scoverEd automati cal l y.FIT1eEEfGEe-the di scussi on applies to
di scretep l ems thatu onl yi ncompl eteenumerati on.
9.1.1 Gl obal Mi ni mum
The gl obal rni ni mumfor the exists if efuncti onis conti nuous on a
nonempty feasibleset S at is cl osedand bounded [1]
352 GLOBAL opnMI ZATI ON
leset S defi nes a regi onof edesi gnspace is feasi bl e. Any poi nt inS is a
candi date desi gn poi nt X. In the n-di mensi onal Cartesi an spa the poi nt X
co ponds toa vector of l cngthIIXII. The boundari es of theregi onarepart of thest
S (c1ose . le l ength of the v tor X shoul d be a finite number (bounded). 1
defi ni ti onfor thegl obal mi ni mumX. can be stated
f(X. ) f(X); X e S (9.3)
The condi ti ons for theexi stenceof thegl obal mi ni mumas gi venabove areattri buted
toWei erstrass. Whi l e egl obal mi ni mumis guar t dif thecondi ti ons aresati sfi cd
ey do not negate eexi stenceof thegl obal sol uti onif condi ti ons arenot satisfied.
Consi deri ng thebroad range d typeof opti mi zati onprobl ems thecondi ti ons ar
fairly l i mi tedin use l ness. The condi ti ons for i al m umar e esame
as constrai nt (9.3) except theset S is l i mi tedtoa smal l regi onaround X. .
Convexl t y: If an opti mi zati onprobl emcan be shown be convex then el ocal
mi ni mumwill al sobe a gl obal mi ni mum. A convex opti mi zati onprobl emrequi r t hc
obj ccti ve functi onbe convex. If constrai nts arepresent thenal1of i nequal i ty
constrai nts must be convex whi l e the ual i tyconstrai nts must be linear.
ConvexSet : l I s usual l y rel ates to desi gn vector (or desi gn poi nts) of e
probl em. Consi der t wo desi gnpoi nts X. and X
2
As before theset S defi nes a desi gn
regi on and by i mpl i cati onbo X. and X
2
bel ong toS. The linej oi ni ngX. and X
2
contai ns a whol e seri es of newdesi gnpoi nts. If al1of thesepoi nts al sobcl ong toS
enS is a convex set. A cI osed set S r l ui s that thepoi nts X
1
dX
2
can al sobe on
theboundary. I f however. thelinej oi ni ngX. and X
2
contai ns some poi nts that do not
bel ongtoS. enS is not a convex set . For a two-di mensi onal desi gnprobl em. convex
dnonconvex sets arc i l l ustratedinFi gure 9. 1.
Convex Funct / ons: A convex functi on is defi ned onl y for a convex set. Once
agai nlet S be a convex set f romwhi ch t wo desi gnpoi nts X. and X
2
arc sel ccted.
Let f be thefuncti onwhose convexi ty is tobe cstabl i shed. Consi der any poi nt X"
on a l i nej oi ni ngX. dX
2
. The functi onfi s convex ifj {XJ is less va1ueof
ecorrespondi ng poi nt on l i nej oi ni ngftX. ) andf t Xu. l nn di mensi ons is is
not easy tose but a one-di mensi onal representati onis prented inFi gure 9.2. le
condi ti onfor convexi ty in efi gureis
f(x ) S r (9.4)
A mathemati cal condi ti onof convexi ty is that th Hessi anma x (matri x of second
deri vati vcs) of thef uncti onf musl bc posi ti vesemi dcfi ni tc at al 1poi nts intheset S.
Note thal this corresponds to e K T condi ti ons for unconstrai ned mi ni mi zuti on

d
s NonconvexSe.
X2J
I X1
Xl
gure9.1 Convexandnonconvex se!.
3
4
k
u
o
h
y
i
J
probl ems. Fr oma practi cal vi ewpoi nt. theei genval ues of functi onmust be greater
anor equal tozeroat al 1poi nts inS.
lt is i mportant toreal i ze agl obal opt i mummay sti1l exi st even if theconvexi l y
condi ti ons arenot met. l nreal opti mi zati onprobl ems. it may be difficult 10estubl i sh
convexi ty. Inpracti ce. convexi ty checks arerarel yperformed. An assumpti on is made
agl obal opt i mumexi sts for theprobl em. is a subj ecti veel ement forced
by experi ence. i nsi ght. and intuition. Thi s is al sooftenj usti fi edby epossi bi l i ty
a better sol uti onthan ecurrent one can be found.
x
X] X
a
X2
Fl gure9.2 Convex functl n.
354 GLOBAL opnMI ZAnON
9. 1. 2 Nat ure of the Sol utl on
It is assumed egl obal opti mumsol uti onwiIl be obtai nednumeri a11y. A good
understandi ngof thenatureof thesol uti onis possi bl ewi threferenc aconti nuo
desi gnexampl e. Exnsi on todi re probl ems is possi bl eby consi deri ngthat any
conti nuous sol uti onco ' esponds toa speci fi ccombi nati onof theset of disc teval ues
for thedesi gnvari abl es. Fi gure9.3is an i1I us nof a fi gurewi thscveral mi ni ma
and maxi ma wi th a s onggl obal mi ni mumat x = 4 X2 = 4. le fig eavai l abl e
roughf l g9_3. m1 is theplot of efuncti on(aI soExampl e 9.1)
sin(0.1+ ( X. - 4) l +( X2- 4) (9.5)
f(x. Xz) =- 20
.. ~ 0.1 + 1- 4) z+( x
2
- 4)Z
Fi gure9.4iIl ustrates econtour pl ot The contour pl ot canbe bet r app ci atedin
col orusi ng col orbar for interpretation. Runni ng them- fiI e will createbothfigures.
Fr omrelation(9.5) it is cI ear theripples wiIl subsi di zeasXI and x2 move much farthcr
away f romwhere the gl obal mi ni mumoccurs be aus of th i nfl ucnce of the
denomi nator. The numerator isconstrai ncdtoIi c between -1 and + 1 mul ti pl i edby thc
Several ml nl m ndmaxl ma wlthglobal solullon
5
5
0

q
H
-15
- 20
10
.t::
x
z
-10 -10
Fi gure9.3 Surfaceplot of Exampl 9.1.
Ipiles10be downl oaded mthewebsi!ear indicnllldbyboldfacesnnsserif typc.
91 PROBLEMOEFI N ION 355
Several ml ni ma and maxl ma withglobal solution- contour plot

1
4
1
5
2
2
F
5
J
l
~ ~ ~ ~ 0 2 4 6
x val ues
Fl gure9.4 Conl our plot of Exampl e9.1.
scal i ngfactor of 20. Inthelimit theri ppl es shoul dstart tobecome a plane. This can
be confi rmcdby executi ngf l g9_3. mwi thdifferent ranges for variables.
Fr omFi gure9.3it is easy tospot gl obaI sol uti onby observati on. Surroundi ng
e gl obaI mi ni mumare ri ngs of 10 1extr ma. Numeri cal l y it is a chaI l engc 10
negoti ate a1l of the bumps and dips. A si mpl e way to charactc .e numeri cal
techni Ques for a mi ni mi zati onprobl emis that eyattempt tofinda di tch(or val l ey)
tofaI l into. If this is as anunconstrai ncdmi ni mumprobl em theK1 condi ti ons
wi l l l eadthesol uti ontobe ppcd yoneof el ocaI mi ni mumri ngs dependi ng
on thel ocati onof cstartingpoi nt for thciterations. lf theiterations ' CstartedcI ose
toX( = 4 x2 = 4 enit is more I i kel y c gl obal opt i mumwil1be di scovered.
Stari ngatxl = - 5 X2 = - 5 usi ngstandardnumeri cal t hni ques will probabl y not l ead
to gl obaI mi ni mum.
Thi s is a si mpl e illustration but not Gt hemareMani tBl ocal mi ni mums and
m ums daEi ngl egl obal mmm. I n i gnprobl ems wi morc 2
V iablesit is not readi l yapparent howmany locaI mi ni mums exist and if egl obaI
mi ni mumis rcally that distinct even if it is postul at to cxist. What about the
compl cte enumerati ontechni que? Thcreisno way thcgl obaI mi ni mumcanescapethc
356 GLOBAL OPT1MI ZATI ON
siftingdue to is techni que. There is a1sono techni que at is si mpl er compl ete
enumerati on. le probl emis Ihe ti me to finishsearchi ng Ihe infinitepoi nts of Ihe
desi gnspacetodi scover Ihe solution.
The sol uti on to gl obal opti mummust Iherefore lie betweenstandard numeri cal
techni ques andcompl ete enumerati on. The former will not findIhe sol uti onunl ess it
is fortunate. The 1atter carmot find Ihe sol uti on in ti me to be useful. Gl obal
op zati on chni ques tendtofavor thelatter approachbecauseof esi mpl i ci tyof
i mp1ementati on. To be eful l hesol uti onmust be di scoveredina reasonabl etime.
9.1.3 El ement s of a Numerl cal Technl que
leprob1ems facedby a numeri cal techni quesearchi ngfor Ihe gl obal mi ni mumhave
b ntouchedon in eprevi ous section. Inmost standardnumeri cal techni ques E
loca1rni ni mumserves as a strongma et or a beaconfor Ihe solution. Inmost cases
desi gnchanges arenot acceptedif Ihereis no i mprovement inthesolution. If standard
numcri cal techni ques have tobe adaptedtol ook for gl obal rni ni mum IhenIhey must
encourage sol uti ons ati n eshort runmay i ncreaseIhe obj ecti veval ueso ey
canfindanol her val l eyat whose bottomis egl obal mi ni mum. Theymust achi eve
Ihis wi l hout sacrificing signific tpotency of Ihe local rni ni mum. becausea local
mi ni mumsol uti onis better thanno sol uti onat a11. Fi gure9.5illustrates Iheseideas
usi nga singlevariable.
InFi gu 9. 5 eiterations sta at poi nt A. Ina standardn neri 1techni que. a
S hdir tion is foundwhi ch leads to poi nt B. At B. if nventi onal numeri cal
techni ques were used a searchdirectionl eadi ngtopoi nt C will be accepted whi l ea
searchdirectionl eadi ngtopoi nt D will be strongl ydi scouraged. But if poi nt D was
somehowreachabl e enit cou1dmove on toE and enF and conti nuetoIhe g1oba1
f(x)
x
F1gure9. 6 Strategyfor global opti mumsearch.
3
2
9. 2 NUMEAI CAL TECHNI QUES AND ADDI TI ONAL EXAMPLES 357

mi ni mum. The mai n i deainfi ndi ngthegl obal mi ni mumis toencouragesolutions at
will escape the al ction of Ihe local mi ni mum. Thi s means al poi nt E it is quite
possi bl e esearchdi recti onmay 1eadtopoi nt G. 11is enr l ui red at a 1arge
number of iterations bepermi ttedso at Ihe va11eyho1di ngIhe g1oba1mi ni mumwi1l
berevisited. From isbrief i1l ustrationIhe characteristics for a techni que edtofind
a gl obal mi ni mumcanbesummari zed as:
Short-termi ncreases intheobj ecti vefuncti onmust be encouraged.
Tradi ti onal gradi ent-basedtechni ques will not be particu1ar1y useful bcause
eyarebasedon searchdir ctions at d eIhe obj ecti vefuncti onat Ihe
current itcration. Theywi11not pcrmi t incr of Ihe obj ecti vefuncti onvalue.
If descent-based search directions are not part of Ihe techni que any
direction decre es eobj ecti veshouI d be acceptedtopromote attraction
by Ihe 10cal rni ni mum.
As l ocati onof Ihe gl obal mi ni mumis not known searchmust be permi tted
over a 1argcregi onwi l hno bias and for a 1argenumber of iterations.
Si nce acceptabl e di recti ons can decrease or i ncrease obj ve functi on
val ue a hcuristicdetermi nati onof S hdi recti on(or desi gnchange) must
bep t ofthe chni que.
These features resemb1e Ihe zero-order techni que for unconstrai ned
op zati on d Ihercfore will rcqui re a large number of iterations to be
effective. lelargenumber of iterations wi1l forceIhe techni que10be si mp1e.
T he sol uti on can onl y be concl uded based on Ihe number of iterations as
necessary condi ti ons (availab1e for conti nuous probl ems onl y) cannot
di sti ngui shbetweenloca1and gl obal opti mum.
Wi l hout a doubt Ihe best techni que is compl ete or exbausti ve enumerati on. It is
Ihereforenecessarytoassume that compl ete enumerati onis Il Ot all opti ol l infurther
di scussi on.
The prob1emdescri pti on illus nanddi sc sionhave dealt wi l huncons ai ned
rni ni mi zati on. Cons aints canbe i ncl udedby addressi ngIhe above characteristics to
appl y tofeasiblesol uti ons only. It is 1eft to Ihe reader tocont mpl ate eability to
he stica1lygeneratea largenumber of feasibledesi gnchanges for consi derati on.
9.2 NUMERI CAL TECHNI QUES AND ADDI TI ONAL EXAMPLES
Thissecti oni ntroduces two popul ar mel hods for gl obal optirnization. Bol h arederi ved
fromnaturaI physi cal process at hel p identify opti mumstates. le first one
Si mul ated Anneal i ng (SA). refers a physi cal process at establishes differcnt
crysta1line structures each of whi ch can be associ ated wi different mi ni mum
potential energy. Thi s is attemptedby reheati ngandcool i ngof materi al s/meta1s
whi l econtrol l i ng raleof cool i ng. lesecond Ihe Geneti c Al gori l hm(GA) works
throughmutal i onof candi datesol uti ons unti l l hebest sol uti oncanbe esl abl i shed a
358 GLOBAL OPTI Ml ZATI ON
process at can be rel ated to natur1 sol uti on or bi ol ogi cal evol uti on. lese
techni ques areiII ustratedwi threspect tounconstrai nedprobl ems inthis secti on.
9.2.1 Sl mul at ed AnneaJl ng ( SA)
Thc first appl i cati onusi ngSA for opti mi zati onwas presentcdby Ki rkpatri ck et al.
[ 21. The exampl es covered probl ems f romel ectroni c systemsi n puti cul ar
component pl acement en by theneed for mi ni mumwi ri ngconnecti ons. The
al gori thmitself was based on the use of statistical mechani cs demonstratcd by
Metropol i s et al. [3] toestabl i shthermal cqui l i bri umina col1ccti onof atoms. The
termitself is mor e reprcsentati veof theprocess of cool i ngmatcri al s parti cul arl y
metal s after rai si ng thei r temperature to achi eve a defi ni tc crystal l i ne statc. As
menti oned earl i er the extensi on to opti mi zati onis mai nl y hcuri sti c. Thc si mpl e
i deabehi nd S A can bEi l l ustratedthrough theI nodi aed basi c al gori thmof S A due
toBochavesky ct al. (4).
Al gorl thm: Basl c Sl mul at ed Anneal l ng (A9. 1)
Stcp 1. Choosc starti ngdesi gn"0. Cal cul atefo= J ("o)
( Necd stoppi ngcritcria)
Step2. Choos arandompoi nt on thcsurfaceof a uni t n-dIl III : Rsi onal hypcrsphere
(it is a sphere inn-di mensi ons) toestabl i sha searchdi recti onS
Step3. Usi ng a stepsi ze cal cul ate
ft =f ( X
o
+) ; Af =
Step4. If Af S 0 thenp = 1
El se p = e-fJt.f
Step5. A randomnumber r (0S r < 1) is generated.
If r S p thenthestepis acceptedand thedesi gnvector is up ted
seno ch geis made to edesi gn
Go toSt ep2
If i mpl emented al gori thmhas to executed for a reasonabl y l arge number of
i terati ons for thesol uti onto toa gl obal mi ni mum. The si mpl i ci tyof theal gori thm
is very appeal i ng. A ca refi 110 bsc rva6o n wi l l
t
pa ra a
berlel at totheBol t zmannp ro ba.bi litydistrib uti o n. Ins tatistical
mechani cs it isexp ssedas -kl T; T is consi dered canneal i ngtemperatu andk e
Bol tzmann constant. It di ctates thecondi ti onal probab at a worse sol uti oncan
beac pted. For theal gori thmtobe effecti ve it is commendedthat p be intherange
0.5S p S 0.9. Informal S A termi nol ogy repr sents eann i ngtemperature. The
foIl owi ng i deas intheal gori thmarc apparent :
V
9.2 NUMERI CAL TECHNI QUES ANO AOOt nONAL EXAMPLES 359
The al gori mis heuristic.
The al gori thmis suggesti vc of a bi asedrandomwal k.
Descent of thefuncti onis di rectl yaccommodal ed.
Dirl ti ons at i ncrease eval ueof thefuncti onaresometi mpermi tted. Thi s
is i mportant toescape l ocal opti mum. Thi s is thepotenti al todi scover th gl obal
opt i mumthrough eal gori thm.
This basi c al gori thmcan defmi tel y be i mproved. Duri ng theinitial iterations it is
necessary toencourage sol uti ons toescape l ocal opti mum. After a reasonabl y l arge
number of itcrations the opportuni ty to cscapc l ocal opt i mum( wi ndow of
opportuni ty) must be conti nuousl y rcduced sothat thc solutioll docs not wandcr all
over thepl ace and canccl thei mprovements tothesol uti on at have al ready taken
p1ace. Thi s can be i mp1emen by schedul i ng changes e p ameter
i terati ons i ncrease. InS A tenni nol ogy is is referredto eanneal i ngschedul e.
l esc modi fi cati ons do not inany way decreasetheprobl em!user dependence of the
al gori thm. Thi s probl emdependenc is usual l yunattracti vefor automati c or standard
i mpl ementati on of the SA mcthod. Rcsearchers are engagcd in constructi on of
al gori thms at do not depend on uscr-speci fi edal gori thmcontro1paramete [5J.
The SA al gori thmfor conti nuous vari abl es i mpl cmented inthebook i ncl udes many
of basi c i deas f romthe ori gi nal al gori thm. There are many addi ti onal changes
no he . First negati ve si gn in the prob ility r mis fol dedinto itscl f. A
si leprocedure is i ncl udedfor obtai ni ngtheval ue It isbased on theinitial val uc
of "0 and fo. is ca1cul a yi cl da val ueof p = 0.7for a Af ual to50%of
ini eal gori thmfavors a negati veval ueof by d i . Si ncc theprocess for
establ i shi nganinitial i1I us tedabove can detcrmi nc a po veval ue theal gori thm
corrects this occurrencc. It is latcr illustrated through numeri cal experi ments that a
smal 1magni tude of thc pararncter al l ows thesol ut i onwander all over thep1acc
whi l e a 1argeval ue provi des a s' Ong 10cal at acti on. Insteadof theSA stepbei ng
i ncorporatedwhen obj ecti vei ncreases it is tri ggeredwhen obj ecti veis 1ikely
toi ncreasc bascd on sl ope. In ea1gori thmbel ow theco 01parameter is kcpt
constant at its initial val ue. The cal cul ati on of i s borrowed f romthe tradi ti onal
one-di mensi onal stepsi zecal cul ati onusi ngtheGol den Secti onmethod (Chapter 5) if
theobj ecti veis like1y todecre e. If thesl opeof thc obj ecti veis posi ti veor zero a
fi xedi s chosen. Thi s val ueis a maxi mumof one of theprevi ous
Al gorl thm: Si mul at ed Anneal l ng 9.
Step 1. Choose "0.J ( "o) p = 0.7 = -l n(p)/0.5xJ ("o)
< l s = 1 If s > Os = l xs
360 GLOBAL OPTI MI ZATI ON
Step2. Choose a randompoi nt on thesurfaceof a uni t n-di mensi onal hypersphe
toestabl i sha searchdi recti onS
Step3. Cal cul ateVj {Xo);
If Vj{X
o
) < 0 thencomput e ID.
AX= S; =Xo+AX; X
o i + 1
If Vj {X
o
) 0
4f =f ( + f () ; a =m ( 1)
p =
A r andomnumber r; (0S r < 1) is gencrated.
If r S p en estepis accep dand edesi gnvector is updated
AX=aS; =Xo+d. X; i i + 1
El se conti nue i i + 1
00 toSt ep2
The al gori thmis avai l abl c in Si mul at edAnneal l ng. m. It is cal l ed through
Cal l _SA. m. The al gori thmis appl i edtoExampl e 9.1 whi ch was usedfor iIl ustrati on
inSccti on9.1. To executc theal gori thmthefol l owi ngm-fi l es (i ntroducedinChapter
6) are ncccssary: Upper Bound_nVar Gol dSect i on_nVar . m. Fol l owi ng the
practi ceof drawi ng contours for tW( vari abl eprobl crns t wo files arenecessary for
descri bi ng .eexampl e the second pri m ily to qui ckl y drawa contour pl ot usi ng
ma ix cal cul ati ons. In this case the files are Exampl e9_1. m and
Exampl e9_1 pl ot. m. The latter filename must be set inSi mul at edAnneal i ng. m.
The cal l i ngstatement and vari abl es arecxpl ai nedinSl mul at edAnneal l ng. m. Th y
can be readby typi ng
>> hel p Si mul at edAnneal i ng
inthecommand wi ndow(thecurrent di rectorymust contai nthefiles)
The ader is strongl y encouraged torun theexampl e 00 hi s computer. It woul d be
a si ngul ar coi nci dence if any of the sults presented here are dupl i cated. Runni ng
numeri cal experi ments issi mpl y a mattcr of calIi ng theS A al gori thmand taki nga l ook
at therul ts (especi al l y e pl ot). The desi gnchanges are i l l ustrated usi ngcol ors:
red- represen g d asi ng sl opes- and bl ue- f or posi ti ve sl opes t l ead to
desi gn changes. The n pment in the l ocal mi ni mumand the es pearc1earl y
iIl ustratedon thecontour plots. The contour pl ots efilledcol or pl ots toi denti fyareas
on the fi gure where the functi onhas a l ocal mi ni mumor maxi mum. Darker col ors
present mi ni mums and l i ghter col ors themaxi mum. The exampl es were run usual l y
for 1000 iterations. It is concei vabl e they coul dhave becn stoppedearlier. On a
200 Hz PII l aptop it requi red less a mi nute for each run. A l arge number of
i terati ons is useful toi ndi cateif thesol uti onescapes thegl obal mi ni mumonce havi ng
obtai nedit . Whi leit was not cvi dcnt in is cx npl eit is l i kel y probl emdcpendenl
si nce in is cxampl e I he gl obal mi ni mumwas si gni fi cantl y l arger an thc l ocal
J
7
P
9
1
2

9.2 NUMERI CAL TECHNI QUES ANDADDI TI ONAL EXAMPLES 361

"
mi ni murns. Several features of theS A al gori thmareiIl ustratedusi ngsome numeri cal
expenments.
Exampl e 9.1 Thi s is thefuncti onusedinSecl i on9.1. The gl obal mi ni mumis at X=
[44] wi th an 0 tive val uej " = -19. 6683. In e fi gur thecontours suggest a
stronggl obal mi ni mumenci rcl edby al tematel ocal maxi mums and mi ni mums. For
most of theruns estarl i ngval uewas far f romthegl obal mi ni mum.
si n(0.1+ (XI - 4) :L + (-'1. - 4)
f(x. x
2
) = - 20
l' - 0.1+ (x
1
- 4) :L + (x
2
- 4) l
(9.5)
Mi ni mi ze
-pd
3
F
-
3
2
q
p
H
K
Z
5
2
5
1
2
2
2
2
Fi ve numeri cal experi ments arepresentedbel ow. It shoul dbe emph i zedthat ese
runs cannot be dupl i cated. Overal l theal gori thmworked very well. For exactl y
same starti ng val ues and number of i terati ons (cal l i ng Cal l _SA. mwi thout any
changes- about 40 s) egl obal mi ni mumwas f ound 95%of theti me.
Run 1: Starti ngdesi gn= [-5 -5]; s = 2. 7567; Number of i terati ons = 1000
Useful number of i tcrati ons ( when dcsi gnchanges occurred) = 383
Fi nal desi gn= [3. 9992 3. 9997]; Fi nal Obj ecti ve = 19. 6683
Fi gure 9.6i1l ustrates thetraceof thedesi gnchang . It escapes thestarti ngrel ati ve
mi ni mumfairlyeasi l ytoget .ppedby a di fferent l ocal mi ni mum. It is abl etoescape
to the gl obal mi ni mumwhere it is strongl y attrac d to it. be .ces in the l ocal
mi ni mumappear toprefer tostay inthel ocal mi ni mumsuggesti ngthesol uti ons are
not bei ngencouraged 10escape.
Run 2: Starti ngdesi gn= [44]; 0. 0362; Number of i terati ons = 1000
Useful number of i terati ons ( when desi gnchangsoccurred) = 991
Fi nal desi gn= [100. 72 127.75]; Fi nal Obj ecti ve = 0.128
Fi gure 9.7iII u ates thc traceof esearchfor thegl obal mi ni mum. Most of thepoi nts
arenot inthepl ot as can be seenf romthesol uti on. Thi s sol uti onwas startedat the
gl obal mi ni mum. The searchprocess app rs towander all over thepl aceand finally
off map. There is no patteminth l i mi tedpoi nts secnon the fi gure except e
suggesti on at the search is encouraged t ocape. This can be ticd to the l ow
magni tude of s.
Run 3: Starti ngdesi gn= [44]; s = -1; Number of i terati ons = 10
Useful number of i terati ons ( when desi gnchanges occurred) = 553
Fi nal desi gn= [3. 8832 4. 3709]; Fi nal Obj ecti ve = 19.173
Fi gure 9.8ilI ustra thetraceof thedesi gnsearch. Inthis runtheval ueof s was sel
to-1. Once agai nthestarti ngval ue of thedesi gnwas thegl obal mi ni mumitself. Fr om
thefig eit is c1ear th esearchis not al l owed toescape egl obal mi ni mumthat
easily. The choi ce of thecontrol parameter was arbi yand several ex npl es have
tobe studi ed10seeif is val ueis probl emi ndependent . The val ueof I is typi cal l y a
363 9.2 NUMERI CAL TECHNI QUESANDADOmONAL EXAMPLES
2
-
U
A
I
/
5
GLOBAL OPTIMIZAT10N
10
a
B
4
2
u
g
a
g
N
M

362
o 2
lu
- 2
x
'
- 4 - 6 - 8
10 B s 4 o 2
values
-2
X
1
4 - 6
Si mul atedanneal i ng owp.
Di scussi on on the Numeri cal Experi ments: 'he fol l owi ng observati ons are
basedon theexperi enceof appl yi ngSA (A9.2) toExampl e 9. 1. They correspondto
ereportedexperi encewi thSA in egeneral literatu .
F1gure9.7
Sl mul atedanneaJ ingExampl e9.1-convergedsolution.
good choi cefor opti mi zati ona1gori msp ularlyfor a weU-sca1edprobl em. What
is clear f romtheruns sofar is that a negati veva1ueof s of theorder of 1 i mpl ements
a loca1at ctor.
Fl gure9.6
Anneal i ngschedul e(changi ngs) is i mportant consi derati oninSA. Starti ng
wi tha l owva1ue econtrol p me r s must bei ncreascd-not toofast or it
will get pped aloca1mi ni mum.
Anneal i ng schedul is devel oped through tri a1 and error insight is not
obvi ous and basedon probabl ebehavi or of thes h.
Lengthof ti me(or number of iterations) that eprobl emneeds tobesearchcd
is difficult to dtermi ne. Whi l e is can be based on moni tori ng gi ven the
stochasti c naturc of thesearch it cannot be assertedthat egl obal mi ni mumhas
beenestab1i shed. The SA al gori thmitself canbe expectedtoconvergewhi l ethis
propertyis toosl owfor practical use[6]. A larger number of iterations is more
likelytoyi el dthegl oba1solution.
SA searchdoes not subscri beto iterativei mprovement. It i5anexampl e of a
stochasti c evol uti onary process. The i mpl ementati on uscd here P seots a
memoryl ess se .
Run 4: Starti ngdesi gn= 5-5]; s = -2. 7567; Number of iterations = 1000
Useful number of iteratioos (when desi gnchanges occurred) = 558
Fi nal desi gn= [-3. 3397 6.2019]; Fi na1Obj ecti ve= -2. 5674
This is a repeat of Run 1: Fi 9.9traces thesearchfor thegl oba1mi ni mumfor this
run. It appears at 1000 iterations is not sufficieot todetermi oethegl obal mi ni mum.
Whi l e thesol uti onhas escapedthestartingloca1mi ni mum it has not yet escaped e
next local mi ni mum. Si ncetheva11eys (loca1mi ni mums) inthisexampl e arecircular
note at thesol uti onmoves aroundintheva11ey but has not managed tocape.
Run 5: Starti ngdesi gn= [-5 -5]; s = -2. 7567; Number of iterations = 10000
Useful number of iterations (when desi gnchanges occurred) = 3813
Fi nal desi gn= [4.0037 3.9996]; Fi nal Obj ecti v = 19. 6682
No figureis presented. The sol uti onconverges tothegl obal mi ni mum.
"
3
t
365 9.2 NUMERI CAL TECHNI QUES ANDADDI TI ONAL EXAMPLES
g
a
E
'
N
H
2
4
0
4
b
p
q
d
N
J
S
t
A
q
d
f
d
GLOBAL OPTl MI ZATI ON 364
g
a
E
'
M
10 B e 4 - 2 0 2
- val S
- 4 - 6 - 8 10 8 4 - 2 0 2
X -val ues
- 4 - 8 - 8
Dl screte Probl ems: In di scrcte probl ems the S A al gori thmwill use a di screte
desi gnvector det uned stochasti cal l y o mthe avai1abl e set Each el ement of e
desi gnvector is randoml y chosen f rom epermi ssi bl eset of val ues. The remai ni ng
el ements of th S Aal gori thmaredi rectl y appl i cabl e. If eobj ecvedecrccs then
the desi gn change is accepted. lf it i ncreascs then a condi ti onal probabi l i ty of
acccptance is determi ned al l owi ngthesol uti ontoescape thc mi ni mum. Thi s is lcft to
ader. The si gni fi cant effort is thei denti fi cati onof erandomdesi gnvector.
Exampl e 9. 2 Thi s is a functi onof t wo vari abl es that MATLAB uses for iIl ustrati ngthe
graphi cs capabi l i tyof software. Typi ng peaks at theCommand prompt wi l l draw
the surface pl ot. Typi ng pe peaks. mat the Command prompt wi l l provi de e
uati ons that were uscd to obtai n the curve. Thi s functi onused as eobj ecti vc is
avai l abl e in Exampl e9_2. mand Exampl e9_2pl ot. m. S A is used~ i denti fy the
gl obal mi ni mum. Fi gure 9. 10illustrates thecontour pl ot a10ng wi changes in e
desi gn. The functi on has t wo l ocal mi ni mums and thr local maxi mums. The
fol l owi ngreprcsents a converged run.
Sl mul al edanneal i ng: non onvergence- l nsulficienlllerallons. Fl gure9. 9
Sl mul al edanneal l ng: Increasl ng- decreas dopp unitytoescape. Fl gure9. 8
Rest ti ngf romdi ffercnt d si gnmust stiUbe i mpl emented toovercome strong
l ocal i nfl uences.
Restarti ng f romthe same initial de is a1so i mportant for establ i shi ng
convcrgence prope es as is is a stochasti c searchprocess.
Const rsl ned Pr obl ems: This chaptcr does not deal wi th constrai nl s or di scrctc
probl cms. l erc arc t wo ways of handl i ngconstrai nts di rcctl y
1. lf thenewdesi gnpoi nt is not feasi bl e it is not acceptedand no dcsi gn change
tak pl ace. This is qui te1 cti vc for r andoms hdi rccti ons.
2. lefuncti on at dri vcs thea1gori thmduri ngeach i ternti onis theone wi ththe
l argcst vi ol ati on. If thecurrent poi nt is feasi bl e thentheobj ecti vefuncti onwi l l
dri ve search. This make sense e process is l argel y stochasti c and is
. expected towander around edesi gnspace.
Starti ng i gn= [- 2 2]; 2.5; Numbcr of i terati ons = 1000
Useful number of i terati ons ( when desi go . changcs occurrcd) = 338
Cons ncd probl cms c be expected to take a si gni fi cantl y Iarger numbcr of
S rches. The is currentl yno pr' uretoi denti fyconvergencc prope of t hcSA
wi th sp t toconstrai nedopti mi zati on.
366 GLOBAL OPTl MI ZATl ON
- 4 - 3 - 2 -1 0 2 3 4 5
X
1
values
Fl gure9.10 Sfmul atedanneal i ng: Exampl e 9.2.
Fi nal desi gn= [0. 02279 -1. 6259]; Fi nal Obj ecti ve = - 6. 5511
Thi s is thegl obal mi ni mumof theprobl em.
Therewas a si gni fi cant change made to eS A al gori thmbased on initial stswi
eex uti on. At thestarti ngdesi gn thedesi gnwas l argel yposi ti ve(8.95). Changi ng
it toa negati venumber di dnot permi t th sol uti ontoescape thel ocol rni ni mumor to
swi ng oround the nearby l ocal maxi mum. Numeri cal experi menl s suggest e
constant f 3be mai ntai ned between
2.5sf 3S- 1 (9.6)
Otherwi se no change was made to eal gori thm.
9. 2. 2 Genet i c Al gor i t hm( GA)
leGeneti c Al gori thmis an i mportant part of 0 newareaof appl i edresearchtermed
Evol uti onary Computati on. Thcse areactual l y searchprocses and natural l y useful
9.2 NUMERI CAL TECHNI QUES ANDAOOI TI ONAL EXAMPLES 367
for di scoveri ngopt i mumsol uti ons. Hol l and [7] was thefirst tousc thetechni que but
i use as an opti mi zati ontool began ineamest in elatc 19808 devel oped moment um
inthemi d- 1990s and conti nuto attract seri ous i nterest to y. The termevol uti onary
is suggesti veof enatural process associ atedwi bi ol ogi cal evol uti on-pri mari l y
theOarwi ni an rul of thesel ecti onof thefittest. This takes pl acebecause econstant
mutati onand recombi nati on of thechromosomes in epopul ati onyi el da better gene
StructUI . A11 of these terms appe in the di scussi ons rel ated to evol uti onary
computati on. Thi s basi c pro ss is al sofol dedi nto evari ous numeri cal techni ques.
The di scussi ons can become cl ear if termchromosomes can be associ ated wi th
desi gn vari abl es. Over the y Evol uti onary Computati on has i ncl uded Geneti c
Pr ogr ammi ng- whi ch devel ops prograrns rni rni ck e evo1uti onary proc ;
Geneti c Al gori thms opti rni zati onof combi natori aVdi screteprob1ems; Evol uti o ry
Programmi ngopt i mi zi ng conti nuous functi ons wi thout recombi n on;
Evol uti onary Strategi es-opti rni zi ngconti nuous functi ons wi th recombi nati on [8].
This c1assi fi cati onis qui tel oose. For exampl e GAs have been used and conti nue to
be used effecti vel y to sol ve conti nuous opti mi zati on prob1ems. Evoh onary
al gori thms have been successful l y appl i edtoopti mi zati onprob1cms inseveral areas:
engi necri ng desi gn pararneter fitting knapsack probl ems transportati onprob1ems
i mage processi ng travel i ngsal esman schedul i ng. and soon. As can be infe f rom
elist. a si gni fi cant number are o mthc arcaof di scrctc programmi ng.
Thi s secti on and is book i ncl ude onl y GAs though al1of the evol uti onary
al gori thms share many common features. GAs generate and use stochasti c
i nformati on in thei r i mpl ementati on and therefore can be consi dered gl obal
opti mi zati ontechni ques. They areexcepti onal1y useful for handl i ng ill behaved
di sconti nuous. and nondi fferenti abl eprobl ems. Unl i ke theS A convergence tothe
gl oba1 opt i mumin GAs can onl y be weak1y establ i shed. GAs generate and use
popul ati on of sol uti ons- desi gn vectors ( X) - un1i ke S A where a search
di recti on (8) was generated. This di sti ncti on is essenti al to understand the
di ffi cul ty of handl i ng conti nuous probl ems in OAs. For exampl e. usi ngr andom
numbers tog neratea val ue for Xl o theval ues of 3. 143. 141 or 3. 1415 wi l l al1be
di sti nct and have no rel ati on to each other.
Gener l c Genet l c Al gorl t hm( GGA) (A9. 3)
Step1: Set up an initial popul ati onP( O) - an initi a1set of sol uti onor chromosomes
Eval uate ini 1sol uti on for fi tness-di rentiate. col l ate and rate
sol uti ons
Generati oni ndex I = 0
Step2: Use genel i c: operal ors togeneratetheset of chi l dren(crossover mutati on)
368 GL08AL OPTI MI ZATI ON
Add a newset of randoml y generatedpopul ati on(i mmi gral l ts)
Reeval uate popul ati on-f i tness
Pcrf ormcompeti ti ve sel ecti on - whi ch mcmber s wil1 be part of next
generati on
Sel ectpop ati onP(t + l ) - same number of member s
If not converged tt + 1
GoToSt ep2
The vari ous terms areexpl ai nedbel ow.
Chr omosomes: Whi l e it is appropri ate to regard each chr omosome as a desi
vector thereis i ssueof representati onof thedesi gnvector for handl i ngby geneti c
oprators. It is possi bl e to wor k wi th desi gn vector di rectl y (as in e i ncl uded
exampl e) or use some ki nd of mappi ng t l 1 (real encodi ng) or bi nary (bi nary
encodi ng). Earl i er wor k inGA used bi nary encodi ng. In is book thedesi gnvector
is used di rect1y for a11 transacti ons. Real cncodi ng is commended. A pi ece of the
chr omosome is cal l edan allele.
Fi tness and Se/ ecti on: The fi tness functi on can be rel ated to thc obj ecti ve
ncti on. be popul ati on can be rankedl sorted accordi ng to e obj ecti ve functi on
val ue. A sel ecti onf romthis popul ati onis necess y for (1) i denti fyi ngparents and (2)
i denti i ngi f eywi11be pr omo dt o enext generati on. There arevari ous sel ecti on
schemes. A fracti on of the best i ndi vi dua1s can be chosen for reproducti on and
promoti on theremai ni ng popul ati onbei ng made up of newi mmi grants. This isca11d
tournament sel ecti on. A modi fi cati on of is procedure is to assi gn a probabi l i ty of
sel ecti on for orderedset of thepopul ati onfor bo reproducti onand promoti on.
This probabi l i ty can be based on roulettewheel elitist selection linear or geometri c
rankillg [10]. The i mpl ementati on in this book uses ranki ng wi thout assi gni ng a
probabi l i ty of sel tion. 1t uses theobj ecti vefuncti onval uetorank sol uti ons.
Inlt I Popul ati on: The paramet r here is the number of initi a1 desi gn vectors
(chromosomes) for estarti nggenerati on N
p
' number is usual l y kept thesame
i nsuccessi ve generati ons. Usual l y a nor ma1r andomgenerati onof thedesi gnvectors
is recommended. In some i nstances dopi ng addi ng some good sol uti ons to e
popul ati on is performed. It is di ffi cul t to establ i sh how many member s must be
consi deredtorepresent thepopul ati oninevery generati on.
Eval uati on: The vari ous desi gnv tors have to be eva1uatedto s if ey can be
used for crea g chi l dren at can become part of the next generati on. Fi tness
attri butes can serv wel l for this pu ose. For thecaseof unconstrai nedopt i mum the
obj ecti ve functi onJ(X) is a good measure. Si nce gl obal mi ni mumis unknown a
progressi vemeasure may be used.
l ~ m ~
9. 2 NUMERI CAL TECHNI QUES AND ADDI TI ONAl EXAMPLES 369
Geneti c Operators: l ese provi ways of defi ni ng new popul ati ons f rom
exi sti ngones. l I sis al sowhere theevol uti ontakes pl ace. Two types of operati onare
recogni zed. The first is crossover or recombi nati on and thesecond mutati on.
Crossover Operators: There are several types of crossover strategies. A si mpl e
crossover is ilI trated in Fi gure 9.11. Here a pi ece of e chr omosome- an
a11el e- i s exchanged between t wo p ents P" P
2
to produce chi l dren C" C
2
The
l eng of thepi ece is usual l y determi ned randoml y. Mathemati cal l y if theparents e
X = [x" X2 . . Xn] and Y (defi nedsi mi l arl y) and r is therandoml y sel ectedtruncati on
i ndex. enthechi l drenU ( =C 1) and V ( =C
2
) by si mpl e crossover aredefi nedas [11]
x
.
r
t
1
1
r
1
4
l
l
L
-
-
u
v

i fi <r
o erwi s
if i < r
otherwi se
(9.7)
Ari thmeti c crossover is obtai nedby a Ii near combi nati on of t wo parent chromosomes
yi el d t wo chi l dren [9]. If Il.
1
and ^'z are randoml y generated numbers then the
chi l drencan be defi ned
y
y

h
h

+
+

x
x

h
L
.

=
q
q

If
(9.8)
1; i ' 0 it is a convexI i nati on
Pt
Parents
P2 Ct C2
Chi l drcn
gure9
If thereareno restrictions on ' s it is anq necombi nati on. For real ' s it is a l i near
combi nati on. An i mportant crossover operator is a di recti on-based crossover
operator if JtY) is less thanJtX) and r is a unit randomnumber:
c= r ( Y - X) + Y
Mutati on: Mutati onrefers to erepl acement of a singleel ement of a desi gnvector
by a randoml y genemted value. The el ment is alsorandoml y chosen. For desi gn
vector X for a randoml y sel ectedel ement k
c=JX
j
i *k
IX
k
i =k
There are oth r types of mutati on i ncl udi ng nonuni formmutati on and directional
mutati on. The reader is referredtoReference9 for a rel evant description.
I mm. 'fants: Thi s is eaddi ti onof a set 01 ' l' andoml y generatedpopul ati onineach
generati onbeforetheselectionis made for enext generati on. The searchprocess in
GA is broadl y classified pl orati on and expl oi tati on. Expl orati on i mpl i es
searchi ngacross theentiredesi gnspace. Expl oi tati onis focusi ngmore in thearea
promi si nga solution. Typi cal l y crossover and mutati onareways tonarrowthesearch
to anareaof desi gnspa basedon echaracteristics of thecurrent popul ati on-
suggesti nga local mi ni mum. To keep esearchopen for thegl obal mI nI mum other
areas must bepresentedfor expl orati on.l i s Is achI cvedby bri ngi nginanunbI ased
popul ati onandhavi ngthemcompete for promoti onto enext generati onal ongwi
thec nt set . Inmany vari ati ons theseareal l owedtobreedwI th ecurrent good
sol uti ons beforeeval uati on.
The ideas presentcdherearebasi c toall of evol uti onarycomputati on dspecific
toGA. They arealsosi mpl ebut compl ete. For eachcategorylistedabove thereare
more sophi sti catedmodel s at arebei ngresearchedanddevel opedand ereader is
encouragedtoseek i nformati onfromthecurrent literature. Thc specificGA algorithrn
i mpl emented in book i ncorporates all of e el ements above in a si mpl e
i mpl emen on.
Genet l c Al gorl t hm(A9. 4) The al gori thmactua11yi mpl emented isgi venbel ow. In
line wi th many other al gori thms avaIl abl e In thIs text It Is provI ded as a way to
understandandappreci atetheworki ng of theGA. It isa1someant toprovi dea starting
poi nt for theuser toi mpl ement his own variationof theGA si ncethesetechni ques are
sensitiveto especific probl embei ngs died. Another i mportant featuremI ssi ng
om Isi mpl ementati onisanencodi ngscheme. The desi gnvector ishandl eddirectly
toprovI dea direct fcedback.
. . .

R
H
U
-
-
l
J
u
-
-
U
F
I
F
t
i
-
-
j
i
n
-
-
J
r
9.2 NUMERI CAL TECHNI QUES AND ADDI TI ONAL EXAMPLES 371
S P 1. Set up an initial popul ati on. Ten sets of desi gn vectors are randoml y
created. Fromthis set anIniti a1generati onof two vectors is selected. Thi s is
pool avai l abl e for propagati on. Thi s is also the mI nI mumto perform
crossovers. The fitness functi onIs eobj ectIvefunctIon.
Generati onI ndext = 1
Maxi mumgenemti onI ndex = tmax
Step2. Geneti c Operati ons
Two Si mpl e crossovers
Two Ari thmeti c crossovers (convex)
One Di recti onal crossover
Mutati onof all sevendesi gnvectors sofar
Fouri mr g
Sel ecti onof thebest two desi gnvectors (from18vectors) for promoti on
to enext generati on
If 1 = 1m Stop
I ncrement generati onindex: I t + 1
Go toStep2
Exampl e 9.3 The exampl e for issectionisa Bezi er curvefittingprobl em. Gi vena set
of data findthefifth-order Bezi er curve at w eda. leobj ecti vefunctionis
le tsquar errorbetw n eoriginal dataandthedatageneratedby theBezi ercurve.
sprobl emhas beenadclressedearlier rougha conti nuous deterrninistictechni quein
anearlier chapter. ereareehtdesi variablesassiatedwi ththeprobl em. lereare
several reasons for thechoi ceof theprobl em. Fi theglobal solution if it exists must
be closezer o d cond is provI des a way descri be eight desi gnvariables in
two-di mnsi onal space(plot). lebest val uefor thedesi gnvector everygener oni s
plotted to see howthe curve fit i mproves. Si nce encodi ng Is not bei ng used in the
al gori thm a reasonabl yl ongdesIgnvector is necessaryobserve th crossovers. le
original dataareobtai nedfromthefolI owi ng function:
Jtx) = 1 + O. 25x + 2e-
x
cos 3x (9.9)
Fi gure9.12 illustrates eseP of theexampl e i ncl udi ngtheeight desi gnvariables
whi ch are thex and y compo of thei ntemal vertices t determi ne efitted
curvecompl etel y. 'be functi onishardcoded inCaI l GA. m. There areseveral reasons
why is Is not ebest exampl e for illustration of GA. FIrst it Is exampl e in
conti nuous variables. Second local mi ni mums arenot verydi sti ngui shabl eand there
are i nnumerabl e m y of them. The gl obal mi ni mumis not substantially different
fromthelocal mi ni mumval uetow rant expl omti onof thegl oba1solution. Thi rd the
probl emIs not ill behaved. The most i mportant reasondepends not on theal gori thm
buton ecomputatI onof theobj ecti vefuncti on-ca1cul atIngtheleast squarederror.
372
2. 5
2
g
1. 5
h
0.5
GLOBAL OPTIMIZATl ON
02
x
z
0. 4
gure9.12
0. 6
Bezier curves
Orfgln Dala
- - Rlleddala
o Vertlces
0
(Xs' X
o
)
... . / 0 "
} / . }
0.8 1
x-data
12 1.4 1. 6 1.8
GenetlcAlgorithm. descriptionof Exampl e9.3.
2
It uses a Newton- Raphson techni que at is prone tofai1urc if thc curvefolds over
severel yor cvertices areveryc10setoge er. One way toovercome thisis tocause
thex-Iocati onof thev rtices tomonotoni cal l y i ncrease(whi chis i mpl emcntcdin e
popul ati ongcncrati onfunction). Thi s upsets many of thecrossover operati ons SIOCC
arenowco allyrandomin dof p lyrandom.
lefol l owi ngfiles arenccessaryfor ssp ificex npl erelatingtothehandl i ng
of Bezi cr curves: c . mFactorl al . m Combl nat l on. m Bez_SCLError. mand
areavai l abl einthecode di rcctoryfor thechapter.
The GA inAI gori thrn(A9.4) is i mpl ementedusi ngthefol l owi ngm- flI es.
CaI l GA. m
Genetl cAl gorl thm. m
popul ator. m
Sl mpl eCrossover. m
Ari metl cCros: I ver.m
Di cti onal Crossover. m
Mutatl on. m
Dr awCur ve. m
Settingup parameters anddatafor thea1gori thm
Di cts theal gori thm
Gencrates randomdesi gnvcctor
Pcrforms si mpl ecrossover betweentwo parents
Performs nvex crossovcr betw ntwo parents
Performs one dircctional crossover
Performs mutati onof edesi gnvectors
Draws thc curvefor eachdesi gnvcctor
9.2 NUME ICALTECHNI QUESANDADOI TI ONAL EXAMPLES 373

Fi gures 9.13to9.17capturc thc app1i cati onof GA toExampl e 9.3through200
generati ons. Thc reader shoul dbe abl etorunthecxampl e fromthecode di rcctory
of Chapter 9. Because of the heuristic natureof the gcncrati onof sol uti ons the
sol uti on trail will be di fferent f rom at shown bel ow. Runni ng the exampl c
several ti mes by i nvoki ngCal l GA f romtheCommand wi ndowshoul dillustrate
the GA al ways manages to cl oscl y approxi mate thc curve after 200
generati ons. The maxi mumgcnerati on va1ue is not predi ctabl e and cannot bc
estab1i shed wi th certainty. It relies on numeri ca1 experi mentati on. This number
appears casi l y to be over 1000 in many i nvesti gati ons. The reader is urged to
expcri mcnt wi ththeexampl e tocxpl orethis issue. Inrunni ngCaJl GA. mthebcst
val ues of the desi gn vector and thc correspondi ngobj ccti ve functi on val uc are
capturcdin the vari abl eXbest - an array of 201 rows and 9 col umns (assumi ng
200 generati ons). The first 8 eol umns carrythebest desi gnfor thegenerati onand
thelast col umn is thecorrespondi ngobj ecti vefuncti onval ue.
Fi g e 9.13 is e best desi gn fromthe starting generati on. It has obj ecti ve
functi onva1uc of 32.13. lis is ecor squaredover 100datapoints of theorigina1
dataand ecurvegeneratedby c desi gn variables. Thc desi gn vari abl es arc e
vertic iII ustratcdinthefigurc. The basi c curveof Fi gu 9. t3is obtai nedusi ng
2. 5
2
1.5
a
. .
>0
0.5 6
-0. 5
0
:
0.5
Fl gure9.13
1.5 2 2.5
x data
Best fltatg neratlont" 1.
374 GLOBAL OPTI MI ZAnON
9.2 NUMERI CAL TECHNI QUES AND ADDI TI ONAL EXAMPLES 375
g 0.5
=
0-"
" . . . . .. 4" " " ' . . . . . . . . . . . . . . ' + .
- 0.5
-1
ZO L
0.4 0.6 O. 1.2 1.4 1.6 1.8 2
x data
Fl gure9.14 Best f i t : generati ont=200.
35
30
2s
a
u
P
3

10

50 200 250
x data
gure9.15 Obj ecl i vefunctionfor everygenerati on.
>> Dr awCur ve( Xbest ( 11: 8) )
is pr' ure. 1t is possi bl e to scparatel y accumul ate onl y desi gns are feasi bl e
whi l e conti nui ngtoi mpl ement thestandardprocedure. Sccond usea strategyto i ve
the search process through constrai nt sati sfacti on and decreasi ng e obj ccti vc
si mul taneousl y or separatel y. Thc al gori thms i ndi catcd in this chapter can easi l y
i ncorporatc n dedch ges.
The a1gori thmi c procedure as i mpl ement ed do not real1y care if eprobl em
is conti nuous or di screte. 1t j ust works on a current set of val ues for the desi gn
vari abl es. Di screte va1ues wil1 pri mari l y requi re mi nor changes in thc way the
popul ati on is generated. A1so thc i mpl cmcntati on of the crossover and mutati on
operators wil1need tobe changed. These changes inno way detract f romthemor
i dea bchi nd the GA.
D wCur ve. mfuncti on. To drawtheiniti a1curve ( efi gureis edi tedsubsequentl y
wi thpl otedi t) intheCommand wi ndow:
Fi gure 9. 14is thebest desi gnat e fina1generati on(generati on number = 2) .
leobj ccti vefuncti onis 0. 0729. 1t is cl ear f romthefi gure G Ais attempti ng
to fi ndthesol uti on. This can be establ i shed by Iki ng at Fi gure 9.15 whi ch is e
i terati onl generati on hi story of the G A for this exampl e. The obj ccti ve functi on is
genera11ydccreasi ngthrough egenerati ons. Fi gur 9. 16is a sca1ed representati onto
di spl ay thebehavi or of theG A at latr iterations. Fr om edatait is al 80apparent thnt
is cxampl c needs torunfor mor c 200generati ons. The fina1fi gure Fi gure 9. 17
represents movcment and thecl usteri ngof theverti ces through thegenerati ons. It
is suggesti vc of l oca1refi ni ng is associ atedwi ththeGA.
Suggest ed Extensi ons: Most of the engi n ri ng desi gn probl ems i nvol vc
cons ints. Currcntl y themost popul ar way tohandl e constrai nts is toreformul ate thc
probl emusi ng pcna1ty functi ons (Chapter 7). 'here are scvcral othr al temati ves to
Cl osl ng Comment s: I n summar y for a heuri sti c techni quc e GA is very
i mp ssi ve. lC samc can be sai d for SA. Incl udi ng the fact at practi cal
engi neeri ngprobl ems must requi re partl y di sete rep sentati on e tcchni ques in
Chapte 8 d 9 arevery rel evant to epracti of desi gnand devel opment . Wi t h
the enormous computati ona1 resou e in e desktops and l aptops an effecti vc
programmi ng l anguage Ii ke MAT L AB and a wi de vari cty of al gori thms at arc
avai l abl e it is possi bl c nowto desi gn an i ndi vi dual approach to many opti mi zati on
0.5
0.45
0.4 scal edfor l owest val ues
0. 35
u
t
-
t

n
u
2
Z

- 0.5
-1
-1.5
-20 .4
0.6 0.8
0.15
0.1
0.05
REFERENCES 377
Locatl onof desl gnvarl abl es over 200generati ons
60 80 100 120 140 160 180 200
x data
2. 2 2.4
20 40 1.2 1.4 1.6 1.8
x val ueof vertex
2
Fl gure9.16 Ob tivefunctlonscal ed10showcontlnual decreaselor laler generatlon.
gure9.17 Trace01Ihedesl gnvarlables Ihroughgeneratfon.
REFERENCES
probl ems. Thisisi mportant topracti ci ngengi neers who canbri nginluitionand i nsi ght
for theparti cul ar probl em eyhave tosol veina tight ti meframe. Intheengi neeri ng
curri cul ur adi ti onal desi gn opti mi zal i on courses l end to deal onl y wi
dete stic conti nuous al gori thms. That initself was mo theti me avai l abl e
for i nstructi on. Di sc te opti mi zati on or gl obal opti mi zati on are neeessari l y skills
l medby personal endeavor. lCovercmphasi s on determi ni sti c programmi ng needs
tobe revi si tedinthelight of current devel op in earcaof opti mi zati on. T ay
computer programmi ng inengi n ri ngdepar ents and its rei nforccment is usual l y
adj uncl 10other devel opments in ecurri cul um. New programmi ng pl atforms Ii ke
MATLAB offer a way to resol vc is di sadvanl age. New al gori thms al so reflecl a
si mpl i ci tythat canbe sily sl atedand appl i ed.
is book has been about programmi ng inMATLAB l eami ng about traditional and
current techni ques in desi gn opti mi zati on and transl oti ng these opti mi zati on
i ques intoappl i .tions. lt is al soabout expl orati onand devel opi ngani nsi ght o
theprocess of desi gnopti mi zati on. The code is provi dedis onl y meant tobe a
startingpoi nt .
1. Ar ora J. S. l ntroducti on10 Opti mal Desi gn McGr aw- Hi I I NewYork 1989.
2. K'kpa ckS. Ge)att G. C. lI ndVecchi M. P. Optimizationby Si mul atedAnnea1ing
Sci ence Vol. 220 No. 4598 May 1983.
3. Me polisN. Rosenbl uth A. Rosenbl uth M. Telle A. andTeller E. Equlltions of
Sate Ca1culations byFast Computi ngMachi nes J oumal of Chemi cal Physi cs Vol . 21
pp. 1087-1092 1953.
4. Bochavesky 1. 0. J ohnson M. E" andMyron L. S. General i zedS ula dAnnea1ing
for Functi onOpti mi zati on Tecl mometri cs Vol . 28 No. 3 1986.
5. J ones A. E. W. andForbes G. W. An Adapti veSi mul atedAnnea1i ngAl gori thmfor
Gl obal Opti mi zati onover Conti nuous Vari abl es J our : 1of Gl obal Opl i mi l.Cl tion Vol. 6
pp. 1-371995.
6. Cohn H. andFielding M. Si mul atedAnneal i ng: Searchi ngfor anOpti ma1Tempe ure
Schedul e SI AMJ oumal on Opti mi zati on VoL 9 No. 3 pp. 779-782 1999.
7. Hol l and J. H. Outl i neof a Logi cal Theoryof Adapti veSystems J oumal of A CM Vol.
3pp. 297-3141962.
8. Dumi trescu D. Lazzerini B. Jai n L. C. and Dumi trcscu A. Evol uti onary
Computati on Intemati ona1Seri es on Computati ona1Intel l i gence L. C. Jain (ed CRC
Press Boca Raton FL 2000.
9. Gen M. andCheng R. Geneti c Al gori thms and Engi neeri ngOpti mi zati on Wi l ey Seri es
inEngi neeri ngDesi gnandAutomati on H. R. Parsaci (scrics cd.) Wi l cy- Intcrsci cnce
NewYork 2000.
10. Gol dbc D. Geneti c Al gori thms UI Search Opti mi zari on and Machi ne Lea mUl g
Addi son- Wesl ey R di ngMA 1989.
11. Houck C. R. J oi nes J. A. and Kay M. G. A Geneti c Al gori thmfor Functi on
Opti mi zati on p er accompanyi ng the GAOT (Geneti c Al gori thmOpti mi zati on
TIbox) No Carol i naStateUni ve ity.
PROBL EMS
No fonnal probl ems are suggested. le reader is encouraged to devel op hi s own
vari ants of t wo al gori thms i ncl udi ng f onnal ones avai l abl e in e sugges
literature. He is al so encouraged to consi der extensi ons to constrai ned and di screte
probl ems. He is r ecommended touse hi s versi ons toexpl o vari ous exampl es f ound
in the literature. I n compari son wi th edetenni ni sti c techni ques (until Chapter 8)
thes heuri sti c met hods are trul y fasci nati ng and il1ustrate consi derabl e robustness
and what is cven mor e chal l engi ng10our pcrcepti on they can sol veany probl emwi th
no reservati ons what soever. hey are i ncredi bl y basi c in constructi on si mpl e to
program and easy todepl oy. They onl y requi rea lot of ti me tosearchthedesi gnspa .
"^
F J
10
OPTI MI ZATI ONTOOLBOX
FROM MATLAB
l eMA11. ABf i lyof producl s i ncl udes theOpti mi zati onTool box whi ch is a l i brarv
of mfil . n be used to ve p l ems in aof opti mi z on. 1t con-
many of the met hods that have been expl ored in is book and several at arenot
avai l abl e in the Tool box. For os consi deri ngthepracti ce of desi gn opti mi zati on
seri ousl y theOpti mi zati onTool box is a worthy i nvestment . Many of the techni ques
and di scussi ons inthebook were kept si mpl e toil1ustrate con pts d em-fi l es
were used toprovi de MA11. ABprogrammi ng experi ence. Most of them-fi l es avai l abl e
on e w wi l l need to be d l oped further for scri ous prac appl i cati on.
Another i mportant reason for havi ng access to e Opti mi zati on Tool box is it
al l ows one 10takeadvantage of MA 11. AB' S open source pol i cy at is tohave cess
10the source files. You may be abl e to modi f y and extend those f i l esmeet your
speci al opti mi zati on qui r ments. Whi l e some of the programs f romthe Tool box
have b n used or referencedel sewhere in is book is chapter col l ects t hemina
si ngl epl ace for syreference. As can be expected the materi al of is chapter wi l l
be dated. 'he chapter is based on MA11. AB Versi on 5.3. As of is wri ti ng Versi on
6. 0f 6. 1was shi ppi ng. I n general the MATt AB productsespeci al l y theOpti mi zati on
Tool box have reached a l evel of maturi ty such t newer versi ons s ho
i nval i datethei nfonnati oninissec o n. What shoul d be exp ted however is t
themateri al in is chapter wi11most l i kel ybe upgraded enhanced and extended wi th
each new rele eof MA .
7 Q
380 OPTI MI ZATI ONTOOLBOXFROMMATI.AIl
10. 1 THE OPTl MI ZATl ON TOOL BOX
Most of the basi c i nformati on in is chapter is f rom user's gui de for the
Opti mi zati onToo1box [1]. If you arebuyi nga speci a1lybundl edCO that i ncl udes the
Opti mi zati onTool box thenyou may not have Reference 1 inbook fonn. You may
have Reference1 as a POF fileon thcCO or you may have access toit. For exnmpl e
on eWi ndows NT pl atformfor dcfaul t di rcctory insta11ation the POF versi on of
Reference 1 is avai1abl c at
C: MA TLABR 11hel ppdCdocopti mopti m_tb. pdf
Refcrence 1 provi dcs exampl cs of usc di scusses sp ia1cases a1gori ms ed d
many 0 r si gni fi cant topics. is chapter does not attempt to document them.
Li mi d dupl i caon is ed to bri ng attenti on to the most useful f l tures. be
emphi s is on i1l ustrati ngtheuseof eTool box for vari ous exampl es in du d
thebook. Infact a l nrgenumber of thetechni ques inth Tool boxarenot usedin e
book and are not covered in is section. The experi ence provi ded in this chapter
shoul dbe sufficient tounderstandand usethenewones cectivcly.
To use c programs or m.fi l cs in theTool box thre i mportant acti ons must bc
initiated. One formul nte eprobl emintheformat expcctedby MATLAB wi th pcct
to c particular functi on thnt will bc calI ed. Most1y is will i nvol v usi ng
approp teprocedures tosct theri ght.handsideof theconstrni nts tozeroor constant
va1ues. The Tool box a1so makes a poi nt of di sti ngui shi ng betwecn linear and
nonl i near constraints. Scond set/ change e paramete for e op ni znti on
a1gori thms. Thi usetheappropri ateopti mi zer and ensure ava1idsol uti onhas
bcenobtai ned. AII of seacti ons natura1lydepend on theactua1mathcmati cal model
of eprobl cm.
0.1.1 Pr ogr ams
Thc different types of opti mi zati onprobl ems at canbc sol vedwi thc versi onof
theTool box installedon computer canbe di scoveredintheCommand wi ndow
Hel p wi ndow or inthebrowser usi ngHel p Oesk. In eMATLAB Command wi ndow
type
>>hel p opt i m
The output isreproducdbel ow. They area11 of thevari ous m.fi l es intheTool box and
their opti mi zati onfuncti onis a1soidcntified.
Opt i mi zat i on Tool box.
Ver si on 2. 0 (Rl 1) 09Oct 1998
What ' s new.
Readme - New f eat ur es bug f i xes and changes i n t hi s
ver si on.
10.1 THEOPTI MI ZATI ONTOOl BOX 381
Non1i near mi ni mi zat i on of f unct i ons.
f a i a ba d- - Sc a l a r bounded nonl i near Eunct i on
mi ni mi zat i on.
mi : 22: :Mul i men const r ai ned nonl i near
mi nsear ch- Mul t i di mensi onal unconst r ai ned nonl i near
mi ni mi zat i on by Nel der - Mead di r ect sear ch met hod .
f mi aunc- Mul t i di mens i onal unconst r ai ned nonl i near
mi ni mi zat i on.
f gami nf - MI l l t i di mensi onal cons t r ai ned mi ni mi zat i on
semi - i nf i ni t e const r ai nt s.
Nonl i near mi ni mi zat i on of mul t i - obj eci ve f unct i ons.
f goa1at ? ai n- Mul t i di me i ona1 goa1 at t ai nment
opt i mi zat i on.
mi ni max Mu1t i di mensi ona1 mi ni max opt i mi zat i on.
Li near 1east squar es (of mat r i x pr ob1ems) .
1aql i n- Li nea?l eas t s quar es wi t h l i near const r ai nt s.
:: : t : : - L L n l east ar es wi t h non at i vi t y
Nonl i near l east squar es (of f unct i ons) .
?s qcuwef i t - Nonl i near cur vef i t t i ng vi a l east squar es
(wt h bo l ds) .
l sqnonl i n - Nonl i near l east squar es wi t h upper and l ower
bounds.
Non1i near zer o f i ndi ng ( equat i on sol vi ng) .
~ z r o - Scal ar nonl i near zer o f i ndi ng.
f sol ve - Non1i near syst emof equat i ons s01ve f unct i on
Sol ve.
Mi ni mi zat i on of mat r i x pr obl ems.
1i npr og - Li near pr ogr ammi ng
quadpr og - Quadr at i c pr ogr ammi ng.
Cont r ol l i ng def aul t s and opt i ons.
opt i mset Cr eat e or a1t er opt i mi zat i on OPTONS
st r uct ur e.
opt i mget - Get opt i mi zat i on par amet er s f r om OPTI ONS
st r uct ur e.
Demonst r at i ons of l ar ge- sca1e met hods.
ci r cust eat - Quadr at i c pr ogr ammi ng t of i nd Shape of a
ci r cus ent .
mol ec ul e- MOl ec ul e conf or mat i on SOI Ll t i on usi ng .
unconst r ai ned nonl i near mi ni mi zat i on.
382 OPTl MI ZATI ONTOOLBOXFROMMAn
opt debl ur mage debl ur r i ng usi ng bounded
l i near
l east - squar es.
Demonst r at i ons of medi um- scal e met hods.
opt demo - Demonst r at i on menu.
t ut demo - Tut or i al wal k- t hr ough.
bandemo - Mi ni mi zat i on of banana f unct i on.
goal demo - Goal at t ai nment .
df i l demo - Fi ni t e- pr eci si on f i l t er desi gn ( r equi r es
Si gnal Pr ocessi ng Tool box) .
dat demo - Fi t t i ng dat a t o a cur ve.
The t ut demo is recommendcd for a demonstrati onof uresof Tool
Of the above i tems opt i mset l i npr og quadpr og f mi nunc and
f mi ncon arefurther expl oredin is chapter.
A cl earer pi ctureof thecapabi l i tyof efuncti ons can be obtai nedf romthetabul ar
graphi rcpres a 1of thc P l emscts inthefirst chapl er of Ref erence 1. For
exampl e efour tcchni ques referencedin is chapter areiIl ustratedinTabl e 10. 1.
10. 1. 2 Usl ng Pr ogr ams
Eacb of thc functi ons intheTool box i mpl ements an opti mi zati os tcchl l i quecan
be edinsev al ways. 'hesearedocumented inRef erence 1 and areal soavai l abl e
onl i ne through one of the mechani sms of ob i ni ng hel p inMATLAB For exampl e
t ypMghel pmneoni n t hCommandwi ndowi dent i f i es thef ol l owi ngdi f f emtways
of usi ng functi onl program(thefol l owi ngis edi tedtoshowonl y types of usage).
FMXNCON Fi nds t he const r ai ned mi ni mumof a f unct i on of
sever al var i abl es.
FMNCON sol ves pr obl ems of t he f or m:
mi n F( X) subj ect t o:
A* X <= B Aeq* X Beq ( l i near const r ai nt s)
Tabl e 10.1 Graphi cal D cri pti onor Some Tool box Functi ons
Typ
Uncons ai ncdmi ni mi zauon
Li near programmi ng
Quadrauc programmi ng
Cons nedmi ni mi zauon
Notati on
mi n.f(x)
mi n. f
T
x suchthat
[A][x] [b]; [A] [ x] =[ b];
m T l +
[ AJ[x] [ b]; [ A.J[;x] = [ . b.ql x S u
m nin.f(x: );
[AJ[.[X] ] S [b); [A.qUx] =[ b];
funcuon
f mi nunc
f mi nsear ch
l i npr og
quadpr og
f mi ncon
10.1 THE OPTI MI ZATI ONTOOLBOX 383
C( X) <= 0 Ceq( X) 0 ( nonl i near const r ai nt s)
LB <= X <= UB
X=FMI NCON( FUNXOAB)
X=FMI NCON( FUNXOABAeqBeq)
X=FMI NCON( FUNXOABAeqBeqLBUB
X=FMI NCON( FUNXOABAeqBeqLBUBNO CON)
X=FMI NCON( FUNXOABAeqBeqLBUBNONLCONOPTONS)
X=FMI NCON( FUNXOABAeqBeqL BUBNONLCONOPTI ONS
Pl P2... )
[XFVAL] =FMI NCON( FUNXO. . .)
[XFVALEXI TFLAG] =FMI NCON( FUNXO . . . )
[XFVALEXI TFLAGOUTPUT] =FMI NCON( FUNXO. . . )
[XFVALEXTFLAGOUTPUTLAMBDA] =FMI NCON( FUNXO. . . )
(XFVALEXTFLAGOUTPUTLAMBDAGRAD] =FMI NCON ( FUNXO ...)
[X FVAL EXTFLAGOUTPUT LA DAGRADHESSI AN] =FMI NCON
FUNXO. . . ) .
There 12 ways of the ni ncon functi on. 1C di erens depend on e
i nformati on in the mathcmati cal model at is bci ng usoo and the postopti mi zati on
i nformati onthat shoul dbe di spl ayed. Inusi ngtbedi ffercnt ncti oncalls ymi ssi ngdata
at do not exist for theparticu1ar exampl es must be i ndi tOOusi nga null vector (0).
bere are eke vord the above list of functi on calls at need speci al
al tenti onincons i nOOnonl i near opti mi zati on.
EXI TFLAG: For f mi ncon is i ndi ca s if thc opti mi zati onwas successful .
I f EXI TFLAG i s:
>0 t hen FMI NCON conver ged t o a sol ut i on X.
o t hen t he maxi mumnumber of f unct i on eval uat i ons was
r eached.
< 0 t hen FMI NCON di d not conver ge t o a sol ut i on.
OUTPUT: is speci a1MATLAB structureel ement. Structureis a programmi ng
cons ct usOOinmany hi gher-I evel l anguages. It is si mi1ar toarray inMATLAB. An
array wi l l contai nonl y si mi el ements ukc all numbcrs or all strings. A s cture
on theother hand can contai ndi sparateel ements suchas severa1numbers and strings.
The el ements of e structure can be accessed by usi ng the name of the 5 C
el ement fol l owed by a peri odand CPI efi nOOkeyword for theparti cul ar el cment .
For cxampl e t wo of theel ements of theOUTPUT are:
OUTPUT. i t er at i ons: thenumber of functi oncval uati ons
OUTPUT. al gor i t hm: t henameof eal gori thmusOOtosol vetheprobl cm
LAMBDA I This is al 50 another structure in MATLAB and contai ns Lagrange
m ti puers. 1eval uesof es mul ti pl i ers areuseful for veri i ngthesati sfacti onof
e necessary condi ti ons of opti mi zati on. The val ues can be recover by esame
pr ' C i ndi catedfor thc OUTPUT structurc.
-
384 OPTI MI ZAnONTOOLBOXFROMMAn
LAMBDA . l ower : val ueof themul ti pl i ers at thel ower bound
LAMBDA. i neqnonl i n : val ue of the mul ti pl i ers tied to the i nequal i ty
cons n( s' sin ok)
Thesc i tems werc associ ated wi th the functi on ni ncon. Other functi ons may havc
di fferentl addi ti onal type of i nformati onavai l abl .
10. 1. 3 Setti ng Optl mi zati on Par amet er s
There ar several parametcrs that areusedtocontrol the appl i cati onof a parti cul ar
opti mi tion techni que. Thcse parameters are avai l abl e through thc OPTI ONS
strucl ure. AIl of t hemareprovi dedwi thdefaul t val ues. Thcre will be occasi ons when
it is ncessarytochange thedefaul t val ues. One exampl e may be at your constrai nt
tol erances arerequi redtobe l ower th thedefaul t Or EXI TFLAG recommends a
l arger number of functi oneval uati ons tosol ve E obl em. Thesep meters areset
usi ng a name/ val ue p r. Gencral l y ename is usual l y a string whi l c the val ue is
typi cal l y a numbr. The list of parameters is avai l abl e in Reference 1 and can be
accessedmost easi l y intheCommand wi ndowby typi ng
>> hel p opt i mset
A partiallist of theparameters intheOPTI ONS structure copi edf romtheCommand
wi ndow. fol l ows.
OPTI MSET PARAMETERS
Di agnost i cs pr i nt di agnost i c i nf or mat i on about t he
f unct i on t o be mi ni mi zed or sol ved [ on I {of f }l
Di f f MaxChange - Maxi mumchange i n var i abl es f or f i ni t e
di f f er ence gr adi ent s [ posi t i ve scal ar I . {l e- l } 1
Di f f Mi nChange - Mi ni mumchange i n var i abl es f or f i ni t e
di f f er ence gr adi ent s [ posi t i ve scal ar I {l e S} 1
Di spl ay - Level of di spl ay [ of f I i t er I {f i nal } ]
Gr adConst r - Gr adi ent s f or t he nonl i near const r ai nt s
def i ned by user [ on I {of f } 1
Gr adObj - Gr adi ent ( s) f or t he obj ect i ve f unct i on( s)
def i ned by user [ on I ( of f }l
Hessi an - Hessi an f or t he obj ect i ve f unct i on def i ned by
user [ on I {of f } ]
HessUpdat e - Quasi - Newt on updat i ng scheme [ {bf gs} I df p
I gi l l mur r ay I st eepdesc 1
Li neSear chType Li ne sear ch al gor i t hm choi ce
cubi cpol y I {quadcubi c} 1
MaxFunEval s - Maxi mumnumber of f unct i on eval uat i ons
al l owed [ posi t i ve i nt eger 1
Maxl t er Maxi mum number of i t er at i ons al l owed
posi t i ve i nt eger ]
10.2 EXAMPLES 385
Tol Con Ter mi nat i on t ol er ance on t he const r ai nt
vi ol at i on [ posi t i ve scal ar J
Tol Fun. Ter mi nat i on t ol er ance on t he f unct i on val ue
posi t i ve scal ar J
Tol X - Ter mi nat i on t ol er ance on X [ posi t i ve scal ar 1
The parameter names arei denti fi edinbol dfon1. Theseparameters aretheones
wi11con 1a standardopti rni zati onappl i cati on. The defaul t val ues for parame r
aregi venwi thi nthecurl ybraces. For exampl e:
>> opi ons opi mset ( Di agnost i cs' ' on' ' Di spl ay'
' i t er ' ' Maxl t er ' ' 500' ' HessUpdat e' ' df p' ' Tol Con' ' l .
Oe- OS' )
U theabove statemcnt is usedpri or toactual l ycal l i nganopti mi zati ontechni que e
defaul t val ues for thosep neters wi11be changed intheOPTI ONS structure. The
defaul t vaJ ues for a parti cul ar techni quecanbe found by typi ngtheCommand wi ndow
>> opt i ons opi mset ( f mi ncon' )
lere several othcr ways to use theopti mset programitself. For details consul t
Reference 1.
10. 2 EXAMPLES
1 exampl es i11us ated in is secti on are conti nuous opti mi zati on probl ems
di scussedin ebook. l ey are f romIi near programmi ng. quadrati c programmi ng.
unconstrai ned mi ni mi zati on. and constrai ned mi ni mi zati oo. Onl y the appropri ate
mathemati cal model isi otroducedpri or toi11us .tingthe eof theTool box functi on.
10. 2. 1 Li near Pr ogr amml ng
The L P exampl e uSed here i ncl udes all three types of constrai nts. The tool box
functi onis l i nprog.
Exampl e 10. 1
Maxi mi zc z = 4xI + 6x
2
+ 7X3 + 8x
4
Subj ectto: XI + X2 + x) + X4 = 950
X ~ 400
2x
1
+ 3X2 + 4X3 + 7X4 S 4600
3xI +4x: 5X3+ 6x4 S 5000
Xl o X2. X3. X4 ;?; 0
386 OPT1M1ZAT10NTOOLBOX FAOMMATI.AIl
lel i nprogprogramsol ves thefol l owi ngprobl em:
Mi ni mi ze f Tx; subj ect to [A](x] S [b]
The obj ecti ve functi on for the exampl c needs to b changed to a mi ni mi zati on
probl em. The greater or equal toconstrai nt needs tobe changed toa less or
cqual toonc. The l i nprogprograma1l ows equal i ty constrai nt dexpl i ci t bounds on
vari abl es. leequal i tycons ai nt can be set up wi thA andbTherearenot many
p ers at need tobe changed except swi tchi ngof the ' LargeScal c' probl emto
off.' For onl y four desi gnvari abl cs I cavi ngit on shoul d be permi ssi bl e. The f ormof
l i nprogthat wi l l be used is
[XF VALEXI TFLAG] LI NPROG( f AbAeqbeqL BUB)
Thi s f ormis not expl i ci t1y document ed inRef erence 1 or inthehel pdocumentati on.
Ex10_1. m
1
is em-ftl ethat is uscd toset up and run theprobl em. It is not di spl ayed
herebut runni ng thefilein Commandwi ndowproduces th fol l owi ngoutput
Opt i mi zat i on t er mi nat ed successf ul l y.
X
0. 0000
400. 0000
150. 0000
400. 0000
F VAL
- 6. 6500e+003
EXI TFLAG =<
1
The sol uti onto probl emis
x; = 0; X; =400; x; = 150; x; =47. " =6650
10. 2. 2 Quadr at i c Pr ogr amml ng
Quadrati c programmi ng refers to a probl cmwi th a quadrati c obj ecti ve functi onand
Ii near constrai nts. Whi l e such a probl mcanbe sol vedusi ngnonl i near opti mi zati on
t chni ques of Chapter 7. speci al qua tic programmi ng a1gori thms based on the
tech ques of Ii near programmi ng arenormal l y edas eyaresi mpl e and faster. If
theobj ecti vc functi onis a quadrati c functi oninthevari abl es thenit can be cxpressed
i nqua form. For exampl c
f ( x
1
) =x
1
2 x
2
X 2 ~
can be cxpressed
I Fi lcs 10be downl ondedfromthc websilenrc indicalcdbyboldface nsserif type.
10.2 EXAMPLES 387
) = [1 2] + 1 x2] [ i !][::J
M
f(. ) = [c]T[x] . X]T [x]
Exampl e 10. 2 Sol ve thequa atic programmi ng probl em 3
M in ni m n ni ze i ze ) = [ 1 - 2 3]T T
Subj ect x 1+2x2x 3 2.5 L 1
2x
1
- X2 + 3X3 = 1
X1. X2' x) ~ 0
- 1 11
2 l l[x]
1 41
The programtobe used f romtheTool box is quadprog. The standardformat for this
program18
Mi ni mi ze f(x) = c
T
X + x
T
Su ect to: A x S b
A x=b.
L BSx S UB
The sol uti oncan be f ound in Ex10_2. m. The defaul t opti mi zati on p :ters are
used so at the opti ons structure does not havc to be changed. The acaI cal l i ng
statement used is
[x f val EXTFLAG] quadpr og( Hc Ab AeqbeqLBUB)
The sol uti onis formattedand di spl ayedinthc Command wi ndow. 1t is copi edf rom
theCommand wi ndow:
Opt i mi zat i on t er mi nat ed successf ul l y.
Fi nal Val ues
opt i mumDesi gn Var i abl es
- - - - - -
0. 5818 1. 1182 0. 3182
Opt i mumf unct i on val ue
- 1. 9218
388 OPTI MI ZATI ONTOOLBOXFAOMMATWI

Lagr ange Mul t i pl i er s f or i nequal i t y const r ai nt
0. 0218
nequal i t y const r ai nt
2. 5000
Lagr ange Mul t i pl i er s f or equal i t y const r ai nt
0. 0164
Equal i t y const r ai nt
1. 0000
10. 2. 3 Unconst r al ned Opti mi zati on
Exampl e 10.3 is used 10 ilI ustral e usc of f mi nunc e programto be ed for
unconstrai ned mi ni mi zati on. 11 is a si mpl e curv i ng exampl c (a 101 of
unconstrai ned mi ni mi zati on probl ems in engi neeri ng are curve fitting used for
devel opi ngcorrel ati onf romempi ri cal data). The Tool box has programs speci fi cal l y
for curve fitting. It may be a useful i deafor thereader tocompare thesol uti ons usi ng
thos programs is provi des an opportuni ty touseprograms f rom eTool box not
illus tedinthis chapter.
Extunpl e 10. 3 A set of dataover 100poi nts represen ga cosi nefuncti onis ted.
A qua atic pol ynomi al is used 10 fit the data. An unconstrai ned mi ni mi zati on
probl embased on the sumof squ of the error bel ween ori gi nal data and the
pol ynomi al over all edatapoi nts is sol vedfor thecoeffi ci ents of thepol ynomi al .
is an ex npl eof thecl assi cal l east sq rede .rprobl em.
The exampl e is sol vedin m- fiI eEx1 0_3. m. Runni ng thc exampl e al sodi spl ays
a compari son of the fit on a fi gure whi ch is not i ncl uded here. Two sol uti ons are
obtai nedintheexampl e startingf romthesame initial guess. The first one is sol ved
usi ng 'er-s tpliedgradi e . The second is thedefaul t finite di fference dients.
The results ar copi edf romtheCommand wi ndow:
Fi nal Val ues Oser speci f i ed Gr adi ent s
Opt i mumDesi gn Var i abl es
- 0. 4309 - 0. 0365 1. 0033
2. 4055e- 004
Gr adi ent s of t he f unct i on
1. 0e- 007*
0. 1324 - 0. 2198 - 0. 3275
10.2 EXAMPLES 389

Fi aal val ue- - Def aul t Fi ni t e Di f f er ence Gr adi ent s

- -
0. 4309 - 0. 0365 1. 0033

.
2. 4055e- 004
Gr adi ent s of t he f unct i on

1. 0e- 004 *
- 0. 0121 - 0. 0270 0. 5715
It is general l y bel i e of gradi ents wouJ d Jead10a f i ter and
better ution. For t i nunc to gradi ents computed by changcs havc
tobc made to eopti ons s cl u wel l as thefilewh theobj ecti vefuncti onis
c i( I J 10_3m). In ons struct e' OradObj' parameter shoul dbc
sct to ' on' i nI SI SEmponmt Otherwi seMATUB domnot cxpECt thEgradi ents tobE
provi d bytheuser. The opti onfor this ex npl eis sel by
TZ; ; ? ? sTol Fun' ;? ; J : ??;;zp;;;;ff; pl ay'
bi s causes epro mtol ook for di en esecond el em oftheretum
val ue. The functI onStatement usdf or obj 103. mis
f unct i on [f gr ad] obj l O_3( x)
Bo e occti vefuncti on d egradi e (grad) arecal cul ated in the same
functi onm- fiI e.
10. 2. 4 Const rai ned Opti mi zatl on
Exampl e 10. 4 The exampl e is the same as Exampl e 8.4 and P sents lhe
constrai nedex npl efor this sccti on. ne mathemati cal model is
Mi ni mi ze f (X) = pLAc
Subj ect to: g.(X): WL
J
- 12EI S 0. 25 (dcfl tion)
g2(X): WLx. - 8/1yS 0 (normal stress)
g3(X): WQc - 21xJ 'ty S 0 (shear sl ress)
Some gcomctri c constrai nts torelutc thevari ous desi gnvari ubl es are
390 OPTI M(ZATI ONTOOLBOXFROMMAn. AB
g4(X): x -3X2 0
gs(X): 2x
2
- x ( S 0
g6(X): X3- 1.5x4S 0
g7(X): 0.5x4- x3S 0
The si deconstrai nts on thedesi gnvari abl es are
3 Sx( S20 2Sx
2
S 15 0.125 S X
3
SO. 75 0.25S X
4
S 1. 25
The foIl owi ng rel ati ons defi nethecross-secti onal area(A
c
)' moment of inertia( and
first moment of areaabout thecentroi d(Qc):
Ac = 2x2x4 + x(x Z
I x t ( - xJ ( x-
2x
i
12 12
Qc=0. 5. l +0. 5x _ XJ 2
The parame rs for probl emar w=1. 4 2000L = 96 E = 29 E+06 P = 0. 284
' y=36E+03:y= 21E+03. There arethreenonl i near i nequal i tyconstrai nts and four
l i near i nequal i ty constrai nts. le code for runni ng the prograrn is avai l abl e in
Ex10_4. m.
The programto be used is frni ncon. The fol l owi ng OPTI ON par neters are
changed.
opt i ons opt i mset ( Lar geScal e' of f ' Di spl ay'
i t er ' Tol Fun' 1. Oe 08) ;
le cal l i ng statement used for f mi ncon wi th the obj ecti ve functi on avai l abl e in
obj ecti ve. m and thenonl i near constrai nts innonl i n_cons. m is
[x f val exi t f l ag out put l ambda]
f mi ncon( obj ect i ve' xOAB LBUB nonl i n_cons'
opt i ons) ;
The Ii near i nequal i tyconstrai nts areset up inthematri x A and vector B. There are
no Ii near equal i ty constrai nts and they are i ndi cated by the nuIl matri ccs. The
l ower and upper bounds are appl i ed through the vectors L B and UB. The
customi zed part of e output di spl ayed in the Command wi ndowis i ncl uded
bel ow. Accessi ng vari ous Structure val ues wi l l provi de detai l s about the
perf ormance of al gori m.
Fi nal Val ues
-
- - - - -
6. 8671 2. 2890 0. 1802 0. 2500
.
62. 4905
Fi nal Nonl i near Const r ai nt s
- - - - -
1. 0e+009
- 2. 7304 - 0. 0025 - 0. 0001
REFERENCE 391
Lagr ange Mul t i pl i er s f or Nonl i near const r ai ns

. .
0 0 0
Lagr ange Mul t i pl i er s f or Li near const r ai nt s

- - -
2. 5879 0 0 0
The reader is encouraged to run the exampl e wi th Di agnosti cs' on expl ore
opttons structure and eoutput s ucture.
REFERENCE
1. l mmJ - R. BmEI M. A. andGrnce A. Opti mi zati on Tool box- f or Use witlr
MATL AB User's Gui de Versi on2 MathWorks Inc 1999.
l i i I i
Abstract mathemati ca1model 5 9- 10
Acceptabl esol uti ons 17
Acti veconstraint set 293
Acrospacedesi gn 1
Al gori thms21
AL M mcthod seeAugmcnl cd Lagmnge
Mul ti pl i cr method
Ana1ytica1condi ti ons 175-176
Anneal i ng8chcdul c 359
Ani fici a1var bles98
Augmented Lagmnge Mul ti pl i cr ( ALM)
m thod 276-281
algori m277- 278
appl i cati on 278-281
unconstr nedprobl em
unsform i on10 276- 277
Augmented matri x 103- 104
Bez.i er parame ccurvcs fittingof
258- 262
BFOS method seeBroydon-Fl etcher-
Ool dfarb- Shanno me d
Bi sccti ontcchni que 209- 210
Bmnch andBound
gori m333
appl i ca n333- 335
tcchni quc 329 336
Broydon- F1etchcr- Gol dfarb-Shanno
(BFOS) method
n1gori thm. 250
application. 250- 25 1
Carti anspaces. 106
I NDEX
C (I anguage) 24
Cl assof probl cms. 21
Computati ona1rcsources. 2
gateOmdi cnt mcthod. 244- 246
255
al gori thm 244- 245
appl i cati on 245- 246
Constmi nedopti mi zati onprobl ems.
265- 315389-391
Augmentcd Lagmnge Mul ti pl i er
( ALM) mcthod for sol vi ng
276- 281
beamdesi gn 310- 312
control opti ma 1 313- 315
dcfinitionof probl cm. 266
ect methods for sol vi ng 281- 302
Extcrior Pcnnl tyFuncti on PF)
method for sol vi ng 271- 276
fl agpol eprobl em 307- 310
formul ati onof probl em 266- 267
Gencrnl i zedReduced Gmdi ent ( GRG)
me dfor sol vi ng 297- 3
generi c a1gori thmfor 269- 270
indircct methods for sol vi ng 270- 281
rycondi ti ons for 267- 269
S uentia1Gradi ent R tomti on
a1gori thm( SGRA) 3 307
S ucntiaJ Li near Progmmmi ng
(SLP) gori m284- 289
S uential Quadmti c Programmi ng
(SQP) a1gori m289- 296
Cons int functi ons 6 7
Cons nts15
393
394 I NOEX
Cons n (conri nued)
ual i ty.7-8
inequality. 8
inLP prob1ems. 98
inNLPprob1ems. 157-159. 172-175.
179-181
si de.8
Conti nuous re1axalion. 321- 322
Conl i nuous vnriablcs. 20
Con 1.0p nal 313- 315
Convcrgi ng10thcsolution. 21
Convcx functi ons. 352- 353
Corners. 20
Davi don- Fl cl cher-PoweU(DFP) method.
246- 249
Dcgrees of freedom( 00 . 104
Dcscent funclion. 292
D i gn. 1. 2
Dcsi gnfuncti ons. 5- 6
Dcsi gnoeCl i vcs.5
Dcsi gnpammel crs. 5
Desi gnspace. 9
Designvariab1cs. 4- 5
Desl gnveclor. 5
Df P method. seeDavi don- Fl etcher-Powel l
me d
Di graphs. 336
Di screteopti mi z i onprob1ems (DP).
318- 348
Bmnch nndBound l echni quefor sol vi ng.
329- 336
conti nuous re1axati onapproachfor
sol vi ng. 321-322
discretedynami c programmi ng ns
I hni quefor sol vi ng. 336-341
Exhnusti veEnumerati ontechni qu for
sol vi ng. 326- 329
I ndcsi gn. 341- 343
sl nnd dapproachtosol vi ng. 322- 324
Zero- Onc Integcr Programmi ng (ZIP).
343 348
Di teprogrammi ng. 20
Di scretevarinblcs. 20
DOF (degrees of frecdom). 104
DP. seeDi scretcopti m ationprob1cms;
Dynami c progrnmmi ng
Dual probl cm. 138-148
Dynami c programmi ng( DP). 336- 341
algori m 340
functi onal U onfor. 338- 339
EBV (Enl eri ngBasi c Vari abl c). 118
Edi tor (MATLAB). 33- 37
Engi nee I g.2
Enteri ngBasi c Vari ab1e( EBV) 118
EPF mcthod seeExtcrior Penal tyFuncti on
method
Equal i tynstrai nts.7-8
Eucl i dcnnspaccs. di mesionll1. 1O
Exhausti veEnumcraLi ontechni quc. 326- 329
al gori thm. 327
appl i cal i on. 327-329
Exteri or Penal tyFuncti on(EPF) method
271- 276
al gori thm. 272-274
applicll lI on. 274- 276
Fcasi b1edomai n. 23
Feasi b1esolutions. 17
First-order co ditions (FOC). 176-178
First-orderllincar variation. 169
Fl agpo1eprob1em. 307- 310
FOC. seeFirst-order condi ti ons
Formu1ati onof prob1em seeProb1em
fonn8!ti on
FORTRAN. 24. 227
Functi on(s):
constrai nt 6. 7
obj ecti vc6
of one variuble. 162-164. 169-170
of t wov ab1cs.56- 63.164- 171
GA. seeGenctic A1gori thm
Guuss-J ordanc1i mi nati on. 107
General i zedReduecdGmdi cnt ( GRG)
med 297- 302
a1gori thm. 299- 300
upplicalion. 3 302
stepsize. ca1cu1alionof. 299
GenericGeneti c A1gori m( GGA) .
367- 370
Gcncti c A1gori l hm(GA). 350. 366- 377
gcneri c fonn. 367- 370
suggcsl edextensi ons 10. 374. 375
G10bal opti mi zati on. 350- 377
definitionofprobl cm. 351- 357
Geneti c A1gori l hm(GA) for ei n.
366- 377
mi ni mum global. 351- 353
numcri cal techni qucs for. 356- 358
Si mul utedAnneal i ng(SA). 358- 366
solution. global. 354- 356
Gol densecti onmethod. 214- 217. 219-220.
223- 225
Gmphi ca1opti mi za 16- 18. 45-92
definitionof probl em. 45- 48
nndformat for gmphi cal display. 47- 48
functi onof two variables. 56- 63
nndgraphi ca1user interface. 81-91
wi Hnndl cGni phi cs 79-81
at transfer examp1e. 73- 79
solution. lphi 1. 48-56
structural engi neeri ngdesi gnexamp1e.
64-73
Graphi cal solutions:
linear progrnmmi ng(LP) prob1ems.
07- 15
nonl i near progrnmmi ng(NLP) prob1ems.
171-175
one-di mensi onal probl cms. 205- 206
Gmphi cal User Interface(GUI ). 24. 48. 49
79. 81- 91
GRG method seeGencral i zedReduced
Gmdi ent method
GUI . seeGmphi cl l 1User Interfacc
Handl e Gmphi cs. 79-81
HnndJ cs. 49
Heat ansfer. 73- 79
In ualitycons ints.8
Infinitesol ul i ons 17
lnitial d i gnsolutions. 21
Jnl eger progr l mi ng. 20
IlI teger variab1cs. 20
llcrativetecbni qucs. 21
Kuhn-Tucker condi ti ons. 192-193.
196- 200267- 269
Lagrnngemu1tipliers. 180-192
LAMBDA pr ogmm(Op Z8ti onTI1box).
383- 384
I NOEX 395
Leavi ngBasi c Vari ab1e(LBV). 118
Li neard ndence01
Li near progmmmi ng (LP) prob1cms. 23-24.
93- 153
cons nts in 98
del ermi nants . 02- 104
dual prob1emassoci a dwi . 38- 148
cqual i tyconsb intsexnmpl ei nvol vl ng.
130- 134
cx .plcs of. 11 111 124- 138.
385- 386
four-vari abl eprob1cms 134- 138
graphi cal Sol ul i ons of. 07- 115
i nfmi tesol uti onof. 114
MATLAB sol uti onusi ng 120-124
modcl i ngissucs in98- 107
negati ved ignvari abl cs in. 98
NLP probl ems vs.. 20 93- 94
nosol onfor. 114- 115
pri m probl em. 138
sensitivitynnal ysi s wi . 148-151
Si mpl cx methodforsol vi ng 115-124
stllndardformat for. 94- 98
transponal i onprob1emex np1e.
124- 130
unboundedsol uti onof 114
uni quesol uti onof. 114
tvtors i06- 107
Mni nf mme computi ngsysl ems. 24
Mul druw. 47
Mat hem8!ti cal mode. 5
Mat hem8!ti cnl mode1s/model i ng. 3. 10-16
MathWorks Inc. 2s
MATLAB. 24- 44
codesnippet. creal i nga. 37- 40
edilor in. 33- 37
l-limeuseof. 27. 29- 33
bi gh-Jeve1gr phi cs functi ons of. 48- 50
insl a11ationof. 26- 28
operurs in. 30-31
Opti mi zati onT 1boxof. 123-124.
379- 391
progrnm. c ati nga 40- 44
Si mp1ex Method using. 120-124
Symbol i c Math T 1boxof. 159-161
usefu1ness of 26
Matri c:
augmenl . 103- 104
396 I NDEX
MBI I I s (continue.
rankof 104- 106
Mu i mumprobl ems 6
M- fi127
Mi ni mi zati onprobl ems 6
Modi fi edNewton me d252- 253
Mu1ti di sci pl i narydesi gns 6
Mu1ti obj ccti vedi gns 6
n-di mensi onal Eucl i dennspa : 5 106
Newton- Raphson techni que 206- 209 252
Non1i ncar programmi ng ( NLP) prob1cms
21- 23 154-2'
W1a1yti 1condi ti ons in 175-176
consaints in 157-159 172-175
179-191
examp1es of. 194-2
firsl -ordcr ndi ti ons in 176-178
formu1ati onof probl em 155-157
graphi ca1sol uti ons of 171-175
Kuhn- Tuckcr condi ti ons for 192-193
196- 2
Lagrangc mul ti pl i er mcthod useof
180- 192
LP prob1ems VS. 20 93- 94
one vari abl e functi ons of 162-164
169- 170
mnd0 ercondi ti ons in 178-179
symbol i c computoti on 159-161
Tay10r scries for useinsol vi ng 169-171
t woor mo varillblcsfuncti ons of
164- 171
Obj tivcfuncti ons 6
0cct ori entcdprogram g24
One-di mcnsi onal probl ems 203- 225
bi sccti ontccbni quefor usewi 209- 210
constr ned204-205
dcfinitionof probl em 204
gol densccti onmcthodfor usewi th
214- 217219- 220223- 225
g M1sol uti onof 205- 206
i mportanceof 217- 219
Newton- Rapbson tcc quefor usewi th
206- 209
pol ynomi a1approxi mBti onwi th 211- 214
two-poi nt boundary val ueprobl em
220- 223
Operators 30- 31
timal solu 17
Opti mi zati onl - 4
Opti mi zati onTool box 123-124 379-391
constrai ncdopti mi l.ationusi ng 389-391
exampl es usi ng 385- 386
1inear progmmmi ng exampl c usi ng
385- 386
opti mi zati onparameters setting 384- 385
qua icprogr uni ngcxamp1e usi ng
386- 388
unconstr i nedopti mi zati onusi ng
388- 389
usi ngprogmms in 382- 384
OUTPUT program(Opti mi zati onToo1box)
383
Parame d ign 5
Partial derivativ 1
Pascnl 24
Pntl emScarchmetbod 234- 238
Po1ynol1u pproxi mati on211- 214 217
Powel l ' s mctbod 238- 240
a1gori m238- 239
andconj ugacy 240
Pri mal probl em 138
Probl cmdefi onandfirmu1ati on4- 10_
SeealsoMatbemati cal model sJ
model i ng
coS nedopti mi zati onprobl ems
266- 267
nstr nt functi ons in 7
desi gnfuncti ons in 5- 6
di par neters in 5
anddesi gnspa 9
desi gnvnriab1es in 4- 5
itynstr i 7- 8
gl obal opti mi l.at i n351- 357
gruphi cu1opti mi zati on 45- 48
i nequal i tyconstraints in 8
nonl i near probl ems 21.. 23
I i n r programmi ng(NLP) prob1ems
155-159
obj ccuvefuncti onin 6
one-di mcnsi onal prob1em 204- 205
sidens aints in 8
standardformat for 9- 10
unconstmi ncdopti mi zati onprobl ems
227- 230
Prob1emrclaxation 321- 322
Probl emsol uti ons 16- 24
Programmi ng discrcte 20
grammi ngintcg 20
Programmi ng l anguag24
Programmi ng pamdi gms 24
Pseudornndomnumbers 230
Quadrati c convergence 238
Qua aticpo1ynomi al s 238
Qua aticprogr ng386- 388
RundomWal k method 230- 233 254- 255
Redundancy101
Rosenbrock probl em 253- 255
Rota Igdi sk thrce-di mensi onal 110wnear
255- 258
SA seeSi mul atedAnnenl i ng
Script m- fiI es 27
S ha1gori m218
Searchdircction 22
Searchme ods21
Second-order condi ti ons (SOC) 178- 179
ScI I d-urderl l l near variation 169
Sensitivityan ysis148-151
Sequenti al dcci si onprobl ems 336
Sequenti nl Gradi ent Restorati ona1gori tbm
( SGRA) 302- 307
nl gori tbm 305
appli n305- 37
gradi ent phnse 303- 304
restorationphase 304- 305
Sequenti al Li near Progr ng(SLP)
284- 289
gori m285- 286
Sequenti al Qua atlcProgmmmi ng (SQP)
289- 296
a1gori thm 294
and BFGS method 293- 294
multipliers cal cul ati onof 293
stepsize ca1cul ati onof 292- 293
Sequenti al Unconstrni ncdMi ni mi zati on
Tcchni ques ( SUM' 270- 271
SGRA seeSequentl ul Omdi cnt Restorati on
aJ gori tbm
Si deconstrai nts 8
Si mpl cx method 115- 124
appl i cati onof 117- 120
I NDEX 397
Enteri ngBasi c Vari abl e(EBV) 118
features of 115- 117
Leavi ngBasi c Vari ub1c ( LBV) 118
MATI-AB sol uti onusi ng 120-124
t wo- phase 125-130
Si mul l l tedAnneal i ng(SA) 350 358- 366
a1temati vc a1gori m359- 365
basi cal gori thmof 358- 359
nstraints hand1i ng 364
indlscrcteprobl cms 365- 366
Sl ackvariab1es 95
SLP seeSequenti al Li ncar ' Ogrammi ng
SOC seeSccondordcr condi ti ons
Softw 24
Sol uti ons 16- 24
acceptobl c 17
infinite 17
initial desi gn 21
opti m 17
SQP seeSequenti al Quadrnti c Programmi ng
Stnndardformal 9- 10
Stecpest Dcent me d241- 244
SUMT J-eeScquenti Unconstrai ned
Mi ni mi zati onTcchni ques
Symbo1i c Math T ol box(MATLAB)
159-161
Tayl or scric 169-171
Tol ance209
TPBVP secT wopoi nt boundary yal ue
probl em
Trans nprobl cmexampl e 124- 130
Twc poi nt boundary va1ueprobl cm
'PBVP) 220- 223
Unconstrai nedoplimizlltionprobl ems
227- 262 388- 389
Bczi cr paramctri ccurves fittlng
258- 262
Broydon- F1etcher-Go1dfarl: Shanno
methodfor sol vi ng 249- 251
CougateGradi ent mctbodfor sol vi ng
244- 246
Davi don- F1etchcr-Powcl l mel hodfor
sol vi ng24 249
definitionof prob1em 227- 230
generi c a1gori tbmfor 229- 230
396 I NDEX
Mal ri (conti nued)
rankofl04- 106
M umprobl cms. 6
M-fi l 27
Mi ni mi zati onprobl cms 6
Modi f i Newt on met hod 252- 253
Mul l i di sci pl i nnrydesi gns 6
Mul ti obj ecl i vedesi gns 6
n-di mensi onal Eucl i deanspaces 106
Newl On- Rapbso tech quc206- 2252
Nonl i near programmi ng ( NLP)
21- 23 154- 200
anal yti cal condI ti ons in 175- 176
constrllInts in 157- 159 172- 175
179- 191
cxampl 194- 200
1- 0 er condi l i ons in 176- 178
mul ati onof probl ern 155- 157
graphi cal sol uti ons of 171- 175
KuhnTucker condi ti ons for 192- 193
19 2
Lagmnge mul l I pl i er mel hod useof
180- 192
L P probl ems vs. 20 93- 94
one vari nbl e. funcl I ons of. 162- 1 64.
169- 170
second-order condI l i ons in 178- 179
syrnbol i c computaon159- 161
Tl1yl orseri for useinsol vi ng. 169- 171
t wo or mor o varI abl es funcl i ons of
164- 171
Obj ecti vefuncti ons 6
0 l ori cn programmi ng24
One-dI mensI onal probl cms 203- 225
bi sccti ontechni quefor usewI th 209- 210
COI ned204- 205
defi ni ti onof probl cm 204
gol d sccti onmet hod for usewi th.
214- 217219- 220223- 225
grnphi cal sol uti onof 205 - 206
i mportanceof 217- 219
Nc wl On-Raphson l echni quefor usewi th
206- 209
pol ynomi al approxi mati onWi l h 211- 214
two-poI nt boundnry val uc probl em
220- 223
Opeml o 30- 31
Opti mal sol uti ons 17
i onl - 4
Opti mi zati onT l box 123- 124379- 391
consl mI ncdoptI mi znl i onusi ng 389- 391
exampl es usi ng 385- 386
l i near prograrmni ngex npl eusI ng
385- 386
opti mi zal i onp me rsseui ng 384 385
qua .tic programmi ng cxampl e usi ng
386- 388
un i nedopti mi zati onus E
388- 389
usi ngprograms In 382- 384
0 r pUT pr ogmm(Opti mi zati onT l box).
383
Parameters desi gn 5
Pani al deri vati ves 164
Pascal 24
Pat t cmS rchmcthod 234- 238
Pol ynomi al opproxi mati on 211- 214. 217
Powell' s me od238- 240
a1gori m238- 239
app1 n. 239- 240
and conj ugacy 240
Pri mal probl em 138
Probl emdefi ni ti onand f ormu1ati on 4- 10.
Seeal soMal hemati cal mode1s1
model i ng
cons dopl i mi zati oDprobl ems
266- 267
nsuai nt functi ons in 7
desi gnfuncti ons in 5- 6
desi gnparamete in5
and desi gnspace 9
desi gnvnri abl cs in 4- 5
equal i tyconsuai nts in 7- 8
gl obal pt ati on351- 357
graphi cal opl i mi zati on 45- 48
i nequa1i tynsuai nts in 8
nonl i near probl ems 21 23
nonl i near programmi ng ( NLP) probl ems
155- 159
0 tivefuncti oDin 6
one-di mensi onal probl em 204- 205
si deconstmi nts in 8
standordformnt for 9- 10
uncons i ncdopti mi zati onprobl ems
227- 230
Probl emrel ati on321- 322
Probl emsol uti ons 1624
Pr ogmmmi ng di sc te20
Programmi ng i ntegcr 20
PrognunoUngl anguages24
PrognunoUng paradi gms 24
Pseudorandomnumbers 230
Quadrati cconvergen238
Quadrati c pol ynomi al s 238
Quadmti c programmi ng 386- 388
RandomWa1k method 230- 233 254- 255
R undancy101
R05cnbrock probl em 253- 255
Rotati ngdi sk thr c-di mensi onal f l ownear
5- 258
SA .rtt Si mul atedAnneal i ng
Scri pt m-fi l cs 27
Searcha1gori thm 218
S hdi tion22
S rchmethods 21
Second-ordcr condi ti ons ( SOC) 178- 179
Second-or r/l i near vari nti on 169
Sensi ti vi l yannl ysi s 148- 151
S uenti al d i SI ODprobl ems 336
Sequcnti al Gradi ent Restorati ona1gori thm
( SGRA) 302- 307
a1gori m305
gradi ent phe303- 304
restorati ODphase. 304- 305
Sequenti al Li near Programmi ng (SLP)
284-289
al gori thm 285- 286
app1i cati on 286- 289
Squenti al Quadrati cProgrammi ng ( SQP)
289- 296
al gori m. 294
appl i cnti on 295- 296
andBFGSmeod 293- 294
mu1ti pl i ersI i onof293
stepsi ze cal cul ati onof 292- 293
Sequenti al Unconsuai ncd I I mi 7.otion
Techni ques (SUMT> 270- 271
SGRA soe Scqucnti aJ Gradi enl Res[orati on
n1gri thm
Si dcns nts8
Si mpl ex met hod 115- 124
pl i cati onof 117- 120
I NDEX 397
Enteri ngBasi c Vari abl e( EBV) 118
features of 115- 117
Leavi ng8asi c Vari abl e( LBV) 118
MATl AB sol ul i onusi ng 120- 124
two- phasc 125- 130
Si mul edAnneal i ng( SA) 350 358- 366
a1temati veal gori thm 359- 365
basi ca1gori thmof 358- 359
n5Uin h ldJ i ng364
indi screteprobl ems 365- 366
Sl ack vari abl es 95
SLP seeSequenl i al Li near Programmi ng
SOC sl!e Sccond-order condi ti ons
Soware24
Sol uti ons 16- 24
aeptab1eI 7
infinite 17
inilial desi gn 21
opti mal 17
SQP seeSequenti al Quudrnti c Prograrnmi ng
Standardfonnnt 9- 10
St st O nt met hod 241- 244
SUMTseeS uenti al Unconstmi ned
Ml nI mi zati onTechni ques
Sym l i cMat hT l box( MATl AB)
159- 161
Tayl or scri es 169- 171
To1erunce 209
TPBVPs T wopoi nt bound y va1ue
probl em
Transpon i onprob1emcx p1e124- 130
Twc poi nt boundnry val ueprobl em
' PBVP) 220- 223
Unn5U i nedopti mi zati onprobl ems
227- 262388- 389
8ezi cr pnrametri c cu fitting
258- 262
8roydon-Fl etcher-Gol dfart Shanno
met hod for 50l vi ng 249- 251
Conj ugateGradi ent mel hod for 801vi ng
244- 246
Oavi don- F1cl cher-PowclI mel hod for
sol vi ng24 249
defl DI tionof probl em 227- 230
generi c a1gori thmfor 229- 230
E
398 I NDEX
Uncorul l rai nedoplimiz. i onprobl ems
(con e
g di ent-based l ethods for sol vi ng
241- 251
modi fi edNeWl On method for sol vi ng
252- 253
neces ry Il Dd sufficient condi ti ons for
228- 229
nongradi ent mel hods for sol vi ng
230240
PattemS hmethod for sol vi ng
234- 238
PowcU' s mel hod for sol vi ng 238- 240
randomwa1ka1gori l hmfor 230- 233
Rosenbrock probl em 253- 255
rotatingdi sk thrce-di mensi onal sow
r255- S
second-order mel hods for sol vi ng
251 3
Sl eepcst Dcscent mcl hod for sol vi ng
241- 244
Uni que opti rna1solutions. 17
Uni tv ors106- 107
VarillbleMetri c Methods ( VMM) 246
Vari nbl es:
nrtifici H l 98
conti nuous 20
desi 4- 5
discrel e 20
i nteg 20
sl ack95
Vecr(s):
desi gn 5
uni t l O 107
V MM (Vari abl eMetri c Mel hods) 246
Zero- Onel n P r ogramrni ng(ZI P)
343- 348
Zero- order methods 230
ZI P seeZero- One l nl eger Programmi ng

Applied Optimization With MATLAB Programming

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Applied Optimization With MATLAB Programming

Uploaded by

Copyright:

Available Formats

MAn. AB is a regisleredl rademarkof leMal hWorks Inc.

Thi s book is p ledonncid paper . @

Appl i edopl i mi Zl l lionwi th Programmi ngI P. Venkal ar mlln.

The book was made possi bl ethroughsupport f romJ ohn Wi l ey dSonsInc. d

Fl guro1.3 Exampl e 1.2- Cross-sectlonal shape.

1.1 OPTI MI ZA TI ONFUNDAMENTALS 15

Conl our plol 01Exampl e 1.1.

call it XoX.. is some mul ti pl eof SI' s SI X

1.2 I NTRODUCTI ONTOMA11.AII 25

echo. Defaul t is a comma.

These comment s wi l l appear when t he user t ypes

>>coef f 1 mypol yf i t ( XY2

Thi s conc1udes e exerci se where a code s p W written to lc ate E

2.1 PROBL EM DEFI NI TI ON

50 GAAPHI CAL OPTI MI ZATI ON

gr aphi cal sol ut i on usi ng MATLAB ( t wo desi gn var i abl es)

(x1 3) **2 + (x2 2) **2

NOTE: The hash mar ks f or t he i nequal i t y

f ( x1x2) a*xl ""2 + b*x2^2 c cos ( aa*x1)

L ' " L ' " Lp)/(E'"

gr aphi cal sol ut i on u6i ng MATLAB ( t wo desi gn var i abl es)

gr aphi cal sol ut i on usi ng MATLAB ( t wo desi gn var i abl es)

GUI i s cr eat ed wi t hi n a f i gur e wi ndow

scr i pt M f i 1e f or 20 Cont our P10t t i ng of f unct i ons

The Number of p10t s and r ange f or t he p10t

GU2 4 cal l t he GUI

2.4 AODI TI ONAl MA11AB GRAPHI CS 89

a and its detenni nant is

det(A) = IAI = (1)*(2) ( 1) 2) ==0

Li nes ar e r epr esent ed as: ax + by c ( c >= 0)

The f unct i on wi l l dr aw l i ne( s) i n t he f i gur e wi ndow

f unct i on dr awLi ne( x1x2abct yp)

g2(X): 3xI - X2:s; 25

0. 0007 0. 0010 - 0. 0001 o 0. 0052

0. 0007 0. 0010 0. 0001

- 0. 0010 0. 0010 0. 0450

0. 0001 0. 0001 0. 0000 o 0. 0005

3.4 ADDI TI ONAL EXAMPLES 129

Sujectto: gl : X1 +X21- X22 +81 =5

g3: X1 + X21- X22 - 82+ a2 = 1

3.5 ADDI TI ONAL TOPI CSINLl NEAR PROGRAMMI NG 139

1. 3333 0 -1. 3331 1. 3333

Tabl e 3. 25 Exampl e 3.5: Dual Probl em Phase 1 Tabl e 3

o - 0. 2857 -1. 1429

0. 1429 -0. 8571

o -0. 25 - 0.875 0. 875 2. 4999

0. 125 -6E-O - 0. 1875 6E- 06 0.5 0. 1875 - 6E - 0.5 1. 25

0.25 4E- 05 0. 125 - 4E- 05 - 0. 125 4E- 05 12.5

146 Ll NEAR PROGRAMMI NG

162 NONLl NEAAPAOGAAMMI NG

F1gure4.9 Three-di mensi onal plot 01Fi gure4.6.

The matri x is posi ti vedefinite.

Opt i mi zat i on wi t h MATLAB Sect i on 4. 4. 2

t he unknowns have t o be def i ned as a vect or

m. For Cases b and c theappropri atemul ti pl i er is set tozeroand the

of f unct i on eval uat i ons

t(or zero) of a functi onis trappedor bound between

The final iterationis reproducedby runni ngthefol l owi ngcode:

scanmnginterv ( u t 1) pa55edtoUp per Bou md_1Va r' .m;them u mE

234 NUMEAI CAl TECHNI OUES FO UNCONSTAAl NEDOPTI MI ZATI ON

250 NUMERI CAL TECHNI QUES FORUNCONSTRAI NEDOPTl MI ZATl ON

Thc statcequati ons (6.19) .c i ntcgral edusi ngMATLAB functi onode45 .

6.5 AOOI TI ONAL EXAMPLES

(x1 3) 2 + (x2 2) 2

f ( x1x2) axl ""2 + bx2^2 c cos ( aa*x1)