6 Analytical Multidimensional (Multivariable) Unconstrained Optimization Leaming Objectives After studying this chapter, you should be able to: 1. Describe the method of solving problems of multivariable unconstrained optimization. 2. Classify the multivariable optimization problems. 3. Explain the necessary and sufficient conditions for solving, multivariable unconstrained optimization problems. 4. Solve unconstrained multivariable functions In Chapter 5, we formulated the single variable optimization problems without constraints, Now let us extend those concepts to solve multivariable optimization problems without constraints, The optimization of such problems is routed in more than one direction, For instance, if there are two variables in the objective function, we eall such problems fwo-dimensional as we have to arch from two directions, Similarly, a three-variable problem can be called shree-dimensional, and so on, However, the method of solving is similar in all such cases, and, therefore, all these problems can he named muitidimensional or multivariahle unconstrained optimization problems. Recollect that we made use of complete differentiation to solve single variable unconstrained optimization problems. Here we use the concept of partial differentiation in solving multivariable unconstrained optimization problems using analytical methods 6.1 Classification of Multivariable Optimization Problems The multivariable optimization problems can be classified broadly into two categories: multivariable unconstrained optimization, i.e. without constraints, and multivariable constrained optimization, i.e, with constraints, Further, since we categorize the constraints into two typs 114Analytical Multidimensional (Multivariable) Unconstrained Optimization 115 such as equality and inequality type we can further classify the multivariable constrained optimization problems into two types. Summarily, the classification of multivariable optimization problems is shown in Figure 6.1 Multivariable optimization problems -——— Multivariable Multivariable lunconstrained optimization, constrained optimization, i.e, without constraints i.e, with constraints | Equality constraints Inequality constraints Figure 6.1 Classification of multivariable optimization problems In this chapter, we discuss the analytical solution methods for multivariable unconstrained optimization. 6.2 Optimization Techniques to Solve Unconstrained Multivariable Functions Let us now discuss the necessary and sufficient conditions for optimization of an unconstrained (without constraints) multivariable function for which we make use of Taylor's series expansion of @ multivariable function. Taylor’s theorem/series expansion is given in Exhibit 6.1. Exhibit 6.1 Taylor’s Theorem/Series Expansion Taylor’s Series In the Taylor’s formula with remainder (Eqs. (i) and (ii)), if the remainder R,(x) > 0 as n ~ «, then we obtain (x=a)" PO ee. @ which is called the Pitch 5 series, When a = 0) we obtain the Maclaurin s series : £00) = (a) #8 p7(@) 4 SO 9) 004 C—O FOI=LO +E LO) + I") 10) (i) Since it is assumed that f(x) has continuous derivatives up to (n + 1)th order, /” * (x) is bounded in the interval (a, x). Henee, to establish that lim | R, (x)|=0, it is sufficient to show 0 for any fixed numbers x and a, Now, for any fixed numbers x and a,116 Optimization Methods for Engineers wwe can always find a finite positive integer N such that [x ~ aj <.N. Denote q = |x — al/N. Then, x —ay""! (nt! (x- ay"! (N-))! (x-ay""! (n+I)! (x-a)"" (N=)! for any fixed x and a, and lim |R,(x)|=0. Now, is a finite quantity and is independent of n. Also q <1. Hen IMlustration 6.1 Obtain the Taylor’s polynomial expansion of the function f(x) = sin x about the point x = 7/4, Show that the error term tends to zero as n > © for any real x, Hence, write the Taylor's series exapansion of f(x), Solution For f(x) = sin x, we have SEM) = ly sine and f"* Me) = 1)" cos x for any integer n, Therefore, eon( 2) ED" ang em 8 f (5) wot 4 + = 7/4 is given by Jeotseacn nyntl eam “a (x) > 0 as n > &, Hence, the Taylor’s expansion of f(x) = sin x about woh a ee (+2) orn(2] (2n +1)! 4 4 since f2"* "(n/4) = |(-1)" cos m/4| < 1, Hence, R,,/ Similarly, we find that R,,,,, x) > 0 as n> 02, Therefore, the required Taylor's series expansion is given by Now, | Raq (x) ne -2)-aal-3) a 1 a 1 atfe( le i tod x siny=—= +—=[ xAnalytical Multidimensional (Multivariable) Unconstrained Optimization 117 6.3 Necessary Condition (Theorem 6.1) If f(x) has an extreme point (maximum or minimum) at x = x° and if the first derivative of f(x) exists at x’, then a(x") _ AO) __ AF") ay Oxy ax, Proof: [The proof for Theorem 5.1 given in Chapter 5, can be extended to prove the present theorem, However, a different approach is presented here for the benefit of read Suppose one of the first partial derivatives, (say &th), does not vanish at x’. Then by Taylor’s theorem, fee + y= me) + 3h, apts’ 2 lpn for0<0<1 + Ry’, h) ie. fix’ +h) — fe" =h a Since 0?/(x" + Oh) is of the order h?, the terms of the order / will dominate the higher-order terms for small h, Obviously, the sign of f(x" + h) ~ f(x") can be determined by the sign of >0, then fix" + h) — fix’) is positive for A, > 0, and is negative for h, < 0. This means that x" cannot be an extreme point. Similarly, this conclusion can be drawn even for Af(x")/dx, < 0. Thus, the conclusion (i.e. x° is not extreme) is in contradiction with the original (i.e. x* is extreme). And this contradiction can be overruled if dfix")/ax, = 0 at x = the theorem is proved. itement *. Hence, 6.4 Sufficient Condition (Theorem 6.2) A sufficient condition for a stationary point x" to be an extreme point is that the matrix of second partial derivatives (Hessian matrix) of f(x) evaluated at x° is: (i) Positive definite when x” is a relative minimum point. (ii) Negative definite when x* is a relative maximum point. Proof: From Taylor's theorem, we have Fo") Lee Hr) ty Ey Go i Yistjat a sO" += fo" Ea118 | Optimization Methods for Engineers a From the necessary condition theorem, we have * =0,1=1,2, Therefore, i . . n fy f(x +h) fe ee PL or a lietaoy MO< 0, Jy> 0, then J is positive definite and x is a relative minimum, (ii) J,> 0, J, < 0, then J is indefinite and x" is a saddle point. (iii) J, < 0, J, > 0, then J is negative definite and x" is a relative maximum, (iv) Jy <0, Jy < 0, then J is indefinite and x" is a saddle point, Table 6.1 presents the sign notation of J. Table 6.1 Sign Notation of J A J, positive or J,>0 Jy negative or J,<0 & (is positive) (is negative) J, positive or J,>0 | Sis positive definiteand x" |./is negative definite and (Fis definite) isa relative minimum x’ isa relative maximum Jz negative or J<0.|.Fis indefinite and" isa |.Jis indefinite and x" isa (Fis indefinite) saddle point saddle point120 Optimization Methods for Engineers Note: If in a function of two variables f(x,, x3) the Hessian matrix is neither positive nor at which affx, = afi0x} is called the saddte point, minimum with one variable when the other is fixed. negative definite, then the point (x. which means it may be a maximum o Illustration 6.2 Determine the extreme points as well as evaluate the following funetion f(x): tad + Or? + 4d +6 Solution The necessary cond ence of an extreme point are or of mero md no a 7 = 3x} + 4x, = x,Gy, +4) = 0 BE oo sef + 8x, exyGxy + 8) 20 wi Fey 4 From Eq. (i), we get xy = 0 or ~> 8 And from Eq, (i), we get.x, = 0 or —= Hence, the above equations are satisfied by the following points: (0, 0), (0, -8/3), (4/3, 0) and (4/3, -8/3) Now, let us find the nature of these extreme points for which we use the sufficiency conditions, ie. by the second order partial differentiation of / 2 ax? 2 oF = 6x, +8 oe wey, ‘The Hessian matrix of fis given by Hence, andAnalytical Multidimensional (Multivariable) Unconstrained Optimization 1211 The values of J, and J, and the nature of the extreme points are given in Table 6.2. ‘Table 6.2 Sign Notation of J in Mlustration 6.2 Point x Valueof Value of Nature of J Nature of x Se) J + (0, 0) +4 + 32 Positive definite Relative minimum, 6 (0, -8/3) +4 32 Indefinite Saddle point since 418/27 afiox;= afi Ay (43,0) 4 —32 Indefinite Saddle point 194/27 (43, -8/3) 4 +32 Negative definite Relative maximum 50/3 - Illustration 6.3 Find the extreme values of the function f(xy, x3) = a7 - x}, Solution Applying the necessary conditions f/x, = 0 and dffdx, = 0, we have dy T= m=O f ary Now, applying the sufficient conditions Since J, is positive (+2) and J, is negative (4), the nature of J is indefinite and the point at 0, = 0 is a saddle point, and also the value the function at this saddle point is /(0,0) = 0. Note: At the saddle point if one of the values of x, or x, is fixed, the other may show extremity. Illustration 6.4 — Sreeja and Co. uses D units per annum of a commodity in an inventory system, which allows shortages if the purchase cost is less than the carrying cost. Suppose the total inventory Q is used in time 1, but Q, and Q, are the quantities in each cycle of stocking and shortage periods (say f, and 1,) respectively, and carrying cost and shortage costs are C, and C, per unit per annum respectively. Assuming consistent order cost on each order as C,, find the optimum inventory stock and the optimum allowable shortage stock so as to keep the total variable inventory cost at minimum, If the order cost is % 40 per order, the monthly demand is 20 units, the carrying cost and the shortage costs are F 1 and € 3 per unit per year respectively, find the optimum stock and the optimum shortages so that the total cost is minimum.122 Optimization Methods for Engineers Solution The inventory system can be represented in a graphical form as shown in Figure 6.2. Figure 6.2 Illustration 6.4 Here Q, = inventory in stock Q, = O ~ Q, = inventory in shortage 0 = 0,+ O, = total inventory consumed {, = period of inventory carrying {, = period of inventory in shortage i = total cycle period Now AABC and ACED are similar. Using the componendo-dividendo rule, g +9, Similarly, (ii) Annual carrying cost (ACC) The area of AABC represents the inventory in stock of the cycle [the annual carrying costs are summation of all such areas (say n cycles) multiplied by the unit carrying cost].Analytical Multidimensional (Multivariable) Unconstrained Optimization 123 Inventory per eyele = 5' . bo 24 [Using (i)] Q; _@ Inventory per annum = SGU 4-41 +--+ mime] = 550) ss xa year) (vint'=Lyear) Uheretore, annual carrying cost (ACC) (mn) Annual shortage cost (ASC) Similarly, the area of ACDE represents the inventory in shortage period. Therefore, shortage cost per cycle = C, (Een, = Get [Using ii] Since Q, = 0 ~ O, we have 2 20 CO-O)t 20 Annual shortage cost (ASC) = $} shortage cycle cost a _ (9-9, “a (0-0) nt 29 CQ 0) x1 year ‘ + year) = a int = 1 year _ €.@Q-2,)* eS Shortage cost per cyel [t+ t+ on times] (iv) Annual order cost (AOC) Total annual usage = D Unit order cost = C, Annual arder costs (AOC) = C,DIO Therefore, total variable inventory cost, T= ACC + ASC + AOC124 Optimization Methods for Engineers Therefore, : 10, 0) = 6.2. +¢,2=@ 20 20 w) The values of Q and Q, can be found by equating the first partial derivatives of (v) watt. Q and O, ar _ 20, 20,224 Sp 20-94) +0 a0, 29 20 a at 22 20, we have Gy 2 a9, or OC. - OC, + OC,=0 a2 . > O° GtG (vi) ar__ 03 ,¢,[200-9)-(0-9"]_¢,o Now a0 20" 7 e e (@-2)20-0+9)] cp “$ oe at 2220, wo have 0g 20° o or CQ? + C.Q- C,Q? = 2¢,D or (C,* C)Q? + € 2c) Using (vi), we get 2 ©, 2 ecg ~(¢,+¢,)| —“— | 9 +¢,0? =2¢,D (ere(cSq} ot seat ae, or =C; +C, |Q* =20,D G.+G, -C?+C,C,+C2) or a | 9? = 20,0 C.4+eyAnalytical Multidimensional (Multivariable) Unconstrained Optimization 125 or Q? =2,D 2 2C,D( C. + C, " (se) Therefore, on, PEP ee C,. \Gac, ci & Now, if C, = & 40/order, D = 20 x 12 C,=% 1 and C, = 3, then 40 units per year ._ [2xaox240 /r+3 . o'= pxdon24o (13) 160 units and OF = |2%40%240(_3_) «196 units 1 3 143 With Q* = 160 units and QF = 120 unite, using Eq. (v) 2 2 Minimum cost = 1x20 4.3 x G60=120)" 5, 40% 240 2x 160 2x 160 160 =45 + 15 + 60 =F 120 Illustration 6.5 ‘Two frictionless rigid bodies R, and R, are connected by three linear elastic springs of constants &,, ky and &, as shown in Figure 6,3. The springs are at their natural positions when the applied force P is zero, Find the displacements x, and x, under the force of 26 units by using the principle of minimum potential energy assuming the spring constants k,, k, and A, as 2, 3 and 4 respectively. Examine whether the displacements correspond to the minimum potential energy, x f *y f Figure 6.3 Illustration 6.5. Solution We know that the potential energy U is given by Potential energy, U = strain energy of springs — work done by external forces, (12k xt + (U2)Kg(Xy — x)? + CDK, where ky, & and k, are the spring constants126 Optimization Methods for Engineers Now taking ,, k, and k, as 2, 3 and 4 respectively, we get the potential energy, U as i 2 J ats Us| Shox? + phen mi) +5 hn |- Pe [dx ax? +b x a(x. =x) +feaui]- Px. BSR Hy XA AY +5 ag [Pee aha +2(xy — x)? +x} — Pry Applying the necessary condition for the minimum. of U, and substituting the value of spring constants, we get (i) and (ii) = (xy =) 42x, -P=0 ‘The values of x, and.x, corresponding to the equilibrium state obtained by the above equations are he = Aiks + hak) +hky (2X3) +Bx4)+ (4X2) 26 a Pky + hs) 4p 7P kyky #haky + hk, 2X3) +Bx4)+ 4X2) 26 At P = 26 units, xj = 4 units and x3 = 7 units, ‘Now, the sufficiency condition for the minimum PE at (xj, x3) can also be verified by testing the positive definiteness of the Hessian matrix of U. The Hessian matrix of U' evaluated at (aj, 5) is 2 2 2 Geb the wage bah Se ky +h =6 vu Fu Ox? OM Axy Inthk, Wk; au au -| mk ky i axax OF Jey 7 -4 Le] The determinants of sub-matrices of J are J,= lt blak + =7>0Analytical Multidimensional (Multivariable) Unconstrained Optimization 127 and Since the values of J are positive, the matrix J is positive definite and hence the values xy and x} correspond to the minimum potential energy. x, =4and x5 IMlustration 6.6 ‘The profit on @ contract of road laying is estimated as 26x, + 20x, + 4ryx, where x, and x, denote the labour cost and the material cost respectively. Find the values of labour cost and material cost that can maximize the profit on the contract. Solution Given, profit Zoxy, x5) = 26x, + 20x, + Anyx, — 3x? ~ 4? Applying the necessary conditions, i.e. az my ay dey 22 964 4ry — 6x =0 @ ay and 2. 20+ 4x Bx, (ii) ax On solving Eqs. (i) and (ii), we get x} = 9, x3 = 7. Now, let us apply the sufficiency condition to maximize the profit (Z) at (x), x3 verified by testing the negative definiteness of the Hessian matrix. The Hessian (x }) is calculated as az az vz 2% 2-6, Ha -8 and ae ae ANON, yp. 4 Hessian matrix Jt] 4g The determinants of the square sub-matrices of J are 6|=-6<0 48-16=32>0 Since J, is negative (i.e. -6) and J, is positive (i.c, +32) which is definite, the mature of J is negative definite, Hence, the values of x, and x, show a relative maximum for Z at respectively, # x= 9x) =7