ME 441 Homework 5b

Aditya Ramkumar 288575

1) Golden section method: Problem 10.17

%Objective function script-objfn1017.m
function f= objfn1017(x)
f= 7*x*x -20*x +22

%golden section iteration script-gss.m

function [a,b]= gss(f,a,b,eps,N) % a,b -initial interval ,eps-error
d=0.05;
imin=0.005;
k0=d;
f0=7*d^2-20*d+22;
%initial interval bracketing
for n=1:10
k(n)=k0+((1.618)^n)*k0
f(n)= objfn1017(k(n))
k(n+1)=k(n)+((1.618)^(n+1))*d
f(n+1)=objfn1017(k(n+1))
k(n+2)=a(n+1)+((1.618)^(n+2))*d
f(n+2)= objfn1017 (k(n+2))
if f(n+1)<f(n)
if f(n+1)<f(n+2)
a=k(n);
b=k(n+2);
end
end
end
%using golden ratio to get subsequent intervals
c=(-1+sqrt(5))/2; %golden ratio
x1=c*a +(1-c)*b;
X=x1;
fx1=feval(f,x1); %evaluating function at that point
x2=(1-c)*a +c*b;
Y=x2;
fx2=feval(f,x2);
fprintf('---------------------------------------------------\n');
fprintf(' x1
x2
f(x1)
f(x2)
b - a\n');
fprintf('------------------------------------------------------\n');
fprintf('%.4e %.4e %.4e %.4e %.4e\n', x1, x2, fx1, fx2, b-a);
for i=1:N-2
if fx1<fx2
b=x2;
x2=x1;
fx2=fx1;
x1=c*a +(1-c)*b;
fx1=feval(f,x1);

%new interval bounds

X=[X;x1];
Y=[Y;x2];

else
a=x1;
x1=x2;

%new interval bounds

fx1=fx2;
x2=(1-c)*a +c*b;
fx2=feval(f,x2);
X=[X;x1];
Y=[Y;x2];

end
fprintf('%.4e %.4e %.4e %.4e %.4e\n', x1, x2, fx1, fx2, b-a);
if (abs(b-a)<eps)
%to check if tolerance is met
fprintf('succeeded after %d steps \n', i);
X
% prints the array of x1 and x2
Y
return;
end
end
fprintf('failed requirements after %d steps \n',N);

ITERATION RESULTS FROM COMMAND WINDOW:

>>
X1
1.3704
1.1587
1.3704
1.2896
1.3704
1.4204
1.4013
1.4202
1.4322
1.4277
1.4249
1.4277
1.4294
1.4287
1.4283
1.4287
1.4286

X2
1.7129
1.3704
1.5012
1.3704
1.4204
1.4512
1.4204
1.4322
1.4395
1.4322
1.4277
1.4294
1.4304
1.4294
1.4287
1.429
1.4287

Succeeded after 16 steps

a=1.4283
b=1.4290
f=7.7143

F(x1)
7.738
8.2239
7.738
7.8496
7.738
7.7148
7.7195
7.7148
7.7144
7.7143
7.7144
7.7143
7.7143
7.7143
7.7143
7.7143
7.7143

F(x2)
8.2801
7.738
7.7512
7.738
7.7148
7.7179
7.7148
7.7144
7,711
7.7144
7.7143
7.7143
7.7143
7.7143
7.7143
7.7143
7.7143

Interval size(b-a)
1.4507
0.8966
0.5541
0.34247
0.21166
0.13081
0.00808
0.00499
0.03088
0.01908
0.01795
0.00729
0.00451
0.00278
0.00172
0.00106
0.00066

Interval size

Convergence of interval to epsilon

1.6
1.5
1.4
1.3
1.2
1.1
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

Interval size

10

11

12

13

14

15

16

17

Iteration number

Observations:
The number of iterations taken to reach optimal solution for an epsilon<0.001 is 17, which
seems to be a lot for an asymptotic problem. This method may not work with multivariable
unbound problems.

2) Steepest descent code: Problem 10.56

% function for calculating steepest descent
function[x]=steepestdescent(x1,x2)
%f(x1,x2) = 25x1^2 + 20x2^2 -2x1 -x2
%Given Values
e=0.001; Epsilon=e;
%tolerance limit
x1 = 3;x2 = 1;
%initial search point
x=[x1 x2];
f=25*x1^2+20*x2^2-2*x1-x2;
%objective function
b=0;
%counter for checking if norm is less than tolerance limit
X=x;
Y=f;
counter =0; %number of iterations
grad = [50*x1 - 2, 40*x2 - 1];
gradf=grad;

% gradient function

descent=-grad;
%descent direction
D=descent;
% stopping criteria using norm
if norm(grad)<e
b=1;
else b=0;
end
counter = 1;
Epsilon=[norm(grad);e];
falpha=@(a)25*(x1 + descent(1)*a)^2 + 20*(x2 + descent(2)*a)^2 - 2*(x1 +
descent(1)*a) - (x2 + descent(2)*a);
a0=[0];
[a,fv1] = fminunc(falpha,a0); %minimising f in terms of alpha
Alpha=a;
%Calculating updated values for x1 and x2
x=[(x1 + descent(1)*a); (x2 + descent(2)*a)];
%updating the list of x1 and x2
X=[X;x];
f = 25*x1^2 + 20*x2^2 - 2*x1 - x2;
%updating the list of objective function
Y=[Y;f];
while b < 1;
counter = counter + 1;
grad = [50*x1-2,40*x2-1];
gradf=[gradf;grad]
%updating the list of gradients

if norm(grad) < e
b=1;
break
else b=0;

end
Epsilon=[norm(grad);e];
descent=-grad
%Calculating new gradient
D=[D;descent]; %updating the list of descent directions
%User defined function
falpha = @(a) 25*(x1 + descent(1)*a)^2 + 20*(x2 + descent(2)*a)^2 - 2*(x1 +
descent(1)*a) - (x2 + descent(2)*a);
a0=[0];
%initial starting value for alpha
[a,fv1] = fminunc(falpha,a0);
Alpha=[Alpha; a];
%Updating new x1 and x2 values and appending to list
x=[(x1 + descent(1)*a);
X=[X;x];

(x2 + descent(2)*a)];

%Updating new function value and appending to list

f = 25*x1^2+20*x2^2-2*x1-x2;
Y=[Y;f];
end
%displaying the counter, list of x1, x2, and function, alpha and epsilon
counter
Alpha
Epsilon
X
Y

Change in x1 and x2 variables

ITERATION RESULTS FROM

X(1)
X(2)
3.000
1.000
0.0011
0.2097
0.049
0.028
0.0399
0.0256
0.040
0.025
0.040
0.025

COMMAND WINDOW:
Epsilon
Alpha
153.0523
7.6418
0.0203
0.4632
0.0246
0.0231
0.0203
0.0014
0.0246
0.00000
0.0145

f
238
0.668
-0.0503
-0.0525
-0.0525
-0.0525

3.5
Change of x1
Change in x2

3
2.5
2
1.5
1
0.5
0
1

Iteration number

Objective function value vs. iteration

290

Objective function value

238
240

190

140

90

40

-10

0.668

-0.0503

-0.0525

-0.0525

-0.0525

Iteration number
Observations:
Steepest descent algorithm takes 6 iterations to reach convergence and final solution with the
descent direction orthogonally changing with every iteration. But there are better methods
such as Conjugate Gradient method as we shall see in the next problem.

3) Fletcher-Reeves Conjugate Gradient method for problem 5b:

% function for calculating conjugate gradient
function[x]=conjugategradient(x1,x2)
%f(x1,x2) = 25x1^2 + 20x2^2 -2x1 -x2
%Given Values
e=0.001; Epsilon=e;
%tolerance limit
x1 = 3;x2 = 1;
%initial search point
x=[x1 x2];
f=25*x1^2+20*x2^2-2*x1-x2;
%objective function
beta=0;
%beta value for calculating new descent direction
X=x;
Y=f;
counter =0; %number of iterations
grad = [50*x1 - 2, 40*x2 - 1];
gradf=grad;

% gradient function

descent=-grad;
%descent direction
D=descent;
% stopping criteria using norm
if norm(grad)<e
b=1;
else b=0;
end
counter = 1;
Epsilon=[norm(grad);e];
falpha=@(a)25*(x1 + descent(1)*a)^2 + 20*(x2 + descent(2)*a)^2 - 2*(x1 +
descent(1)*a) - (x2 + descent(2)*a);
a0=[0];
[a,fv1] = fminunc(falpha,a0); %minimising f in terms of alpha
Alpha=a;
%Calculating updated values for x1 and x2
x=[(x1 + descent(1)*a); (x2 + descent(2)*a)];
%updating the list of x1 and x2
X=[X;x];
f = 25*x1^2 + 20*x2^2 - 2*x1 - x2;
%updating the list of objective function
Y=[Y;f];
while b < 1;
counter = counter + 1;
grad = [50*x1-2,40*x2-1];
gradf=[gradf;grad]
%updating the list of gradients
if norm(grad) < e
b=1;
break
else b=0;
end
Epsilon=[norm(grad);e];

beta=(norm(grad)/norm(gradf(counter-1,:)))^2; %updated beta value

descent=-grad +beta*D(counter-1);
%Calculating new gradient
D=[D;descent]; %updating the list of descent directions
%User defined function
falpha = @(a) 25*(x1 + descent(1)*a)^2 + 20*(x2 + descent(2)*a)^2 - 2*(x1 +
descent(1)*a) - (x2 + descent(2)*a);
a0=[0];
%initial starting value for alpha and solving for alpha
[a,fv1] = fminunc(falpha,a0);
Alpha=[Alpha; a];
%Updating new x1 and x2 values and appending to list
x=[(x1 + descent(1)*a);
X=[X;x];

(x2 + descent(2)*a)];

%Updating new function value and appending to list

f = 25*x1^2+20*x2^2-2*x1-x2;
Y=[Y;f];
end
%displaying the counter, list of x1, x2, and function, alpha and epsilon
counter
Alpha
Epsilon
X
Y

ITERATION RESULTS FROM

X(1)
X(2)
3.0000
1.0000
0.0011
0.2097
0.0387
0.0247
0.0400
0.0249
0.0400
0.0250

f
238
0.668
-0.0525
-0.0525
-0.0525

Change in x1 and x2 with iterations

3.5

x1 and x2 values

COMMAND WINDOW:
Epsilon
Alpha
153.0523
7.6418
.0203
0.0662
.0239
0.0025
.0201
.0000
.0241

Change in x1
Change in x2

3
2.5
2
1.5
1
0.5
0
1

Iteration number

Objective function value

Function value vs Iteration

255
240
225
210
195
180
165
150
135
120
105
90
75
60
45
30
15
0
-15

238

0.668

-0.0525

-0.0525

-0.0525

Iteration

Observations:
We see that with conjugate gradient method the solution convergences in 5 iterations
which is only slightly quicker than steepest descent method in this case. This is because the
descent direction is updated more accurately with the calculation of the beta variable.