One-Dimensional Unconstrained Optimization

1. Golden Section Search

Single-variable optimization has the goal of finding the value of x that yields an
extremum, either a maximum or minimum of f(x).

4
3
Example 1 Golden section method 2
1
Use the golden section method to find the maximum of 0
-1 0 1 2 3 4 5
2
f x   2 sin x 
x -2
within the interval x l  0 and xu  4 -3
10
-4
Solution. First the golden section is used to create the two interior points

Iteration 1

Two interior points x1 and x2 are chosen according to the golden ratio

5 1
d xu  xl   5  1 4  0  2.472
2 2

x1  xl  d  0  2.472  2.472

x2  xu  d  4  2.472  1.528

The function can be evaluated at the interior points

f  x 2   f 1.528  2 sin1.528 
1.5282  1.765
10

f  x1   f 2.472  2 sin2.472 
2.472
2
 0.63
10

The function is evaluated at these two interior points. Two results can occur:
 1. If, as is the case in Fig. 13.4, f(x1) < f(x2), then the domain of x to the left of x2, from xl
to x2, can be eliminated because it does not contain the maximum. For this
case, x2 becomes the new xl for the next round.

2. If f(x2) > f(x1), then the domain of x to the right of x1, from x1 to xupper would have been
eliminated. In this case, x1 becomes the new xu for the next round.

1 6 4 7 5 8 2 9 3
i xlower f(x) x2 f(x2) x1 f(x1) xu f(xu) d Max(Fx1,fx2)
1 0 0 1.528 1.76469 2.472 0.630255 4 -3.1136 2.472 1.764690354

Chart Title
4

0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
-1

-2

-3

-4

Since f(x2) >f(x1) ignore xu and use 2.472 instead of 4. Xlower =0

Iteration 2

Two interior points x1 and x2 are chosen according to the golden ratio

5 1
d xu  xl   5  1 2.472  0  1.527696
2 2

x1  xl  d  0  1.527696 1.527696
 x2  xu  d  2.472  1.527696 0.944304

The function can be evaluated at the interior points

f  x2   f 0.944304  2 sin0.944304 
0.9443042  1.531007
10

f  x1   f 1.527696  2 sin1.527696 
1.5276962  1.764757
10

Since f x1   f x2  , ignore the xlower and use new xlower

xlower  0.944304, xupper  2.472

1 6 4 7 5 8 2 9 3
i xlower f(x) x2 f(x2) x1 f(x1) xu f(xu) d Max(Fx1,fx2)
1 0 0 1.528 1.76469 2.472 0.630255 4 -3.1136 2.472 1.764690354
2 0 0 0.944304 1.531007 1.527696 1.764757 2.472 0.630255 1.527696 1.764757143

0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
-1

-2

-3

-4

Iteration 3

Two interior points x1 and x2 are chosen according to the golden ratio

5 1
d xu  xl   5  1 2.472  0.944304  0.944116
2 2
x1  xl  d  0.944304 0.944116 1.88842

x2  xu  d  2.472  0.944116 1.527884

The function can be evaluated at the interior points

f  x2   f 1.527884  2 sin1.527884 
1.5278842  1.764716
10

f  x1   f 1.88842  2 sin1.88842 
1.88842
2
 1.543347
10

Since f x2   f x1  , ignore the upper value and replace it with x1

xlower  0.944304, xupper  1.88842

4
x2 = 1.527883872,
3 f(x2)= 1.764715891
x lower=0.944304, x1 = 1.888420128,
2
f(x1) = 1.543347362

1 xupper = 2.472,

0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
-1

-2

-3

-4
 The result converging on the true value of 1.7757 at x =1.4276

1 6 4 7 5 8 2 9 3
i xlower f(x) x2 f(x2) x1 f(x1) xu f(xu) d Max(Fx1,fx2)
1 0 0 1.528 1.76469 2.472 0.630255 4 -3.1136 2.472 1.764690354
2 0 0 0.944304 1.531007 1.527696 1.764757 2.472 0.630255 1.527696 1.764757143
3 0.944304 1.531007 1.527884 1.764716 1.88842 1.543347 2.472 0.630255 0.944116 1.764715891
4 0.944304 1.531007 1.304956 1.759453 1.527768 1.764741 1.88842 1.543347 0.583464 1.764741394
5 1.304956 1.759453 1.52784 1.764726 1.665537 1.71363 1.88842 1.543347 0.360581 1.764725637
6 1.304956 1.759453 1.442698 1.775475 1.527795 1.764735 1.665537 1.71363 0.222839 1.775475488
7 1.304956 1.759453 1.390081 1.774198 1.442671 1.775476 1.527795 1.764735 0.137714 1.775476393
8 1.390081 1.774198 1.442688 1.775476 1.475188 1.773248 1.527795 1.764735 0.085107 1.775475834
9 1.390081 1.774198 1.422592 1.775699 1.442677 1.775476 1.475188 1.773248 0.052596 1.77569885
10 1.390081 1.774198 1.410173 1.775397 1.422585 1.775699 1.442677 1.775476 0.032505 1.77569878
11 1.410173 1.775397 1.422589 1.775699 1.43026 1.775718 1.442677 1.775476 0.020088 1.775717657
12 1.410173 1.775397 1.417846 1.775623 1.422587 1.775699 1.43026 1.775718 0.012414 1.775698797
13 1.417846 1.775623 1.422588 1.775699 1.425518 1.775721 1.43026 1.775718 0.007672 1.775721147
14 1.417846 1.775623 1.420777 1.775676 1.422587 1.775699 1.425518 1.775721 0.004741 1.775698803
15 1.420777 1.775676 1.422588 1.775699 1.423707 1.77571 1.425518 1.775721 0.00293 1.775709547
16 1.420777 1.775676 1.421896 1.775691 1.422588 1.775699 1.423707 1.77571 0.001811 1.775698805
17 1.421896 1.775691 1.422588 1.775699 1.423015 1.775703 1.423707 1.77571 0.001119 1.775703231
18 1.421896 1.775691 1.422324 1.775696 1.422588 1.775699 1.423015 1.775703 0.000692 1.775698806
19 1.422324 1.775696 1.422588 1.775699 1.422751 1.775701 1.423015 1.775703 0.000427 1.775700543