Linear And Non-Linear Programming Unconstrained Optimization Homework Name 1.

List the pros and cons of the Steepest Descent Algorithm

2. For the function f = x12 - 2x1x2 + 4x22, starting at the point [4, -1], use the steepest descent algorithm to perform one complete iteration of the algorithm, including the line search. Begin with = 0.05. Show your work. Create a contour plot and plot the path of the algorithm on the contour plot.

3. For the function f = x12 - 2x1x2 + 4x22, starting at the point [4, -1], use the steepest descent spreadsheet macro on Moodle to perform 6 iterations of the algorithm. Create a contour plot and plot the path of the algorithm on the contour plot. 4. List the pros and cons of the Newtons Method algorithm.

5. Perform up to three steps of the Newtons Method algorithm to minimize: f = x14 - 2x2x12 + x22 + x12 -2x1 + 5. Start at the point [4, -1]. Show your work. Create a contour plot and plot the path of the algorithm on the contour plot.

6. If we are beginning the Quasi-Newton method to minimize the function: f = x12 2x1x2 + 4x22, and if we start with a direction matrix N = I (the identity matrix), and if point x0 = [3,3]T, and point x1 = [2,2]T, use the rank one Hessian inverse update formula to find the new direction matrix.

7. What is the search direction for Steepest Descent? 8. What is the search direction for Newtons method? 9. What is the search direction for the Quasi-Newton method? 10. For the Quasi-Newton method, prove that if the direction matrix, N, is positive definite, then the search direction, s, will always point downhill.

11. For the Quasi-newton Method, explain how the direction matrix can become the Hessian inverse if we never evaluate second derivatives, and why we need n steps to do this?