Non-Linear Programming

Summary
TMR4115

Prof. Stein Ove Erikstad

Reality

Decision Models
Design Methods Prof. Stein Ove Erikstad

Agenda

Intro non-linear programming

non-linear objectives and constraints

Classes of non-linear optimization models

One-variable unconstrained optimization

a simple search algorithm

Excel example

Multi-variable unconstrained optimization

gradient search

example by hand

Optimality conditions

concave and convex functions

conditions for optimality unconstrained problems

KKT-conditions for constrained problems

2
Design Methods Prof. Stein Ove Erikstad

Classes of non-linear
problems
Unconstrained
Linearly constrained
Quadratic
linearly constrained
quadratic objective function (e.g. the square of a
variable, or the product of two variables)

Convex programming
Separable programming

f(x), g(x) separable

linear approximation -> use simplex

Nonconvex programming
geometric programming
fractional programming

3
Design Methods Prof. Stein Ove Erikstad

One-dim unconstrained
search
DIRECTION: Left or Right
based on the sign of the derivative

STEP-LENGTH
Bi-section: Midpoint of lower and upper
bound
3000

2500

2000

Series1

1500

Series2

1000

500

0
1

10 11 12 13 14 15 16 17 18 19 20

Design Methods Prof. Stein Ove Erikstad

Multi-Varible,
Unconstrained Search
DIRECTION: The Gradient f(x)
direction of steepest ascend

STEP-LENGTH: t*
t* is the value that maximises f along
the gradient

Initial
solution
Design Methods Prof. Stein Ove Erikstad

Multi-Varible,
Unconstrained Search
Algorithm:
1. Find the next point x as a function
of t, x = x+ t f(x)
2. Find the t that maximises f(x) b y
differentiating wrt t

Initial
solution
Design Methods Prof. Stein Ove Erikstad

Necessary and sufficient

conditions for optimality

Problem

Necessary conditions

Also sufficient if

One-variable
unconstrained

df/dx = 0

f(x) concave

Multivariable
unconstrained

f/ x j = 0 , j= 1 ,2 , , n

f(x) concave

Constrained,
nonnegativity
constraints only
General
constrained
problem

f/ x j = 0 , j= 1 ,2 , , n
or 0 if xj = 0

KKT conditions

f(x) concave

f(x) concave
gi(x) convex
7

Design Methods Prof. Stein Ove Erikstad