02-24-01410 - Non-Linear

and Combinatorial
Optimization
DR. AHMED TAYEL
De p a rtment o f E n gi neering M at h em ati cs a n d P hysi cs, Fa c ul ty o f
E n gi neering , Al exandria Un i ve rsi ty
a h m e d .taye l @ a l ex u . e d u . e g
 Calculus-based optimization: Part 2
1. Mathematical background.
2. Calculus-based unconstrained optimization.
1. 1- variable case. (Previous lecture)
2. Multi-variable case. (This lecture)
1. Mathematical background
What is a matrix?
a rectangular array of elements, arranged in rows and
columns.

Some types of matrices

Row Column Square Identity Symmetric
vector vector matrix matrix matrix
Matrix arithmetic operations

Addition

Subtraction

Multiplication

𝑚×𝑛 𝑛×𝑝 𝑚×𝑝

Determinant of a square matrix?
Principal submatrix and principal minor?

3rd order 2nd order 1st order

principal minor principal minors principal minors
𝑎22 𝑎23 𝑎33
𝑎11 𝑎12 𝑎13 𝑎11 𝑎12 𝑎13 𝑎32 𝑎33
𝐴 = 𝑎21 𝑎22 𝑎23 𝑎21 𝑎22 𝑎23
𝑎11 𝑎13 𝑎22
𝑎31 𝑎32 𝑎33 𝑎31 𝑎32 𝑎33
𝑎31 𝑎33
𝑎11 𝑎12 𝑎11
𝑎21 𝑎22
Leading principal submatrix and leading principal minor?

3rd order leading 2nd order leading 1st order leading

principal minor principal minors principal minors

𝑎11 𝑎12 𝑎13 𝑎11 𝑎12 𝑎13

𝑎11 𝑎12
𝐴 = 𝑎21 𝑎22 𝑎23 𝑎21 𝑎22 𝑎23 𝑎11
𝑎21 𝑎22
𝑎31 𝑎32 𝑎33 𝑎31 𝑎32 𝑎33
Positive definite/negative definite/semi-definite/indefinite matrix?
𝑨𝟐 𝑨𝟑
𝑨𝟏

𝑎11 𝑎12 𝑎13

𝐴 = 𝑎21 𝑎22 𝑎23
𝑎31 𝑎32 𝑎33
Positive definite/negative definite/semi-definite/indefinite matrix?
Examples:

1 1
1 1

0 2
2 3
2. Calculus-based unconstrained
optimization
(multi-variable case)
Calculus-based unconstrained optimization
(multi-variable case)
 Classification of the stationary points

Positive definite → Local minimum

Negative definite → Local maximum

𝐻 Indefinite → Saddle point
Symmetric
matrix
Semi-definite → Test inconclusive
Example:
Determine and classify the stationary points of the function
𝑓 𝑥, 𝑦 = 𝑥 3 + 𝑦 3 + 2 𝑥 2 + 4 𝑦 2 + 6
 6𝑥+4 0
𝐻=
0 6𝑦+8
𝐻ቚ =
(0,0)

𝐻ቚ 8 =
(0,−3)

𝐻ቚ 4 =
(−3,0)

𝐻ቚ 4 8 =
(− ,− )
3 3
Example:
Maximize 𝑓 𝑥, 𝑦, 𝑧 = 𝑥 + 2 𝑧 + 𝑦 𝑧 − 𝑥 2 − 𝑦 2 − 𝑧 2