Professional Documents
Culture Documents
Problem 1. Identify with justification if the given point (x̄1 , x̄2 ) is a saddle point or a local minimum or a
local maximum of the following functions:
(a) f (x1 , x2 ) = 4x41 + x1 x22 − 2x21 , (x̄1 , x̄2 ) = (0, 0)
x1 +x2
(b) g(x1 , x2 ) = x21 +x22 +1
, (x̄1 , x̄2 ) = (− √12 , − √12 )
Solution. (a) As fx1 (0, 0) = 0 = fx2 (0, 0), (0, 0) is a stationary point.
−4 0
The Hessian matrix of f at (0, 0) is , which is negative semidefinite. So, no conclusion can be
0 0
made from the Hessian matrix.
p
Denote Nδ (0, 0) = {(x1 , x2 ) ∈ R2 : x21 + x22 < δ}.
4
Notice that for any 0 < δ < 1, we have δ4 , √δ2 ∈ Nδ (0, 0), √δ2 , 0 ∈ Nδ (0, 0) and
δ 4 √δ δ 16
f 4 , 2 − f (0, 0) = 64 > 0 and f √δ , 0
2
− f (0, 0) = δ 2 (δ 2 − 1) < 0.
p p p
Notice that for any (x1 , x2 ) in R2 , x1 + x2 ≥ − (x1 + x2 )2 = − x21 + x22 + 2x1 x2 ≥ − 2x21 + 2x22 ,
and hence we have
√ √
p
x1 + x2 x21 + x22 ξ 1
f (x1 , x2 ) = 2 ≥ − 2 ≥ − 2 max = − √ = f − √1 , − √1 .
x1 + x22 + 1 x21 + x22 + 1 ξ≥0 ξ 2 + 1 2 2 2
Thus, − √12 , − √12 is a point of (global) minimum of g.
x21 x32
Problem 2. (a) Find the directions in which the function f (x1 , x2 ) = 2 + 4
1
(ii) decreases most rapidly at (0, 1).
(iii) What are the directions of zero changes of f at (1,1)?
(b) Consider the following function and check if it is differentiable:
( p
x1
|x1 | x21 + x22 if x1 =
6 0
f (x1 , x2 ) =
0 if x1 = 0.
Does this function have directional derivative at (0, 0) along every direction? Justify your answer.
Solution. (a) Given that f (x1 , x2 ) = 21 x21 + 14 x32 . The gradient of the function f is given by:
" #
∂f
∂x1 x1
∇f (x1 , x2 ) = ∂f = 3 2 .
∂x 4 x2
2
(iii) Let u
b = (u1 , u2 ) be a unit direction. Then, the directional derivative of f along the direction u
b is
given by
Dub f (x1 , x2 ) = ∇f (x1 , x2 )> u
b.
Direction of zero changes of f at point (1, 1) is calculated by solving the following equation:
∇f (1, 1).b
u=0
=⇒ (3bi + 4b
j).(u1bi + u2b
j) = 0
3
=⇒ 3u1 + 4u2 = 0 =⇒ u1 = − u2 .
4
Also, u
b is unit direction. Therefore,
u21 + u22 = 1
9
=⇒ u21 + u21 = 1
16
4
=⇒ u1 = ± .
5
Hence, the directions of zero changes of f at (1, 1) are ( 54 , − 35 ) and (− 45 , 35 ).
(b) Given that (
x1
p
|x1 | x21 + x22 if x1 6= 0
f (x1 , x2 ) =
0 if x1 = 0.
For the unit vector u
b = (u1 , u2 ), the direction derivative at (0, 0) is given by
f (0 + tu1 , 0 + tu2 ) − f (0, 0)
Dub f (0, 0) = lim
t→0 t
f (tu1 , tu2 )
= lim
t→0
( t
0, if u1 = 0
= u1
|u1 | , if u1 6= 0.
2
Therefore, the directional derivatives in all directions exist.
h
√
|h|
h2 +k2 −h
Note that fx1 (0, 0) = 1 and fx2 (0, 0) = 0. Note that (h, k) = √
h2 +k2
6→ 0 as (h, k) → (0, 0). For
h √1 )
example, h = k gives ( |h| − 2
6→ 0 as h → 0. Therefore, the function f is not differentiable at (0, 0).
Problem 3. Suppose g : Rn → R is twice continuously differentiable and there exists x̄ and d in Rn such
that g(x̄) = g(x̄ + d). If the matrix ∇2 g(x̄ + αd) is positive definite for all α ∈ (0, 1), then prove that there
exists γ ∈ (0, 1) which is a local minimum point of θ(α) = g(x̄ + αd). Is x̄ + γd a local minimum point of g?
Justify your answer.
Solution. By assumption, ∃ x̄ and d in Rn such that g(x̄) = g(x̄ + d) and ∇2 g(x̄ + αd) is PD for all α ∈ (0, 1).
Also, given that θ(α) = g(x̄ + αd) and g : Rn → R is twice continuously differentiable. By mean value
theorem,
equivalently,
γ is local minimizer of θ.
Let (a11 , a12 , a13 ), (a21 , a22 , a23 ) be any two arbitrary points from the set S1 . The set S1 is convex if for
any λ ∈ [0, 1],
3
Since (a11 , a12 , a13 ) and (a21 , a22 , a23 ) ∈ S1 ,
√
|φ1 (t)| ≤ 1 + 2, where φ1 (t) = 1 + a11 t + a12 t2 + a13 t3
√
and |φ2 (t)| ≤ 1 + 2, where φ2 (t) = 1 + a21 t + a22 t2 + a23 t3 .
Denote
Corresponding to x, φx (t) = 1 + (λa11 + (1 − λ)a21 )t + (λa12 + (1 − λ)a22 )t2 +(λa13 + (1 − λ)a23 )t3 .
Also,
then its level set {x ∈ C : f (x) ≤ γ}, for any γ ∈ R, is convex. Here
If f is a convex function, P
n
f (x) = (x> Qx + 1)2 + log( i=1 e−xi ) and γ = 5. Hence, set S2 is convex.
Solution. (a) This function is not coercive since it does not possess the limit property of a coercive function.
Let φ(t) = −t + 2 log |t|, for t 6= 0. Then, f (x, y) = φ(x) + φ(y). Here, φ(1) = −1 and
t2
lim φ(t) = lim log = −∞.
t→∞ t→∞ et
So, lim f (x, 1) = lim (φ(x) − 1) = −∞ =
6 ∞.
x→∞ x→∞
4
1
1
(b) f (x) = x2 on 2, 2 .
Here,
3√
(
0 − √1x cos x1 + 1
2 x sin x , if x 6= 0
f (x) =
0 if x = 0
|f (x1 )−f (x2 )|
and hence sup |x1 −x2 | = sup |f 0 (ξ)| = ∞. Therefore, f is not Lipschitz on [0, 1].
x1 ,x2 ∈[0,1] ξ∈[0,1]
1
(b) f (x) = x2 .
Problem 7. Let A be a matrix of order m × n, b ∈ Rm . Prove or give a counter example for the following
statement.
‘Exactly one of the following systems has a solution:
System 1: ∃x ∈ Rn that satisfies Ax ≤ b
System 2: ∃z ∈ Rm that satisfies z ≥ 0, AT z = 0, bT z < 0’.
Solution. Recall the Farkas’ Lemma: exactly one of the following two systems has a solution—
Notice that given System 1 and System 2 cannot be both true. Otherwise, we have the following impos-
sibility:
0 = xT (AT z) = (Ax)T z ≤ bT z < 0.
Next, we prove that if System 2 does not have a solution, then System 1 must have a solution. This will
prove, contrapositively, that if System 1 has no solution, then System 2 must have a solution.
T T
If System 2 does not hold, then for all z ≥ 0 with
TA z = 0, we have b z ≥ 0.
A z 0
Denote Ā = [A b]. Then, for all z ≥ 0, ĀT z = T = n×1 .
b z ≥0
T 0n×1
Therefore, @z ≥ 0 such that Ā z = = c̄, say. Which shows that System II of Farkas’ lemma does
−1
not have a solution. Thus, System I must have a solution, i.e.,
x
∃ x̄ = ∈ Rn × R such that Āx̄ ≤ 0 and c̄T x > 0
α
i.e., Ax + bα ≤ 0 and − α > 0
x
i.e., A −α ≤ b,
Problem 8. Let function x1 e−(x1 +3x2 ) be convex on the set {(x1 , x2 ) ∈ R2 : ax1 + bx2 + c ≥ 0} for some
constants a, b, c ∈ R. Identify the value of a − 9b + c.
5
Solution. Given
Now,
Also, the nonconvexity of f does not depend on the choice a, b and c. Therefore, we can not identify the
value of a − 9b + c.
∗ ∗ ∗ ∗ ∗∗