1.4 Convex Analysis for subgradients
A number of facts regarding nonsmooth convex functions should be noted.
•
A convex function is always subdiﬀerentiable i.e. a subgradient of a convexfunction exists at every point
•
Directional derivatives also exist at every point.
•
If a convex function
f
is diﬀerentiable at
x
, its subgradient is a singletonset containing only the gradient at that point. i.e.
∂f
(
x
) =
{
f
(
x
)
}•
Let
f
(
x
;
d
) denote the directional derivative of
f
in the direction
d
andlet t
∈
R
. We note that, from deﬁnition
f
(
x
+ t
d
)
−
f
(
x
)t
≥
g
T
d
∀
g
∈
f
(
x
) (3)So, Subgradients are “lower bounds” for directional derivatives.
•
In fact,
f
(
x
;
d
) = sup
g
∈
∂f
(
x
)
g,d
•
Further,
d
is a descent direction iﬀ
g
T
d <
0
∀
g
∈
∂f
(
x
)
1.5 Properties
Just as we have an arithmetic for gradients, we have an arithmetic for Subgradients. Rigorous proofs for these results are presented in [9]Properties:
•
∂
(
f
1
+
f
2
)(
x
) =
∂f
1
(
x
) +
∂f
2
(
x
)
•
∂αf
(
x
) =
α∂f
(
x
)
•
g
(
x
) =
f
(
Ax
+
b
)
⇒
∂g
(
x
) =
A
T
∂f
(
Ax
+
b
)We have maxima and minima conditions in diﬀerential calculus (namely if a point is an extremal point, then its derivative is 0)The extension of this lemma to nonsmooth functions is :Local extremum
⇒
0
∈
∂f
(
x
)We note that this is not very useful in practice. This is because the derivativemay not vary continuously, i.e. its value at a point may not be representativeof its values nearby.This makes ﬁnding minima for general nonsmooth functions impossible. Aswe shall see, we can us the convexity to ﬁnd the location of the minima (as itis unique).As an example, consider the case
f
(
x
) =

x

, the oracle returns subgradient0 only at
x
= 0. So, the magnitude of the subgradient is no indication of howclose we are to the ﬁnal solution. During an iterative method, we may never hitzero, though we come arbitrarily close.3