Lec 7: Total Variation Denoising

Calculus of Variations

A functional is a mathematical operation that maps a
Lecture 7:

function to a real number.
One example of a functional is a definite integral:

Total Variation Denoising
  


The calculus of variations is the study of finding the
Math 490 maxima & minima of functionals.
Prof. Todd Wittman
Many applications:
 Path planning: minimize the distance between 2 points
The Citadel  Physics: systems seek out lowest energy state
 Image processing: minimize noise & blur

Double Integral Shorthand Variational Methods

All things in nature seek out a lower energy state.

A variational method (also known as energy or PDE-based methods)

In image processing, many of our problems will involve prescribes an energy to an image.
minimizing the double integral of an image u:
High energy = "Bad" image Low energy = "Good" image

   ,
 
Bad image f Good image u


where Ω is our rectangular domain.
Energy Minimization

To save a little bit of writing, we will vectorize our

variables and just write a single integral sign:

 () 

By making the energy smaller, we improve the image.

We want an energy that describes the "quality" of an image.

First Variation of Energy Steepest Descent

One way to find the minimum of [] is to take small
step of size ∆ in the direction of −.

For a functional of the form
 =  + ∆ −
  =
 ,  ,  
Ω

According to (Lagrange, 1755), the first variation of energy is
Introducing an artificial time parameter, we can rewrite
given by this as a PDE.
     !"#$ %!"
 = − − = −
   
 ∆&
'!

Note we treat u and its derivatives as separate variables in the = −
'&
calculation.

 acts like a gradient:
 Extrema occur where  = 0 (Euler-Lagrange Equation)
Will this algorithm always guarantee that we find the
  points in the direction of maximum positive rate of change


global minimum?

1
 Lec 7: Total Variation Denoising

H1 Regularization
Convexity

A noise point is a sudden jump, so we

We say our energy E[u] is convex if for any 2 function u expect || to be large near noise.
and v it satisfies:
A classical regularization to smooth a
  + 1 −  ) ≤   + 1 −  [)]
surface is the , norm.

Fact: The local minimum of a convex function is also a
global minimum. [] =
| |- 

So we want to choose convex energies, or else we might

Ω
get stuck at a local min. where Ω is the image domain.

The functionals we look at today will be convex, but next
week we will see what happens when it is non-convex.

First Variation of Energy Steepest Descent

Calculate the first variation of [] = /Ω || - 
So for our energy, the steepest descent gives

= ∆


This is the heat equation!

Minimize the H1 norm

Evolve the heat equation


 - = ∆

Ω

Boundary Conditions The Variational Approach

'! Create energy []

Assume Neumann boundary conditions = 0.
'

Calculate first variation 

'!
Minimize by steepest descent = −
'&

Discretize and code

 =  + ∆ −

The challenge is to cook up better energies
and faster minimization schemes.

2
 Lec 7: Total Variation Denoising

Data Fidelity Data Fidelity

The heat equation blurs the image. If left to run for a
long time, it will eventually give u=const.

To prevent this, we can add a data fidelity term to our min   A =
|| -  + λ
 − A - 
energy to keep our image close to the original image f. Ω Ω

The first variation is easy to modify.
min   A =
|| -  + λ
 − A -   = −2∆ + 2λ  − A
Ω Ω

So the PDE becomes

= ∆ − λ  − A

Regularization Term Data Fidelity Term

The parameter λ balances the two terms. We call λ the

Even with the fidelity term, the , model blurs


fidelity weight or a Lagrange multiplier.

the edges.

Regularization Total Variation

Consider regularization of the form /Ω || E  .
The choice p=1 preserves both sharp and fuzzy

How would this regularization perform on the images below? edges while removing noise.

(Rudin-Osher-Fatemi, 1992) proposed the Total
0 50 100 150 200 250 0 0 0 250 250 250 Variation (TV) denoising model.

min FG  A =
||  + λ
 − A - 

Ω Ω

TV is the most famous and widely used image

restoration model. Its simplicity and adaptability
makes it very useful.

Total Variation
Total Variation Energy
min FG  A =
  + λ
 − A - 

Calculate the first variation of the TV energy Ω Ω

The first variation of energy is
FG  A =
||  + λ
 − A -  
 = − ∙ + 2λ  − A
Ω Ω 
  − 2   +   -
-
= − I/-
+ 2λ  − A
 - +   -

The energy is convex, so we can use steepest descent to evolve

the PDE
   - − 2   +   -
= − 2λ  − A
  -+ -
I/-
 

You will code this up in the next lab!

3
 Lec 7: Total Variation Denoising

Total Variation Minimization Parameter Selection

min FG  A = /Ω | | + λ /Ω  − A - 

The choice of λ is important.
 λ too small -> u will be blurry

TV minimization is good for removing noise from an  λ too large -> u will resemble f
image.
Generally, the more noise we have the smaller we set λ.

There are automatic methods for parameter selection such as the L-curve method
(Vogel, 1996).

Some people recommend using a spatially varying λ (Gilboa-Sochen-Zeevi, 2006).
Noisy λ = 0.01 λ = 0.1 λ=1

50 50 50 50

100 100 100 100

150 150 150 150

200 200 200 200

TV minimization tends to round off the corners of
objects. Why? 250 250 250 250
50 100 150 200 250 50 100 150 200 250 50 100 150 200 250 50 100 150 200 250

Parameter Selection
• As K gets larger, the result resembles the original.

(Getreuer, 2012) Slide taken from Andrea Bertozzi, without permission

Theory Minimization

Existence: A unique minimizer exists in the space of Bounded
Variation (BV), which is where the TV norm is finite.
The technique we used to minimize TV is called

Maximum Principle: The values of the minimizer u will not go
beyond the max/min of the original image f.
Explicit Forward Euler.

Co-Area Formula: The TV norm is equal to the perimeter of the
There exist much faster methods for minimizing
P
level sets /   = /%P LMN  = O O (Giusti, 1984). the TV energy.

Loss of Contrast: A disk of radius R undergoing TV minimization will
 Implicit solvers (Chambolle-Levine-Lucier, 2011)
be decreased by an amount 2/λR (Meyer, 2001).
F  Newton methods (Ng-Qi-Yang-Huang, 2007)
A
 TV filter (Chan-Osher-Shen, 2001)
S
T  Duality (Chambolle, 2005)
E
R  Graph cuts (Darbon-Sigelle, 2005)
 Bregman iteration (Goldstein-Osher, 2009)