Image Analysis and

Markov Random Fields (MRFs)

 Statistical models
• Some image structures are not deterministic, are best
characterized by their statistical properties.
•For example, textures can be represented by their
first and second statistics.
•Images are often distorted by statistical noise. To
restore the true image, images are often treated as
realizations of a random process.
Uses of Markov Random Fields
• MRFs are a kind of statistical model.
• They can be used to model spatial
constrains.
– smoothness of image regions
– spatial regularity of textures in a small region
– depth continuity in stereo construction
 What are MRFs
• Neighbors and cliques

Let S be a set of locations, here for simplicity, assume S

a grid.
S={ (i, j) | i, j are integers }.
Neighbors of s(i,j) ∈ S are defined as:
∂((i, j)) = { (k, l) | 0<(k - i)2 + (l - j)2 < r constant }

r is constant
• A subset C of S is a clique if any two different elements
of C are neighbors.

The set of all cliques of S is denoted by Ω.

 Examples of neighborhood
• 4-neighborhood (r = 1)

cliques:
 Examples of neighborhood
• 8-neighborhood (r = 2)

cliques:
 Random fields
• The random vector X = ( X s )on
s∈S S is called a random
field and assumed to have density p(x).

• Images as Random fields:

If vector X represents intensity values of an image, then
its component Xs is the intensity value at location s=(i, j).
…

S= … X=

640x480 640x480

Vector X: 640×480 Dimension

 Markov Random Fields
• If p(x) of a random field fulfills the so called Markov
condition w.r.t. a neighborhood system, it is called a
Markov Random Field.

p ( X s = xs | X t = xt , ∀t ≠ s ) =
p ( X s = xs | X t = xt , ∀t ∈ ∂{s})

I.E, the value Xs at location S is only depend on its

neighbors.
 Markov Random fields

• p(x) can also be factorize over cliques due to its

Markov properties. i.e.

p ( x) = ∏ Ψc ( x)
C∈Ω

ΨC is a function of X determined by clique C.

 Markov Random Fields
• MRFs are equivalent to Gibbs Fields and p(x) has the
following form:

exp(− H ( x)) −1
p ( x) = = Z exp(− H ( x))
∑ exp(− H ( y))
y

• H(x) is called energy function.

• The summation in the denominator is over all possible
configurations on S. In our case are over all possible
images. For 256 grey values and 640x480 grid, it will
have 256640x480 terms. Z is impractical to evaluate.
• so Z is only known up to a constant.
 Local Characteristics of MRFs
• For every I ⊂ S , we have,

P ( X I = yI | X S \ I = xS \ I ) =
−1
Z I exp(− H ( yI xS \ I ))

S\I means complement of I

• If I is a small set, since X only changes over I, ZI can be
evaluated in reasonable time.
• So p(yI|xS\I) is known.
 Using MRFs in Image Analysis
• In image analysis, p(x) is often the
posterior probability of Bayesian inference.
That is, p(x) = p(x|y0).
• For example:
y0 may be the observed image with noise,
and we want to compute the estimate x0* of
the true image x0 based on p(x) = p(x|y0).
 Using MRFs in Image Analysis
MRF Model (either learned or known)

y0 = ⇒ p ( x | y0 )

Sampling

⇒ x = *
0

X0
 Difficulties in computing X0 *

• A way to compute the estimate X0* is to let,

x = E ( x) p ( x ) = ∫ xp( x | y0 )dx
*
0

• But p(x|y0) is only known up to a constant Z, How to do

above integration?
 Monte Carlo integration
• One solution is to construct a Markov chain having p(x) as
its limiting distribution.
• If the Markov chain starting at state X0, and going through
states X1, X2, X3,…… , Xt,……, then E(X)p(x) can be
approximated by
n
1
E ( x) p ( x ) ≈ ∑
n − m t = m +1
X t

• m is a sufficiently long burn-in time. Xm+1, Xm+2,...... can be

considered as samples drawn from p(x).
 Gibbs Sampler
• Because X is a high dimension vector. (For a
640x480 image its dimension is 640x480).
• It is not practical to update all components of Xt
to Xt+1 in one iteration.
• One version of Metropolis-Hastings algorithm,
called Gibbs Sampler, builds the Markov chain
and updates only a single component of Xt in
one iteration.
 Gibbs Sampler Algorithm
• Let the vector X has k components, X=(X0,X1,X2,
……,Xk). and presently it is in state Xt= (x0,x1, x2,
……,xk).
• An index that is equally likely to be any of 1,……,k
is chosen. say index i.
• A random variable w with density
P{ w=x} = P{ Xi=x | Xj = xj, j ≠ i } is generated.
• If w=x, the updated Xt is Xt+1 = (x0, x1, x2, …, xi-1 , x,
xi+1 , …, xk).
 Two aspects of using MRFs
1. Find an appropriate model class, the
general form of H(x).
2. Identify suitable parameters in H(x) from
observed samples of X.
• This is the most difficult part in applying
MRFs.
 An Example
• Suppose we want to restore a
binary (+1/-1) image with
pepper-and-salt noise added. y0 =
• The Ising model is chosen.

H ( x, y0 ) = −β ∑ xs xt + h∑ xs +c ∑ xs y0s
s~ t s s
 Open Issues / Discussion
• Code Development
– What should our MRF library look like?

• Challenges:
Build MRF model from image samples and then
generate new images using Gibbs sampler
– Need a way to determine the parameters in
H(x) based on image samples.
 My Idea
Image Segmentation in Wavelet domain
with Markov Random Fields:
1. Wavelet transform.
2. Image enhancement.
3. Extract edge information.
4. Build MRF model based on image samples.
5.Image segmentation using Markov
Random Fields scheme.
 Reference
1. Ross Kindermann and J. Laurie Snell, Markov
Random Fields and Their Applications,
http://www.ams.org/online_bks/conm1/, 1980.
2. Gerhard Winkler, Image Analysis, Random Fields and
Markov Chain Monte Carlo methods. Springer, 2003
3. W. R. Gilks, Markov Chain Monte Carlo in Practice,
Chapman & Hall/CRC, 1998.
4. Sheldon M. Ross, Introduction to Probability Models,
Academic Press, 2003.