154 (IJCNS) International Journal of Computer and Network Security,

Vol. 2, No. 5, May 2010

Color Image Segmentation Using Double Markov

Random Field (DMRF) Model with Edge Penalty
Function
Sami M. Halawani1 , Ibrahim A. Albidewi2, M. M. Sani3 and M. Z. Khan3
1,2
Faculty of Computing and Information Technology-Rabigh Campus
King Abdulaziz University
Saudi Arabia
Halawani@kau.edu.sa, Ialbidewi@kau.edu.sa
3
Faculty of computer Science and Info. System
Jazan University K.S.A
sani.nitrkl@gmail.com
&
Department of Computer Science and Engineering, IIET, Bareilly India
zubair.762001@gmail.com,

Abstract: In our paper color image segmentation using DMRF
model with edge penalty function as image model and for color
(
model using RGB and OHTA I1 , I 2 , I 3 ) color model. In our
paper we are proposing Hybrid algorithm based on DMRF
model and The image segmentation problem is formulated as a
pixel labeling problem and the pixel labels are estimated using
Maximum a Posterior(MAP) criterion. The MAP estimate of the
labels are obtained by the proposed hybrid algorithm converges
to the solution faster than that of Simulated Annealing(SA)
algorithm. The DMRF with edge preserving attribute yields
better segmentation result with local edge boundary. It is also
found that Exponential function improved the result to some
extend. The paper is divided in six section in second section we
explain the model use for Hybrid Algorithm.
Keywords: DMRF, MAP, RGB-Plane, Edge penalty function.
1. Introduction & Background
Image segmentation is an important process in automated
image analysis. In color image segmentation, choice of
proper color model is one of the important factors for
segmentation and different color models such as RGB, HSV,
( )
YIQ, OHTA I1 , I 2 , I 3 , CIE (XYZ, Luv, Lab) are used to
represent different colors [1],[2][3]. Besides color model,
image model does play a crucial role for color image
segmentation. Stochastic models, particularly Markov
Random Field (MRF) models, have been extensively used as
the image model for image restoration and segmentation
problems [4][5]. Often, in stochastic framework, the color
image segmentation problem is formulated as a pixel
labeling problem and the pixel labels are estimated using the
Maximum a Posteriori (MAP) criterion[6][7]. By and large,
the MAP estimates of the pixel labels are obtained using
Simulated Annealing (SA) algorithm [11][8]. MRF model
has also been successfully used as the image model while
addressing the problem of color image segmentation in
supervised and unsupervised framework [9]. Here we use,
color image segmentation problem is formulated in
stochastic framework. Markov Random Field model is used
as the image model and RGB and OHTA model have been
used as the color models. we proposed a Double MRF
(DMRF) model that takes case of intra-plane interactions as
well as the inter-plane interactions[10]. These proposed
models have been tested on different images and also the
results obtained have been compared with the results
obtained using only MRF. Both first order and second order
models have also been studied. As expected, it has been
observed that the use of OHTA model yielded better results
than that of using RGB model We have used line process in
the clique potential function and it has been found that
although the line process could preserve well defined edges
it failed to preserve loosely defined boundaries [11][12]. In
order to preserve loosely defined boundaries, we have
employed the edge penalty model proposed by Qiyao Yu et
al [11][12]. Here, we have used RGB color models with
MRF and DMRF models. Qiyao Yu et al [11][12] have used
exponential edge penalty function preserve the weak edges
together with strong edges. We have proposed the
exponential edge penalty function in the DMRF model to
preserve edge. Here also, the problem is formulated as a
pixel labeling problem and the pixel label estimates are the
MAP estimates.
2. Methodology
Here we are proposing the model for our algorithms. The
image segmentation problem is formulated as a pixel
labeling problem. The labels are estimated using the MAP
estimation criterion usually, the MAP estimates are obtained
by Simulated Annealing (SA) algorithm. It is observed that
SA algorithm is computationally involved and hence we
have proposed a hybrid algorithm. Which is found to be
faster than that of SA algorithm. The proposing models are
as fallow.
2.1 Model 1 (Double Markov Random Field (DMRF)
Model)
The DMRF IS DEFINE AS In order color planes, the a prior
 (IJCNS) International Journal of Computer and Network Security, 155
Vol. 2, No. 5, May 2010

model takes case of intra-plane interactions and inter-plane observed image X which is assumed to be MRF is
interactions as well. This MRF model taking case of intra-plane considered as a degraded version of the label process and
and inter-plane interaction is called Double MRF (DMRF) model. hence consists of Z and W. The model parameter for both
Capturing salient spatial properties of an image leads to the inter and intra color plane processes are selected on a trial
development of image models. MRF theory provides a and error basis. The weak membrane model is considered to
convenient and consistent way to model context dependent the model intra as well as inter plane clique potential
entities for eg. image pixel and correlated features. Through function. For example, the weak membrane model for
the MRF model takes into account the local spatial inter-plane interaction process for two color components,
interactions, it has limitation in modeling natural scenes of
( I1 and I 2 ) (as shown in Fig.1(b) ) can be expressed as
distinct regions. In case of color models, it is known that
 I
( )
there is a correlation among the color components of RGB 2 2
1 I
Vc z = α  n '2 ' − n '1 '
 +  n I 2 − n I1  +
model. In our formulation, we have de-correlated the color   i' , j' i' , j' −1 


i , j i −1, j   
components and introduced an interaction process to
improve the segmentation accuracy. We have employed 2 2
 n I 2 − n I1   I I  
inter-color plane interaction (OHTA ( I1, I 2 , I 3 ) color  i' , j ' i' , j ' +1  +  ni'2, j ' − ni'1+1, j '  
    
model) process which reinforces partial correlation among 
different color components. Where α is the MRF field parameter. Superscripts I1 and I 2
We assume all images to be defined on discrete on corresponds to I1 -plane and I 2 -plane respectively. This
rectangular lattice M1 × M 2 . Let Z denote the label process notation is depicted in Fig.1, where the intra plane and inter
corresponding to the segmented image and z is a realization plane interaction are shown. The model parameter both Z
of the label process i.e. the segmented image. The observed and I processes are selected on an Ad hoc manner. The
image X is assumed to be a random field that is assumed to weak membrane model for intra plane as well as inter plane
be the degraded version of the label process Z and this clique potential function.
degradation process is assumed to be Gaussian process W.
The label process Z in a given plane is assume to be MRF.
The prior model takes care of inter as well as intra color
plane interactions. The prior probability distribution of z is
constructed with clique potential function corresponding to
intra-color plane interaction, for example I1 and another
clique potential function within inter plane interactions (for
example between I1 and I 2 ).The of inter color plane
interaction are shows in Figure1.Thus X is a compound
MRF. MRF model taking care of both inter as well as intra
plane interactions. It known that if Z is assumed MRF, then
the prior probability distribution P(Z=z) is Gibb’s
distributed that can be expressed
as P (Z = z | θ ) = 1 e −U (z ,θ ), where Z ' = ∑ z e ( ) is the
−U z ,θ Figure 1. (a) RGB Plane Interaction
'
Z
partition function, θ denotes

the clique parameter vector, the exponential term

U(z, θ ) is the energy function and is of the form
U ( z,θ ) = ∑ c ∈C Vc ( z,θ ) with Vc ( z,θ ) being referred as the
clique potential function. Since the inter-plane process is
viewed to be MRF, we know
 I I I I 
that P  Z 2 = z 2 | Z 1 = z 1 , (k, l ) ≠ (i, j ), ∀(k, l ) ∈ I 
 
 i, j i, j k, l k, l 1

 I I I I I 
= P  Z 2 = z 2 | Z 1 = z 1 , (k , l ) ≠ (i, j ), (k, l ) ∈ η 1  Where
 i , j i , j k, l k, l i, j 
  Figure 1. (b) Interaction of one plane of R-plane with G-
I1 and I 2 denotes color planes respectively. In other words a plane
pixel in one plane (say for e.g I1 -plane) is assumed to have
2.2 Model 2 (Segmantation in MRF-Map Framework
interaction with pixel of I 2 and I 3 planes, the interaction Using Edge Penalty Function)
process of each color plane is shown in Fig. 1(a). For the Let Y denote the observed data (i.e the original image) and
sake of illustration, the interaction of (i, j )th pixel in I 2 X the desired segmentation. The following optimality
criterion is used
plane with the neighboring pixels of I 1 plane for a first
order neighborhood system is also depicted in Fig. 1(b). The
156 (IJCNS) International Journal of Computer and Network Security,
Vol. 2, No. 5, May 2010

x = arg max P ( X = x| Y = y)
)
Where U i (s ) is the number of neighbors of s that does not
(1) '
x
Where P ( X = x|Y = y) the posterior probability is x is the belongs to Ω i . That means the term βU i (s ) is not zero
) '

MAP estimation using Baysian rule. A 2-layer model is only when the site s is at the boundary. Therefore, (4)
constructed from Bayes’ rule. becomes
P (Y = y | X = x)P ( X = x) [ ]
P ( X = x | Y = y) =  Y(s) −µi 
2
n  n
P (Y = y) +β ∑ ∑ U (s)
(2) ) '
x= argmin∑ ∑ logσi +
2  i=1s=Ω i
(6)
Where P (Y = y| X = x) is the posterior conditional
Ωni=1s=Ωi 
Ω1,....
 2 σ i ( )
 i
probability of the segmentation given the observed data,
P (Y = y| X = x)P ( X = x) is the conditional probability ∑in=1∑s =Ωi U i (s ) is approximately proportional to the
'
distribution of the observation conditioned on the length of the obtained boundary. Therefore the role of the
segmentation, P ( X = x) is the prior probability of the region model term actually penalizing the existence of the
segmentation , and P (Y = y) is the probability is the
boundary by its population.
Instead of penalizing of all boundary pixel, greater penalty
distribution of the observation. Maximization of the can be applied to weak edge pixel and a lesser penalty to
posterior P ( X = x|Y = y) gives a segmentation solution, strong edge pixels, so that local statistics such as edge
and since P (Y = y) is a constant it is equivalent to strength can be incorporated. Therefore we can replace the

maximizing P (Y = y | X = x)P ( X = x) describes the grey

penalty term with some monotonically decreasing function
( )
of edge strength g ∇(.) , where ∇(.) is the gradient
level, texture or any other factors while P ( X = x) can magnitude. The proposed objective function is therefore
incorporate spatial context constraint. Two separate models  [Y(s) −µi ]2  n
+ β ∑ ∑ g( ∇(Y(s)) ) (7)
) n
a feature distribution model and a region model are needed x = argmin∑ ∑ logσi +
2  i=1 s=Ωi
to obtain the analytical expression for the P (Y = y| X = x) Ω1,....Ωn i=1 s=Ωi 
 2(σi )  
and P ( X = x) respectively. For the region model, a multi-
level logistic model (MLL) is typically used, whose clique
potential is defined as;
β c :if all sites on c equal
Vc ( X ) =  (3)
0 : otherwise
The term P (Y = y| X = x) is typically assumed to be
Gaussian at each site. Maximizing the posterior is
equivalent to minimizing the objective function of the
equation
n  [
Y(s) − µi
2
]
 Figure 2.Exponential edge penalty function
− βUi (s) (4)
)
x = arg min ∑ ∑ logσ i +
( )
The method of penalizing differently on the edge strength
Ω1,....Ωn i=1 s=Ωi 
2
 2 σi  has the shortcoming of bias on certain classes since
Whereη is the number of classes Ω ,....., Ω are the obtained boundaries between some classes can be generally weaker
1 n than those of other. However, including edge strength is
Classes, µ i is the mean grey level of class Ω i , σ i is standard advantageous in describing local behaviors, and will also
show next that the bias problem can be alleviated by
deviation of class Ω i ,and U i (s ) is the number of neighbor properly manipulating the edge penalty function g(.).
of s that belongs to Ω i . b. Edge Penalty Functions
The edge penalty function as visualize in figure 2. Where
2.2.1Edge Penalty Based Segmentation. the function g(.) can be any monotonically decreasing
a. Objective Function using Edge Penalty function, so that the greater the edge strength the smaller
the penalty. Suppose the grading ∇(Y) has been
To estimate the mean and standard deviation for equation
(4), most approaches assume that the number of classes is
known and solve the Gaussian mixture by E-M based normalized. Them, the penalty function can be formulated
method. The estimation uses global statistics and does not as:
−( ∇(Y) /K )
describe local region well if the image is highly non- 2
stationary. Incorporating local statistics is thus necessary. (
g ∇(Y) = e) (8)
Consider (4) from another point of view. Maximization of The parameter K in figure 2 defines how fast the edge
(4) is equivalent to the minimization of penalty decays with the increase of edge strength. As K
)
 n
x = argmin ∑ ∑ logσi +
Y(s) − µi
( )
[2
' ] 
− βUi (s) (5)
(figure 2) increases, the penalty difference between weak
and strong edges decreases. When K approaches infinity, all
Ω1,...Ωni=1 s=Ω1
 2 σi
2
( )  edge penalties are equally 1. Therefore, the parameter K

(figure 2) of the edge penalty function g(.) is allowed to
increase during the segmentation, details of which are
described the local feature of edge strength is hence utilized,
but the bias is significantly reduced.
3. Proposed HYBRID Algorithm
A new hybrid algorithm has been proposed to obtain the
MAP estimate. In the hybrid algorithm SA algorithm is first
run for some pre specified amounts of epochs and then ICM
algorithm is run until the stopping criterion has been
satisfied. The combination of global and local concept leads
to a faster converting algorithm. The proposed hybrid
algorithm is described below
1) Initialized the temperature Tin .
2) Compute the energy U of the configuration.
3) Perturb the system slightly with suitable Gaussian
distribution.
' (EPF) while for MRF model. This is mission for many other
4) Compute the new energy U of the perturbed system and
'
evaluate the change in energy ∆U = U −U .
5) If ( )
∆U 〈 0 , accept the perturbed system as the new
configuration Else accept the perturbed system as the
(
new configuration with a probability exp − ∆U / t ,
where t is the cooling schedule.
6) Decrease the temperature according to the cooling
schedule.
7) Repeat steps 2-7 till some pre specified number of epochs
of SA.
8) Compute the energy U of the configuration.
9) Perturb the system slightly with suitable Gaussian
disturbance.
' edge penalty functions. As fig. 4(e) correspond to MRF with
10) Compute the new energy U of the perturbed system
and evaluate the change in energy ∆U = U −U .
( )
11) If ∆U 〈 0 , accept the perturbed system as the new
configuration otherwise retain the original
configuration.
12) Repeat steps 8-12, till the stopping criterion is met. The
(
stopping criterion is the energy U 〈 threshold .
4. Simulation and Results
In this paper we have considered three different image
having strong as well as weak edges. These images are
considered deliberately to validated the weak edge
preserving attributes of the schemes. The first image
considered is a cell the image as shown in fig. 3(a), fig. 3(b)
to fig.
3(g) show the results obtained using MRF, DMRF, and
JSEG methods. Although line field is used to preserved the
edges, the weak edges inside the cell boundary are not being
preserve. As observed from fig. 3 some hazy edge could be
formed because of weak edges, still there are no clear edge
for present inside each cell. Fig.
(e) and Fig. 3(f) shows the result obtained with MRF and
DMRF models with edge penalty function incorporated in
the energy functions. We have also proposed Exponential
edge penalty function. Fig. 3(c) and Fig. 3(d) shows the
(IJCNS) International Journal of Computer and Network Security, 157
Vol. 2, No. 5, May 2010
result obtain MRF and DMRF model. The models
parameters are tabulated in table (II). For exponential edge
penalty function (EPF) the value of K is choose to be σ =5.
It is observed that interior edges could be observed and
percentage of misclassification error is less than that of
using edge penalty functions. The results has been improve
in case of DMRF model as compared to MRF model. As
seen from the table (I), the misclassification error is less in
case of DMRF model as compared to MRF. Use of
Exponential EPF preservation of weak edge. Fig. 3(e) and
Fig.3(c) shows the image of MRF model with exponential
EPF yields better results than that of MRF model. As seen in
Fig. 3(e) and Fig. 3(f) the image of MRF and DMRF model
with the Exponential edge penalty function as shows that
the images of DMRF model with Exponential edge penalty
function yields better results than that of MRF exponential
penalty function, As seen from fig. 3(e) and 3(f) the nearer
the diseased part (toward the right hand top) the inside edge
has been formed using Exponential edge penalty function
cells similar observation can also be made. In this also we
have employed SA and proposed hybrid algorithm. The
convergence of these algorithm are presented in fig. 6
where we observe that the hybrid algorithm converges at
) around 600 iteration where as SA converges at 6000
iterations. It is also observed that Hybrid algorithm is fast
Converges than the SA algorithm.
The image consider in shown in fig. 4, where are some weak
edge seen in the water boat etc. fig. 4 shows the results
obtained using MRF and DMRF. As seen from fig. 4(c) to
(d) the weak edge have not been presented. In case DMRF
there are more misclassification as seen from fig. 4 show the
results obtained using MRF and DMRF with exponential
exponential, observed the weak edge of the bottom left side
'
have been preserved and strong have been formed. This
phenomenon is more prominent in case of second order
DMRF model as shown in fig. 4(f). As found from Table (I)
the percentage of misclassification error is less than that of
using Exponential edge penalty functions. As shows the
) convergence curves for hybrid and SA algorithm converges
at 600 iteration where as SA converges after 6000 iterations.
Similarly observation are also made for the third images
shows in fig. 5(c) and 5(d) shows the bird image and the
corresponding ground truth observed the weak edge could
not be preserved. As seen from fig. 5(e) and 5(f), the MRF
and DMRF model with Exponential edge penalty function
performed better than that of MRF and DMRF model . In
this case also the weak edge would be preserved the model
parameter are tabulated in Table (II). Hence here also hybrid
algorithm converged faster than that of the SA algorithm.
158
Table 1: comparison of %age of misclassification error for
segmentation results of four weak image and scene
images
Table 2: parameters DMRF using gradient exponential
penalty function model
Figure 3.Segmentation of cell image of size 300x200 using
weak membrane model and RGB color model: (a) Original
image (b) Ground Truth (c) MRF using hybrid (d) DMRF
using hybrid (e) MRF using Exponential Penalty Function
image (f) DMRF using Exponential Penalty Function image
(g) JSEG image.
Figure 4.segmentation of water boat image of size 200x300
using weak membrane model and RGB color model: (a)
Original image (b) Ground image (c) MRF using hybrid (d)
DMRF using hybrid (e) MRF using Exponential Penalty
Function image (f) DMRF using Exponential Penalty
Function image (g) JSEG image.
(IJCNS) International Journal of Computer and Network Security,
Vol. 2, No. 5, May 2010
Figure 5. Segmentation of Poster image of size 200x248
using weak membrane model and RGB color model: (a)
Original image (b) Ground Truth (c) MRF using hybrid (d)
DMRF using hybrid (e) MRF using Exponential Penalty
Function image (f) DMRF using Exponential Penalty
Function image (g) JSEG image.
Figure 6. Energy Convergence curve comparison for Cell
image: DMRF model with SA and Hybrid algorithm for
whole process
5. Conclusion
In this paper the color image segmentation problem is
addressed using MRF model. We have employed two color
models namely RGB and OHTA model. A new MRF model
called DMRF model is proposed to take care of intra color
plane and inter color plane interactions. In RGB and OHTA
model, the inter-plane correlation are decomposed and
partail correlation has been introduced due to the DMRF
model. The associated model parameters are selected on
trail basis. The segmentation problem is formulated using
MAP estimates. It is observed that the hybrid algorithm
faster than that of using SA algorithm. The algorithm with
different color modeled image model was tested on a wide
variety of images. It is observed that the segmentation result
preserved the strong edge but failed to preserve weak edges.
In order to protect edges, we have introduced the edge
penalty function of Qiyao Yu et al [12] in the clique
potentail function of the a priori model. In this case also we
have used DMRF model with RGB color model. We have
used the exponential edge penalty function of Qiyao Yu et al
and found the results could preserve weak edges. The model
parameters are selected on trail and error basis. We have
proposed Exponential edge penalty function and it is found
that use of DMRF model with proposed function produced
 (IJCNS) International Journal of Computer and Network Security, 159
Vol. 2, No. 5, May 2010

better result than that of using DMRF model. We have
employed hybrid and SA to obtain segmentation and hybrid
algorithm converges faster than that of SA algorithm.
Current work focuses estimation of model parameters and
devising an unsupervised scheme.
Classification and Region Merging,” Proc of the Second
Canadian Conference on Computer and Robot Vision
volume 1(May 2005):pp.579-586.

References

[1] L. Lucchese and S.K. Mitra , “Color Image

Segmentation: A state of the art survey”, Proc of the
Indian National Science Academy , volume 67, No. 2,
(feb 2006): pp. 207-221
[2] Chen H.D., Jiang X.H., Sun Y., and Wang J., “Color
Image Segmentation: advances and prospects,”Pattern
Recognition, volume 34, 2000: pp.2259-2281.
[3] Gonzalez R.C. and Woods R.E. Digital image
processing. Singapore: Pearson Education, 2001.
[4] Stan Z. Li, “Markov Field Modeling in Image analysis,”
Japan: Springer, 2001.
[5] Geman. S and Geman D, “ Stochastic relaxation,
Gibbs distributions, and the Bayesian
Restoration of images, “IEEE Transaction on Pattern
Analysis and Machine Intelligence, volume 6, No. 6,
November 1984: 721-741.
[6] Kato. Z Pong. T. and Lee J.C., “Color Image
Segmentation and Parameter Estimation in a
Markovian framework,” Pattern Recognition Letters,
volume 22, No. 2, 2001: pp. 309-321.
[7] Kato. Z. and Pong. T. , “ A Markov Random Field
Image Segmentation model using Combined Color
and Texture Feature, “ Proceeding of International
Conference on Computer Analysis of Image and
Patterns. (September 2001): pp. 547-554.
[8] Kirkpatrick S., Gelatt C.D., and Vecchi M.P.,
“Optimization by Simulated Annealing,” Science,
volume 220, No. 4598, 1983: pp. 671-679.
[9] Panjawani D.K. and healey G., “Markov Random Field
Models for Unsupervised Segmentation of Texture
Color Images.” IEEE Transaction on Pattern Analysis
and Machine Intelligence, volume 17, No.10,(October
1995): pp. 939-954.
[10] P.K. Nanda, MRF Model Learning and its Application
to Image Restoration and Segmentation
PhD Dessertation, Electrical Engg. Department, IIT
Bombay 1995
[11] Qiyao Yu, David A . Clausi, “IRGS: Image
Segmentation Using Edge Penalties and Region
Growing,” IEEE Transaction on Pattern Analysis and
Machine Intelligence PAMI.
[12] Qiyao Yu, David A. Clausi, “Combination Local and
Global Feature for Image Segmentation Using Iterative