IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO.
2, FEBRUARY 2018
Random Walks for Synthetic Aperture Radar
Image Fusion in Framelet Domain
Xiaoyuan Yang, Jingkai Wang, and Ridong Zhu
Abstract— A new framelet-based random walks (RWs) method
is presented for synthetic aperture radar (SAR) image fusion,
including SAR-visible images, SAR-infrared images, and Multi-
band SAR images. In this method, we build a novel RWs model
based on the statistical characteristics of framelet coefficients
to fuse the high-frequency and low-frequency coefficients. This
model converts the fusion problem to estimate the probability
of each framelet coefficient being assigned each input image.
Experimental results show that the proposed approach improves
the contrast while preserves the edges simultaneously, and out-
performs many traditional and state-of-the-art fusion techniques
in both qualitative and quantitative analysis.
Index Terms— Image fusion, framelet transform, random
walks, SAR image, infrared image, visible image.
I. I NTRODUCTION
W ITH the development of sensor technology, multisensor
image fusion has attracted the research interests of
more and more scholars. The purpose of multi-sensor image
fusion is to obtain a composite image by consolidating image
information from a plurality of sensors at the same scene [1].
SAR is an active microwave remote sensor system which
can work at any time and under all weather and provides
high resolution radar image. It has been widely used in
military and civil field. The visible (VIS) image contains rich
high-frequency details, which can reflect the spatial details
information of the whole scene. The infrared (IR) image can
show shape contour information of objects by intensity of
thermal radiation of objects. In addition, the high band SAR
image mainly displays the appearance of scene, which is
close to optical imaging. While, the low band SAR image
has a stronger penetration, thus hidden ground targets can be
imaged. Therefore, the full scene of images can be provided
by SAR image fusion (SAR-VIS, SAR-IR and Multi-band
SAR), which has become a foundation for classification,
detection, land use mapping and urban area extraction in SAR
images [2]–[10].
Manuscript received December 16, 2016; revised June 5, 2017 and
July 8, 2017; accepted August 21, 2017. Date of publication August 30,
2017; date of current version November 22, 2017. This work was sup-
ported in part by the Natural Science Foundation of Beijing Municipality
under Grant 4152029 and in part by the National Natural Science Foun-
dation of China under Grant 61671002. The associate editor coordinat-
ing the review of this manuscript and approving it for publication was
Prof. Oleg V. Michailovich. (Corresponding author: Xiaoyuan Yang.)
The authors are with the Key Laboratory of Mathematics, Informatics, and
Behavioral Semantics, Ministry of Education, Department of Mathematics
and Systems Science, Beihang University, Beijing 100191, China (e-mail:
xiaoyuanyang@vip.163.com; nmgdxwjk@163.com; rd_zhu@163.com).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIP.2017.2747093
1057-7149 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
851
In recent decades, many SAR image fusion approaches
have been proposed [1], [11]–[25]. The simplest approach
is to calculate the average of all the original images [11],
but this approach reduces the contrast and sharpness of
the fused image. To resolve this problem, many methods
based on multi-scale transform (MST) have been presented,
such as ratio of low-pass pyramid (RLP) [12], gradient
pyramid (GP) [11], Laplace pyramid (LAP) [11], discrete
wavelet transform (DWT) [1], [13], “à trous” wavelet trans-
form (AWT) [14], [15], curvelet transform (CVT) [16], [17],
dual-tree complex wavelet transform (DTCWT) [18], nonsub-
sampled contourlet transform (NSCT) [19], [20] and nonsub-
sampled shearlet transform (NSST) [21]. In addition, many
other image fusion techniques have been developed. For
instance, paper [22] presented a Kalman filter fusion method
to filter and unwrap the phases of interferometric synthetic
aperture radar images. Paper [23] proposed a local non-
negative matrix factorization approach to fuse SAR and visible
image.
Recently, Li et al. [24] proposed a simple and effec-
tive image fusion method based on guided filtering (GF).
However, it may reduce the contrast and prone to smooth
some edges. Xu et al. [25] applied feedback sparse com-
ponent analysis (FSCA) method for remote sensing image
fusion. The sparse components of image can be extracted
by FSCA method while ignoring noise. Although it is good
for the fusion process, this method requires more expensive
computation. Kumar [26] presented a novel image fusion
approach based on cross bilateral filter (CBF). He used detail
images extracted from the input images by CBF for the
computation of fusion weights which was helpful to improve
fusion performance. However, the approach may generate
fusion artifacts. Liu et al. [27] used MST and sparse repre-
sentation (SR) to build a new image fusion framework. In the
proposed framework, the SR fusion method was applied to
fuse the low-pass subbands. The contrast in the fused image
is improved. But this approach does not involve global or local
statistic information about high-pass subbands.
Grady [28]–[30] first proposed the random walks method
for image segmentation. He analyzed the advantages of this
method, such as weak boundary detection, noise robustness
and so on. After [30], some image processing methods based
on random walks have been presented [31]–[34]. Among
them, paper [32] presented a general random walks fusion
approach for multi-exposure image fusion. Then, paper [33]
proposed a novel multi-focus image fusion approach based
on random walks fusion framework [32]. In these methods,
852 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 2, FEBRUARY 2018

all input images were given labels. They focused on solving Compared to previous approaches, our method can obtain
a labeling problem to estimate the probability of each input better fused image.
image pixel being assigned each labeled input image. And the The remainder of this paper is organized as follows.
merged weights for image fusion can be calculated by these In Section II, we give a brief introduction of framelet transform
probabilities. and random walks. Section III presents a novel random walks
Paper [32] and [33] are spatial domain methods. They might approach for SAR image fusion in framelet domain. The
smooth the merged weights excessively, which is bad for experimental results are given in Section IV. In the end,
image fusion. In order to solve this problem, we transform the Section V summarizes the main conclusions of this paper.
input images to a multi-scale framelet domain by performing
nonsubsampled framelet transform (NFT). II. P RELIMINARIES
As a generalization of the orthogonal basis, the framelet
In this section, we shall briefly review some concepts and
relaxes the requirements of the orthogonality and linear inde-
principles with regard to framelet transform and random walks.
pendence to bring in redundancy [35]. Therefore, it is more
precise than wavelet in extracting the spatial information.
Moreover, translation invariance is a desirable property in A. Framelet Transform
image processing [36]. It can be achieved by nonsubsampling For simplicity, we only show the framelets in the univariate
in framelet decomposition. In addition, the framelet transform setting, and the bivariate framelets can be obtained by tensor
can afford the effective sparse representation of the images product of the univariate one.
which include rich textures. More importantly, this transform A tight frame of L 2 (R) can be defined by a countable
also has properties of fast decomposition and perfect recon- function subset of χ ⊂ L 2 (R), if
struction [37]. In recent years, it has been widely used in image 
processing, such as image inpainting [38], image deblur- f =  f, gg ∀ f ∈ L 2 (R) (1)
ring [35], image denoising [36], [39], pansharping [40], [41] g∈χ
and image sparse unmixing [42]. where  f, g represents the inner product for any two func-
In our proposed method, we first perform NFT on the tions f, g ∈ L 2 (R).
input images to obtain their framelet coefficients. The inherent Given a collection of dilations and shifts of a finite set
speckle noise of SAR image affects the further processing.  = {ψ 1 , . . . ψ p } ⊂ L 2 (R), it is defined as a wavelet system
Therefore, we give a speckle reduction method based on hard W () as follows:
thresholding the high-frequency coefficients. Then, a novel
random walks fusion model is built in pre-denoised framelet W () = {ψd,q
t
= 2d/2 ψ t (2d η − q) | t = 1, . . . , p, d ∈ Z,
domain. In this model, we can obtain the probabilities of q ∈ Z} (2)
framelet coefficients being assigned each input image by
finding a global optimal solution. Then, the fusion weights of When W () forms a tight frame, it’s called a wavelet tight
high-frequency and low-frequency coefficients are estimated frame, and each ψ t ∈  is named as a framelet.
by these probabilities, respectively. In the end, the fused image To construct the compactly supported tight frame system,
is reconstructed by employing the corresponding inverse NFT one first obtains a compactly supported scaling function φ ∈
on the merged framelet coefficients. L 2 (R) with a refinement mask 
h 0 satisfying
In random walks model, we first employ to build a φ (2w) = 
 h 0 (w)
φ (w) (3)
weighted undirected graph to model the statistical character-
istics of framelet coefficients. For high-pass subband fusion, where φ  can be obtained by the Fourier transform of φ and
the absolute high-frequency coefficient values are selected to 
h0 is a 2π-periodic function with  h 0 (0) = 1.  h 0 (w) =
h (k)e −ikw , h is named as a low-pass filter.
measure the compatibility of the high-frequency coefficient k∈Z 0 0
node with the input image node, which is good for fusing the Then, for the given scaling function φ, a tight frame can
edge contour information of input images. For low-pass sub- be constructed by looking for a suitable set of framelets  =
band fusion, we take neighborhood information into account. {ψ 1 , . . . ψ p }, which is denoted in the Fourier domain by
The local variance of low-frequency coefficient is proposed to t (2w) = 
ψ h t (w)
φ (w), t = 1, 2, . . . , p (4)
construct compatibility function, which is helpful to improve
contrast and avoid artifacts. where theframelet masks  h t are 2π-periodic functions. Here
The main contribution of this paper is to propose a new 
h t (w) = k∈Z h t (k)e−ikw , h t is called a high-pass filter.
SAR image fusion model based on random walks in framelet The unitary extension principle (UEP) in [43] gives the
domain. This model provides a method for solving high- condition for W () to form as a tight frame system, i.e., the
pass or low-pass subband fusion problem that can be for- masks  h0, 
h1, . . . , 
h p satisfy
mulated as an optimization problem. It not only considers
the correlation of the framelet coefficients, but also takes 
p

h 0 (w)
h 0 (w + γ π)+ 
h t (w)
h t (w + γ π) = δ(γ ), γ = 0, 1
the local characteristics of coefficients into account, which is
t =1
good for merging framelet coefficients. In addition, we provide (5)
both subjective and objective comparisons with traditional
and state-of-the-art methods on three different image sets. for almost all w ∈ R. Here δ(γ ) is a delta function.
YANG et al.: RWs FOR SAR IMAGE FUSION IN FRAMELET DOMAIN
Fig. 1. An example of framelet decomposition with three filters.
Suppose that low-pass filter is h 0 , high-pass filters
are h 1 , h 2 . As an example, Fig. 1(a) shows a process of 2-scale
framelet decomposition, where {L s , s = 1, 2} represent low-
pass subbands and {Hs,k , s = 1, 2, k = 1, 2, . . . , 8} represent
high-pass subbands. s and k indicate sth scale and kth high-
pass subband, respectively. The size of these subbands is the
same as the input image due to nonsubsampling in framelet
decomposition. The decomposition results are also shown
in Fig. 1(b).
B. Random Walks
In this subsection, we review the random walkers fusion
method in spatial domain.
We first define a set of labeled nodes VY = {y1 , y2 , . . . , y K }
and a set of unlabeled nodes V X = {x 1 , x 2 , . . . , x N }, where
K denotes the number of input images and N denotes the
number of pixels of an input image. Each input image pixel
is represented by x i ∈ V X , and each input image is indicated
by yl ∈ VY . Then, a weighted undirected graph is defined
as G = (V, E), where V = VY ∪ V X is a set of nodes and
E = E X ∪ E Y is a set of edges. For an unlabeled node
x i ∈ V X , edges E X are drawn between it and its four connected
neighbors. Edges E Y are drawn between each unlabeled node
x j ∈ V X and each labeled node yl ∈ VY . In addition, wi,1 j is a
weight defined on E X that measures the compatibility between
unlabeled nodes x i and x j , and w2j,l is a weight defined on
E Y that measures the compatibility between unlabeled node x j
and labeled node yl . This graph can be represented by Fig. 2.
853
Fig. 2. Graph used in RW. The blue nodes are unlabeled nodes and red
nodes are labeled nodes.
Fig. 3. Flow chart of the image fusion.
Based on the weighted undirected graph mentioned above,
the image fusion problem can be converted to a labeling prob-
lem. This problem places one random walker at each unlabeled
node (input image pixel) and then calculates probabilities that
they first arrive at labeled nodes (input images) [32]. Luckily,
the combinatorial Dirichlet method provides a simple and
effective approach for computing this problem [30].
For simplicity, we directly provide the expression of this
solution. More details can be found in [32] and [33]. We first
denote L as the Laplacian matrix. Then, we give the block
matrix of L:
 
L LXY
L = TY (6)
LX Y LX
where LY is the submatrix of L that contains the relation
within VY ; L X is the submatrix that contains the relation within
V X ; L X Y is the submatrix that contains the relation between
V X and VY . The superscript T represents the matrix transpose.
In the end, we denote p X as the probabilities of the
unlabeled nodes first arriving at the labeled nodes. The final
solution can be computed by:
L X p X = −LTX Y pY (7)
where pY is obtained by the labeled nodes.
III. R ANDOM WALKS F USION M ETHOD
IN F RAMELET D OMAIN
The flow chart of the image fusion is shown in Fig. 3.
Framelet transform has been introduced in section II-A.
This section focuses on RW fusion method to merge framelet
coefficients. We firstly analyze the statistical characteristics
854
Fig. 4. Correlation of framelet coefficients. (a) Neighborhood correlation. (b) Scale correlation. (c) Subband correlation.
of framelet coefficients in the next subsection. Then, a new
random walks model is built based on these characteristics.
Furthermore, we give framelet coefficients fusion rule. In the
end, the effect of noise in SAR images is discussed.
A. Statistical Characteristics of Framelet Coefficients
Let the source image be {Ir,c , r, c = 1, 2, . . . , M}, where r
and c represent row and column, respectively. M is the size
of row and column. As mentioned in section II-A, given the
J -scale NFT of an image with one low-pass filter and two
high-pass filters, Hs,k,r,c denotes the (r, c)th high-frequency
coefficient in the kth high-pass subband of the sth scale and
L s,r,c denotes the (r, c)th low-frequency coefficient in the sth
scale. Where s = 1, 2, . . . , J, k = 1, 2, . . . , 8 and J is the
largest scale in the decomposition.
As mentioned in [39] and [44], we have the following
statistical properties of the framelet coefficients. Neighbor-
ing framelet coefficients are often highly dependent – if
a specific framelet coefficient is large or small, then its
neighboring framelet coefficients are very likely to also be
large or small [44]. In other words, each framelet coeffi-
cient (black node) is related to its neighborhood (blue nodes),
as shown in Fig. 4(a).
According to [39], large or small framelet coefficients
incline to propagate across scales. That is to say each par-
ent coefficient (red node) connects to its child coefficient
(blue node), as shown in Fig. 4(b). In this paper, we give the
definition of the correlation coefficient between transformed
subbands. It can express the correlation between connected
coefficients in the same subband of adjacent scales.
Let A = {ar,c } and B = {br,c } be the matrices
(e.g., the transformed subbands). The correlation coefficient
between A and B is defined as (8).
M M
c=1 [ar,c −A][br,c −B]
R(A, B) =  
r=1
M M
M M
c=1 [ar,c − A] c=1 [br,c − B]
2 2
r=1 r=1
(8)
1 M M 1 M M
where A = M 2 r=1 c=1 ar,c , B = M 2 r=1 c=1 br,c
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 2, FEBRUARY 2018
The correlation between the high-pass subbands is also
useful for random walks in framelet domain. In addition,
we can also use (8) to capture the correlation between con-
nected high frequency coefficients (blue nodes) at the same
location in adjacent high-pass subbands, as shown in Fig. 4(c).
B. Random Walks Model in Framelet Domain
1) Weighted Undirected Graphs in Framelet Domain:
Weighted undirected graph will play an important role in
random walks model.
In this model, let I = {I 1 , I 2 , . . . , I K } denotes the set of
input images and VY = {y1, y2 , . . . , y K } denotes the set of
labeled nodes, where a labeled node yl ∈ VY is associated
with the lth input image I l ∈ I , l = 1, 2, . . . , K .
For high-pass subbands, let V XHs,k = {x s,k,r,c
H } denotes the set
of unlabeled nodes in the kth subband of sth scale. It should be
noted that the superscript H represents the high-pass subband
in this paper.
Then, we define the unlabeled set:
J
8
V XH = V XHs,k = {x iH = x s,k,r,c
H
,
s=1 k=1
i = 8M 2 (s − 1) + M 2 (k − 1) + (c − 1)M + r } (9)
where i represents the spatial position of high-frequency
coefficient in all high-pass subbands.
As shown in Fig. 5(a), we define a weighted undirected
graph G H = (V H , E H ) based on the correlation of high-
frequency coefficients, where V H = VY ∪ V XH is the set of
nodes. Let V H be arranged in a way that:
V H = {v 1H = y1 , v 2H = y2 , . . . , v KH = y K ,
v KH+1= x 1H , . . . , v KH+i = x iH , . . . , v KH+8 J M 2 = x 8HJ M 2 } (10)
Furthermore, E H = E 1H ∪ E 2H ∪ E 3H ∪ E 4H is the set of
edges, where edges in E 1H are drawn between nodes in V XH
and VY , edges in E 2H are drawn between nodes within V XHs,k
and edges in E 3H are drawn between nodes in V XHs ,k and V XHs ,k ,
1 2
s1 and s2 represent the adjacent scales. In addition, edges
YANG et al.: RWs FOR SAR IMAGE FUSION IN FRAMELET DOMAIN 855

Fig. 5. Graph used in RW based on framelet domain. The red and blue nodes are unlabled nodes and the green nodes are label nodes. (a) High-pass subbands.
(b) Low-pass subbands.

in E 4H are drawn between nodes in V XHs,k and V XHs,k , k1 and k2 Likewise, we can define the compatibility function for low-
1 2
represent the adjacent high-pass subbands. pass subbands as follows:
H
Each edge in E is given a weight to measure the compat- ⎧
⎪
⎨w1,m,n (v m , v n ) ∈ E 1
L L L L
ibility between the adjacent nodes in V H . L L L L
wm,n = w (v m , v n ) = w2,m,n (v m , v n ) ∈ E 2L (14)
L L L
Therefore, we can define the compatibility function for high- ⎪
⎩ L
pass subbands as follows: λw3,m,n (v mL , v nL ) ∈ E 3L
⎧ H
⎪ w (v mH , v nH ) ∈ E 1H where w1,m,n
L , w2,m,n
L and w3,m,n
L are weight functions defined
⎪
⎪ 1,m,n
⎪
⎪ on E 1 , E 2 and E 3 , respectively. λ is employed to balance
L L L
⎨w H (v mH , v nH ) ∈ E 2H
wm,n = w (v m , v n ) =
H H H H 2,m,n
(11) the weight w3,m,n
L . We will explain more details about these
⎪
⎪ αw3,m,n
H (v mH , v nH ) ∈ E 3H weights in the following subsection.
⎪
⎪
⎪
⎩ H 2) Compatibility Functions: In this paper, the function
βw4,m,n (v mH , v nH ) ∈ E 4H w1,m,n
H measures the compatibility of mth high-frequency
where w1,m,n
H , w2,m,n
H , w3,m,n
H and w4,m,n
H are weight functions coefficient node with nth input image node. The large absolute
high-frequency coefficient values in high-pass subbands cor-
defined on E 1 , E 2 , E 3 and E 4 , respectively. α and β are
H H H H

introduced to balance the weights w3,m,n H and w4,m,n

H . respond to the salient features in the original image such as
edges and contours. Therefore, we select the high-frequency
Similarly, let V X s = {x s,r,c } denotes the set of unlabeled
L L
coefficient values to construct w1,m,n
H .
nodes in the sth scale. It should be noted that the superscript
In addition, we normalize these values to be within the range
L represents the low-pass subband in this paper.
[0,1]. At last, w1,m,n
H can be computed by:
Then, we define the unlabeled set:
|Hmn | − min (|Hn |)

J
w1,m,n
H
= (15)
V XL = V XLs = {x iL = x s,r,c
L
, i = M 2 (s − 1) + (c − 1)M +r } max(|Hn |) − min(|Hn |)
s=1 where Hmn = Hs,k,r,c
n , m = 2+8M 2 (s−1)+M 2 (k −1)+M(c−
(12) 1) + r, n = 1, 2, . . . , K , Hs,k,r,c
n denotes the high-frequency
where i represents the spatial position of low-frequency coef- coefficient Hs,k,r,c of the nth input image and Hn is a vector,
ficient in all low-pass subbands. each element is Hmn .
As shown in Fig. 5(b), we define a weighted undi- On the other hand, functions w2,m,n H , w3,m,n
H and w4,m,n
H

rected graph G L = (V L , E L ) for low-pass subbands, where
V L = VY ∪ V XL is the set of nodes. Let V L be arranged in a
way that:
V L = {v 1L = y1 , v 2L = y2 , . . . , v KL = y K ,
v KL +1 = x 1L , . . . , v KL +i = x iL , . . . , v KL +J M 2 = x JL M 2 }
Moreover, E L = E 1L ∪ E 2L ∪ E 3L is the set of edges, where
edges in E 1L are drawn between nodes in V XL and VY , edges
in E 2L are drawn between nodes within V XLs and edges in E 3L
are drawn between nodes in V XLs and V XLs .
1 2
are used to evaluate the compatibility between connected
high frequency coefficients. As previous random walks
approaches [28]–[33], the Gaussian weighting function defined
by Euclidean distance could measure the compatibility
between two adjacent pixels. But this function can’t effectively
(13) capture the geometric property of input images due to its
isotropy. Moreover, it has drawbacks in the treatment of
edges and contours. In our work, we apply the Gaussian
weighting function in framelet domain. It not only measures
the compatibility between two connected framelet coefficients
but also overcomes the shortcomings in the spatial domain.
856 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 2, FEBRUARY 2018

In addition, this function can also be used to suppress the n

Firstly, we define the local variance Ss,r,c at position (r, c)
compatibility of connected framelet coefficients with large in the sth scale of the nth input image.
difference of values. 1 
In our method, we first define the average high-frequency
n
Ss,r,c = (L ns,r,c − μns,r,c )2 (20)
|ℵ|
coefficient value at position (r, c) in the kth subband of sth (r,c)∈ℵ
scale in all input images. where L ns,r,c
denotes the low-frequency coefficient L s,r,c of
K l the nth input image and μns,r,c denotes the average L ns,r,c in
l=1 Hs,k,r,c
H s,k,r,c = (16) neighborhood ℵ, that is,
K
1  n
Then, w2,m,n
H is calculated by the following equation: μns,r,c = L s,r,c (21)
|ℵ|
(r,c)∈ℵ

K
Hml − Hnl 2 Then, we construct the function w1,m,n
L by normalizing
H
w2,m,n = exp(− )+ n
δ the Ss,r,c .
l
H m − H n 2 Smn − min (Sn )
≈ exp(− ) + , w1,m,n
L
= (22)
δ max(Sn ) − min(Sn ) + eps
δ
H m = H s,k,r1 ,c1 , H n = H s,k,r2 ,c2 , δ = (17) where Smn = Ss,r,c n , m = 2 + M 2 (s − 1) + M(c − 1) + r ,
K n = 1, 2, . . . , K , Sn is a vector, each element is Smn . eps is a
where m = 2 + 8M 2 (s − 1) + M 2 (k − 1) + M(c1 − 1) +r1 , n = regularisation term which is set to 10−6 .
2+8M 2 (s−1)+M 2 (k−1)+M(c2 −1)+r2 . (r1 , c1 ) and (r2 , c2 ) Similar to (16), the average low-frequency coefficient value
represent adjacent positions by 4-connectivity. exp(.) is the at position (r, c) in the sth scale in all input images can be
exponential function and .2 denotes Euclidean distance. δ is calculated by (23).
a controlling parameter and is a small constant which is set K l
l=1 L s,k,c
to 10−6 . L s,r,c = (23)
Unlike [32], [33], we take the correlation of different high- K
pass subbands into consideration. Then, we use the improved At last, the functions w2,m,n
L and w3,m,n
L are used to evaluate
Gaussian weighting function to evaluate the compatibility of the compatibility between connected low-frequency coeffi-
connected coefficients in different high-pass subbands. The cients. Similar to high-frequency coefficients, we also utilize
compatibility functions w3,m,n
H and w4,m,n
H are calculated by improved Gaussian weighting function to estimate w2,m,nL and
(18) and (19). w3,m,n by the following (24) and (25).
L

C(s1 ,s2 ),k H m − H n 2 L m − L n 2

w3,m,n
H
= exp(− ) + , w2,m,n
L
= exp(− ) + ,
δ δ
Hm = H s1 ,k,r,c , H n = H s2 ,k,r,c , Lm = L s,r1 ,c1 , L n = L s,r2 ,c2 (24)
C(s1 ,s2 ),k = R(Hs1 ,k , Hs2 ,k ) (18)
where m = 2 + M 2 (s − 1) + M(c1 − 1) + r1 ,n = 2 + M 2 (s −
Cs,(k1 ,k2 ) H m − H n 2 1) + M(c2 − 1) + r2 .
w4,m,n
H
= exp (− ) + ,
δ
Cs1 ,s2 L m − L n 2
Hm = H s,k1 ,r,c , H n = H s,k2 ,r,c , w3,m,n
L
= exp (− ) + ,
δ
Cs,(k1 ,k2 ) = R(Hs,k1 , Hs,k2 ) (19)
L m = L s1 ,r,c , L n = L s2 ,r,c , Cs1 ,s2 = R(Ls1 , Ls2 ) (25)
where the calculation of m and n are similar to (17). where the calculation of m and n are similar to (24). Ls1
Hs1 ,k , Hs2 ,k , Hs,k1 and Hs,k2 are matrices, their elements and Ls2 are matrices and their elements are L s1 ,r,c and L s2 ,r,c .
are H s1 ,k,r,c , H s2 ,k,r,c , H s,k1 ,r,c and H s,k2 ,r,c , respectively. Cs1 ,s2 indicates the correlation coefficient between Ls1 and Ls2 .
C(s1 ,s2 ),k and Cs,(k1 ,k2 ) are the correlation coefficients of the 3) Random Walks Model Building and Solving: The prob-
pairs (Hs1 ,k , Hs2 ,k ) and (Hs,k1 , Hs,k2 ), respectively. ability of each unlabeled node (framelet coefficient) first
In this paper, the function w1,m,n L evaluates the compatibility reaching labeled nodes (input images) can be calculated by
of a low-frequency coefficient node with a input image node. solving the appropriate combinatorial Dirichlet problem [30].
As we all know, the low-pass subbands represent the approx- Then, let u(v) indicate the energy connected with the node v.
imate information of the image. Furthermore, the statistics of The total energy of the high-pass subbands system is:
low-frequency coefficients used to assess local image features
1 
play a crucial role in adaptive image fusion. The local variance T EH = w H (u(v mH ) − u(v nH ))2 (26)
is one of the commonly used statistics of low-frequency 2 H H H m,n
(v m ,v n )∈E
coefficients, because of its ease of calculation and its validity
in evaluating image smoothness. Hence, we use the variance In the same way, we can obtain T E L (total energy of the
of low-frequency coefficient in a local neighborhood ℵ to low-pass subbands system), where the superscript H in (26)
estimate w1,m,n
L . is replaced with L.
YANG et al.: RWs FOR SAR IMAGE FUSION IN FRAMELET DOMAIN 857

Our target is to find a function u(.) that minimizes formulated as:

⎧
T E H (or T E L ). If u(.) satisfies ∇ 2 u = 0, then it ⎪
⎪ Hs,k,r,c ,
1
is guaranteed to minimize such total energy T E H (or ⎪
⎪
⎪
⎪i f Ms,k,r,c
1 > max{Ms,k,r,c
l
, l = 2, . . . , K }
T E L ) [30]. The function u(.) can be efficiently calculated ⎪
⎪
⎪
⎪
by using matrix operations. Here, we define the Lapla- ⎪
⎪ H 2 ,
⎪
⎪
s,k,r,c
⎨i f M 2
s,k,r,c > max{Ms,k,r,c , l = 1, 3, . . . , K }
l
cian matrix for high-pass subbands (or low-pass subbands)
as:
F
Hs,k,r,c = .
⎪
⎪ ..
⎧ ⎪
⎪
⎪ ⎪
⎪
⎨d m
H if m = n ⎪
⎪
K
Hs,k,r,c ,
⎪
⎪
Lm,n = −wm,n i f (v mH , v nH ) ∈ E H
H H ⎪
⎪ if M K > max{Ms,k,r,c
l
, l = 1, . . . , K − 1}
⎪
⎩ ⎪
⎪ 1  Ks,k,r,cl
other wi se ⎩
0 l=1 Hs,k,r,c , other wi se
⎧ K
⎪
⎨d m
L if m = n (30)
L
or Lm,n = −wm,n L i f (v mL , v nL ) ∈ E L (27) F
where Hs,k,r,c denotes the fused high-frequency coefficient
⎪
⎩
0 other wi se value at position (r, c) in the kth subband of sth scale.
l
Ms,k,r,c is the select measurement assigned to the lth input
where dmH (or dmL ) is the degree of the node v mH (or v mL ). l
image for high-frequency coefficient Hs,k,r,c .
Then, (26) can be rewritten in matrix form as: In our method, the select measurement Ms,k,r,cl can be
l
 T  H derived from the probability H Ps,k,r,c :
1 UYH H UY
T EH = L l
= H Ps,k,r,c
l
, l = 1, 2, . . . , K
2 UH X UHX
Ms,k,r,c (31)
 H T    H
1 UY LYH LH UY l
where H Ps,k,r,c denotes the probability of a high-frequency
= X Y
2 U H
X (L H T
XY ) L H
X U H
X
l
coefficient Hs,k,r,c assigned to the lth input image. It can be
1 H T H H H T H T H obtained by the solution U H X.
= ((UY ) LY UY + 2(U X ) (L X Y ) UY
2 On the other hand, low-frequency coefficients fusion is
+ (U H
X ) LX UX )
T H H
(28) formulated as a weighted average. Based on the inverse NFT,
we only need to calculate fused low-frequency coefficients of
where U H X is a vector, each member is the energy of node in J th scale. They can be computed by:
V XH ; UYH is a vector, each member is the energy of node in

K
VYH ; LYH is the submatrix of L H which contains the relation L FJ,r,c = l
W J,r,c L lJ,r,c (32)
within VYH ; L HX is the submatrix which contains the rela- l=1
tion within V XH ; L H
X Y is the submatrix which contains the
relation between unlabeled nodes (framelet coefficients) and where L FJ,r,c denotes the fused low-frequency coefficient value
labeled nodes (input images). at position (r, c) in the J th scale. W J,r,c
l is the weight assigned
T E L can be rewritten in matrix form in the same to the lth original image for low-frequency coefficient L lJ,r,c .
l
way, where the superscript H in (28) is replaced In this paper, the weight W J,r,c can be derived from the
with L. l
probability L PJ,r,c :
Therefore, the minimum total energy solution can be l
gained by setting ∇ (T E H ) = 0 or ∇ (T E L ) = 0 l
L PJ,r,c
W J,r,c = K , l = 1, 2, . . . , K (33)
with respect to U H L
X (or U X ), i.e., solving the following
l
l=1 L PJ,r,c
equation:
l
where L PJ,r,c denotes the probability of a low-frequency
l
X U X = −(L X Y ) UY
LH L LX U LX = −(L LX Y )T UYL (29)
H H T H coefficient L J,r,c assigned to the lth original image. It can
or
be obtained by the solution U LX .
In the above equation, we assign the energy of the labeled
node connected with corresponding input image the value D. Effect of Noise in the SAR Image
1 and assign 0 to the energy of other labeled nodes. The
solution U H L SAR image may generate coherence speckle noise. In gen-
X (or U X ) will afford us the probabilities of a
random walker starting from each unlabeled node (framelet eral, the noise is concentrated in high-pass subbands. In our
coefficient) reaching the labeled node connected with the input algorithm, compatibility functions are determined by absolute
image. value of high-frequency coefficient which is sensitive to noise.
In order to overcome this problem, we add a pre-denoising step
to reduce the noise in the SAR images. In this paper, the hard
C. Framelet Coefficients Fusion Rule thresholding [45], [46] has performed in all high-frequency
coefficients.
On the one hand, the high frequency framelet coefficients 
are fused by the approach of choosing maximum measurement. Hs,k,r,c i f |Hs,k,r,c | > Ts,k
Hs,k,r,c =
D
(34)
The technique of choosing maximum measurement can be 0 or her wi se
858
Algorithm 1 The Main Process of the Proposed Fusion
Method
D
where Hs,k,r,c indicates denoised high-frequency coefficients
of SAR image. Ts,k represents threshold in different high-pass
subbands.
In this paper, the computing method of the threshold Ts,k
is adaptive to different high-pass subband characteristics.
√ √
σs,k 2 ln N 2 2σs,k
Ts,k = √ = √ (35)
ln( N ) ln N
where N denotes the number of high-frequency coefficients.
The variance of noise σs,k is not known in framelet domain,
it is estimated by the median function.
medi an(|Hs,k |)
σs,k = (36)
0.6745
However, the blurred edges may be generated during the
pre-denoising process, which will affect the result of image
fusion. In future work, we will further study how to incorpo-
rate the noise item into the image fusion model to make the
approach have a better robustness to noise.
Based on the above discussion, we summarize the presented
approach for image fusion in Algorithm 1.
IV. E XPERIMENTAL R ESULTS AND C OMPARISON
A. Original Images
In this paper, we assume that there are only two input
images I 1 and I 2 . As shown in Fig. 6, six groups of original
images divided into three categories are applied to verify the
effectiveness of the presented image fusion approach. Among
them, there are two groups of SAR-Visible (S-V) images
(see Fig. 6(a-d)), two groups of SAR-Infrared (S-I) images
(see Fig. 6(e-h)) and two groups of Multi-band SAR (M-S)
images (see Fig. 6(i-l)). Furthermore, we assume that the two
original images in each group have been registered.
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 2, FEBRUARY 2018
Fig. 6. Original images used in our experiments. (a) SAR-1. (b) VIS-1.
(c) SAR-2. (d) VIS-2. (e) SAR-3. (f) IR-3. (g) SAR-4. (h) IR-4.
(i) S-BAND-5-1. (j) S-BAND-5-2. (k) S-BAND-6-1. (l) S-BAND-6-2.
B. Image Fusion Evaluation Metrics
In order to objectively assess the performances of different
image fusion approaches, many fusion evaluation metrics
have been presented in recent years [47]–[53]. It is generally
known that only one evaluation metric can’t reflect the image
fusion quality in quantitative analysis. Therefore, to make
a comprehensive evaluation for fused image, we adopt five
popular fusion evaluation metrics, which are briefly introduced
as follows. For simplicity, we denote A and B as two original
images and indicate F as the fused image.
Fusion evaluation metrics Q W and Q E are proposed by
Piella and Heijmans [49] based on the universal image quality
index(UIQI) [47], [48].
The evaluation metric Q W is defined as

QW = c(w)(λ(w)Q 0 (A, F|w)+(1−λ(w))Q 0 (B, F|w))
w∈W
(37)
where the weight c(w) is the normalized saliency of w among
all the local sliding windows. Q 0 (A, F|w) and Q 0 (B, F|w)
are computed by the proposed method in [47] in a local sliding
window w. The local weight λ(w) is computed by
s(A|w)
λ(w) = (38)
s(A|w) + s(B|w)
where the saliency measures s(A|w) and s(B|w) are computed
with the variance of A and B in a local sliding window w,
respectively.
The evaluation metric Q E is defined as follows [49]:
Q E = Q W (A, B, F)Q W (A , B , F )τ (39)
where A , B , and F are the corresponding edge images of
A, B, and F, respectively. τ is a parameter that represents
YANG et al.: RWs FOR SAR IMAGE FUSION IN FRAMELET DOMAIN 859

Fig. 7. SAR-VIS-1 fused images. (a) LAP. (b) RLP. (c) DWT. (d) DTCWT. (e) CVT. (f) NSCT-P. (g) NSCT-S. (h) NSST-M. (i) GF. (j) GRW. (k) NFT.
(l) Our.

Fig. 8. SAR-VIS-2 fused images. (a) LAP. (b) RLP. (c) DWT. (d) DTCWT. (e) CVT. (f) NSCT-P. (g) NSCT-S. (h) NSST-M. (i) GF. (j) GRW. (k) NFT.
(l) Our.

the contribution of the edge images compared to the source The evaluation metric Q Y based on structural similarity
images. index measure (SSIM) is defined as [52]:
The evaluation metric Q G based on gradient assesses the ⎧
performances of edge information transferred from the original ⎪
⎪λ(w)SS I M(A, F|w)+(1−λ(w))SS I M(B, F|w),
⎪
⎨i f SS I M(A, B|w) ≥ 0.75
images to the fused image [50]. It is computed by
QY = (42)
 ⎪
⎪max{SS I M(A, F|w), SS I M(B, F|w)},
i, j (Q
AF (i, j )w A (i, j ) + Q B F (i, j )w B (i, j )) ⎪
⎩
QG =  (40) i f SS I M(A, B|w) < 0.75
i, j (w (i, j ) + w (i, j ))
A B

The local weight λ(w) has been defined in (38). More

where Q AF (i, j ) = Q gAF (i, j )Q αAF (i, j ), Q gAF (i, j ) and
Q αAF (i, j ) are respectively the edge strength preservation and
edge orientation preservation at (i, j ) from A. The definition
of Q B F (i, j ) is the same as that of Q AF (i, j ). w A (i, j ) is
the weight based on the edge strength (i, j ) in A. w B (i, j ) is
similar to w A (i, j ).
The evaluation metric Q P based on phase congruency
represents the measures of fused image salient characteristics
such as contours and edges [51]. It is calculated by
Q P = (Pp )a (PM )b (Pm )c
where a, b and c are the exponential parameters. p, M and m
indicate phase congruency, maximum and minimum moments,
respectively. More details about Q P can be found in [51].
details about SS I M are available in [52].
In the above five evaluation metrics, the higher the value
of the evaluation metric, the better the fusion effect will be.
In addition, all the parameters of the evaluation metrics are
set to the default values given in the referred papers.
C. Filters Setting
In our study, we adopt the piecewise linear B-spline as
the scaling function φ as mentioned in [43]. This function
(41)
φ has the refinement mask  h 0 = (1 + e−iw )2 /4, and the
corresponding low-pass filter is h 0 = 14 [1, 2, 1].
Two framelets
√ ψ1 and ψ2 are obtained by the framelet masks

h 1 = − 42 (1 − e−i2w ) and  h 2 = − 14 (1 − e−iw )2 , whose
860 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 2, FEBRUARY 2018

Fig. 9. SAR-IR-1 fused images. (a) LAP. (b) RLP. (c) DWT. (d) DTCWT. (e) CVT. (f) NSCT-P. (g) NSCT-S. (h) NSST-M. (i) GF. (j) GRW. (k) NFT.
(l) Our.

Fig. 10. SAR-IR-2 fused images. (a) LAP. (b) RLP. (c) DWT. (d) DTCWT. (e) CVT. (f) NSCT-P. (g) NSCT-S. (h) NSST-M. (i) GF. (j) GRW. (k) NFT.
(l) Our.

Fig. 11. Analysis of parameter J .

Fig. 12. Analysis of parameter δ.
√
corresponding high-pass filters are h 1 = 4 [−1, 0, 1]
2
and
h 2 = 14 [−1, 2, −1]. There are five free parameters, J , δ, α, β and λ in our image
fusion model. We first set the values of these parameters
as (J, δ, α, β, λ) = (3, 10.0, 1.0, 1.0, 1.0). When analyzing a
D. Analysis of Parameters parameter, the values of the other four parameters is fixed.
This section discusses the impact of different parameters As shown in Fig. 11, when J becomes bigger, the Q G ,
on the fusion performance. We utilize the average values of Q P and Q Y increase, while Q W decreases. Moreover, Q E
five evaluation metrics to evaluate the fusion performance. becomes bigger first and then decreases with increase of J .
YANG et al.: RWs FOR SAR IMAGE FUSION IN FRAMELET DOMAIN 861

Fig. 13. Analysis of parameters. (a) α. (b) β. (c) λ.

Fig. 14. Multi-band SAR-1 fused images. (a) LAP. (b) RLP. (c) DWT. (d) DTCWT. (e) CVT. (f) NSCT-P. (g) NSCT-S. (h) NSST-M. (i) GF. (j) GRW.
(k) NFT. (l) Our.

Fig. 15. Multi-band SAR-2 fused images. (a) LAP. (b) RLP. (c) DWT. (d) DTCWT. (e) CVT. (f) NSCT-P. (g) NSCT-S. (h) NSST-M. (i) GF. (j) GRW.
(k) NFT. (l) Our.

In order to balance the evaluation metrics, we suggest using
J = 3.
As shown in Fig. 12, when δ increases, the Q W becomes
smaller, while Q G , Q P and Q Y become bigger. Furthermore,
Q E increases first and then decreases with increase of δ.
The values of the metrics changes slowly when δ >= 10.0.
Therefore, δ is set as 10.0 in this paper.
The analysis of parameters α, β and λ are shown in Fig. 13.
When α increases, the Q W and Q E become smaller, while
Q G , Q P and Q Y become bigger. In order to balance the
evaluation metrics, we suggest using α = 0.6. Next, β is
analyzed in the same way, which is set as 0.8 in this paper.
At last, when λ increases, all evaluation metrics have little
change. Therefore, we suggest using λ = 1.0.
862
TABLE I
SAR-VIS
E. Comparison With Other Image Fusion Approaches
In this section, the proposed framelet-based random
walks fusion method is compared with eleven image fusion
approaches based on ratio of low-pass pyramid (RLP) [12],
Laplacian pyramid (LAP) [11], discrete wavelet trans-
form (DWT) [1], curvelet transform (CVT) [16], dual-tree
complex wavelet transform (DTCWT) [18], nonsubsampled
contourlet transform utilizing pulse coupled neural net-
work (NSCT-P) [20], nonsubsampled contourlet transform
utilizing sparse representation (NSCT-S) [27], nonsubsam-
pled shearlet transform utilizing multi-scale top-hat trans-
form (NSST-M) [21], guided filtering (GF) [24], generalized
random walks (GRW) [32] and nonsubsampled framelet trans-
form (NFT), respectively. The comparison results are shown
in Figs. 7-15 and Tables I-III. It should be noted that the best
value of each fusion metric is marked in bold in each table.
1) Qualitative Analysis: In this part, we provide our fusion
results and compare them with eleven fusion approaches
visually. First, Figs. 7-8 show the fused SAR-visible images
obtained by different methods. As seen from Fig. 7(b) and
Fig. 8(b), the RLP method produces fused image with over
high brightness. However, the LAP, DWT, DTCWT, CVT,
NSCT-P, NSST-M, GF and NFT methods decrease the contrast
of the fused images, and thus make some details invisible (see
Fig. 7(a,c,d,e,f,h,i,k) and Fig. 8(a,c,d,e,f,h,i,k)). The NSCT-S
methods introduce artifacts to the fused images (see Fig. 7(g,i)
and Fig. 8(g,i)). The GRW method produces over smooth fused
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 2, FEBRUARY 2018
TABLE II
SAR-IR
image (see Fig. 7(j) and Fig. 8(j)). Therefore, the proposed
approach can preserve spatial details information without
producing artifacts and brightness distortions.
Then, Figs. 9-10 give two examples of SAR
and infrared image fusion. It can be seen that our
approach can improve contrast and preserve the details,
while other methods may over improve brightness
(see Fig. 9(b) and Fig. 10(b)), decrease the contrast
(see Fig. 9(a,c,d,e,f,h,i,k) and Fig. 10(a,c,d,e,f,h,k,i)), produce
artifacts (see Fig. 9(g) and Fig. 10(g)) or reduce sharpness
(see Fig. 9(j) and Fig. 10(j)). Therefore, our proposed
approach takes the advantage of infrared and SAR images,
the fused image displays better visual effect.
Finally, the fused multi-band SAR images are shown
in Figs. 14-15. From the visual view, the fusion results
produced by the LAP, DWT, DTCWT, CVT, NSCT-P,
NSCT-S, GF and NFT methods are not obvious in edge
and texture information (see Fig. 14(a,c,d,e,f,g,i,k) and
Fig. 15(a,c,d,e,f,g,i,k)). Moveover, the RLP, NSST-M and
GRW methods produce fused images with distortion appear-
ance and blurred edges (see Fig. 14(b,h,j) and Fig. 15(b,h,j)).
Relatively, our fused images have higher contrast and sharp-
ness while preserve edge contour information (see Fig. 14(l)
and Fig. 15(l)).
In summary, the results of subjective evaluation demonstrate
the superiority of the proposed fusion approach when com-
pared with other fusion methods.
YANG et al.: RWs FOR SAR IMAGE FUSION IN FRAMELET DOMAIN
TABLE III
M ULTI -BAND SAR
2) Quantitative Analysis: In this part, we quantitatively
evaluate our results using five evaluation metrics (Q W , Q E ,
Q G , Q P and Q Y ) which are mentioned in Section IV-B. The
quantitative results of different image fusion methods for SAR-
visible images, SAR-infrared images and multi-band SAR
images are shown in Tables I-III. According to these values
in Table I, our proposed approach performs slightly worse than
the GF approach when processing the ’SAR-VIS-1’ original
images in terms of Q Y . In ’Multi-band SAR-1’ images,
the NFT method gives the largest value for Q W and the GF
method gives the largest values for Q G and Q Y . In addition,
the value of Q Y is little smaller than the NFT method when
processing the ’Multi-band SAR-2’ images. However, it can’t
reflect the fusion quality in quantitative analysis due to only
one or two evaluation metric with the highest values. Our
presented approach has the best performance in terms of more
than half of the evaluation metrics. Furthermore, the values
in Table II shows that our method have the largest values for
all fusion evaluation metrics. Therefore, it is verified that better
fusion results can be obtained by our proposed approach.
3) Computational Complexity: In this section, we ana-
lyze the computational complexity of the proposed approach.
Firstly, the original images are converted to a framelet trans-
form domain. The computational of this step is O(M 2 ∗
J ∗ 9 ∗ 2). Secondly, the framelet coefficients are merged
by a given random walks fusion rule. In this step,
the computational of constructing compatibility functions
is O(M 2 ∗ J ∗ 9 ∗ 2). Then, we employ conjugate gradient
863
TABLE IV
E LAPSED TIME ( S )
squared method (CGS) [54] to solve the sparse linear sys-
tems (29). The complexity of this direct solver is propor-
tional to the number of iterations (I nter ) and the number
of the nonzero entries in sparse matric. In our case, it is
O(M 2 ∗ J ∗ 9 ∗ 2 ∗ I nter ). Moreover, the complexity of
calculating fused framelet coefficients is O(M 2 ∗ J ∗ 9).
Finally, the computational of reconstructing the fused image is
O(M 2 ∗ J ∗ 9). Therefore, the total computational complexity
of our approach is O(M 2 ∗ J ∗ 9 ∗ 2 ∗ I nter ).
In this paper, all fusion approaches were run on the same
computer with a 2.5-GHz CPU and 4-GB memory available
for Matlab. The elapsed time of each approach is listed
in Table IV. As seen from Table IV, the LAP approach
runs faster than the other fusion methods because it does
not take global or local statistic information into account.
Compared with traditional fusion methods (LAP, RLP, DWT
etc.), the elapsed time of our approach is higher. Because
we have employed the local statistical features of framelet
coefficients in random walks. However, the parallel processing
technique can be used to accelerate the convergence of the
proposed approach. In addition, the running time of our
approach is faster than the NSCT-P, NSCT-S and NSST-M
method. Consequently, the proposed fusion approach can be
considered as one of the efficient SAR image fusion method.
V. C ONCLUSION
In this paper, we propose a novel SAR image fusion
approach based on NFT and RW. On the one hand, original
images can be transformed from spatial domain to framelet
domain by NFT. On the other hand, a new random walks
method is used to estimate the fusion weights of high-
frequency coefficients and low-frequency coefficients, which
can get much more details and features from original images.
To verify the effectiveness of our presented approach, six
experiments for the fusion of SAR-visible, SAR-infrared and
multi-band SAR images are performed. Five image fusion
evaluation metrics are used for the quantitative analysis. The
experiments represent that the presented approach is superior
to traditional and state-of-the-art approaches in terms of pre-
serving edge contour information and improving contrast and
sharpness, which is consistent with subjective fusion effect.
864
ACKNOWLEDGMENT
The authors would like to thank Y. Liu, R. Shen and
Z. Liu for sharing their codes for image fusion approaches
and evaluation metrics. The authors would also like to thank
the reviewers for their valuable comments.
R EFERENCES
[1] H. Li, B. S. Manjunath, and S. K. Mitra, “Multisensor image fusion
using the wavelet transform,” Graph. Models Image Process., vol. 57,
no. 3, pp. 235–245, May 1995.
[2] B. Solaiman, L. E. Pierce, and F. T. Ulaby, “Multisensor data fusion
using fuzzy concepts: Application to land-cover classification using
ERS-1/JERS-1 SAR composites,” IEEE Trans. Geosci. Remote Sens.,
vol. 37, no. 3, pp. 1316–1326, May 1999.
[3] J. Chanussot, G. Mauris, and P. Lambert, “Fuzzy fusion techniques for
linear features detection in multitemporal SAR images,” IEEE Trans.
Geosci. Remote Sens., vol. 37, no. 3, pp. 1292–1305, May 1999.
[4] G. Simone, A. Farina, F. C. Morabito, S. B. Serpico, and L. Bruzzone,
“Image fusion techniques for remote sensing applications,” Inf. Fusion,
vol. 3, no. 1, pp. 3–15, Mar. 2002.
[5] A. Jouan and Y. Allard, “Land use mapping with evidential fusion of
features extracted from polarimetric synthetic aperture radar and hyper-
spectral imagery,” Inf. Fusion, vol. 5, no. 4, pp. 251–267, Dec. 2004.
[6] C. Tison, F. Tupin, and H. Maitre, “A fusion scheme for joint retrieval of
urban height map and classification from high-resolution interferometric
SAR images,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 2,
pp. 496–505, Feb. 2007.
[7] B. Waske and S. van der Linden, “Classifying multilevel imagery from
SAR and optical sensors by decision fusion,” IEEE Trans. Geosci.
Remote Sens., vol. 46, no. 5, pp. 1457–1466, May 2008.
[8] V. Poulain, J. Inglada, M. Spigai, J.-Y. Tourneret, and P. Marthon,
“High-resolution optical and SAR image fusion for building data-
base updating,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 8,
pp. 2900–2910, Aug. 2011.
[9] M. Gong, Z. Zhou, and J. Ma, “Change detection in synthetic aperture
radar images based on image fusion and fuzzy clustering,” IEEE Trans.
Image Process., vol. 21, no. 4, pp. 2141–2151, Apr. 2012.
[10] A. Salentinig and P. Gamba, “A general framework for urban area
extraction exploiting multiresolution SAR data fusion,” IEEE J. Sel.
Topics Appl. Earth Observ. Remote Sens., vol. 9, no. 5, pp. 2009–2018,
May 2016.
[11] W. Xueyun, X. Huaping, L. Jingwen, and W. Pengbo, “Comparison
of diverse approaches for synthetic aperture radar images pixel fusion
under different precision registration,” IET Image Process., vol. 5, no. 8,
pp. 661–670, Dec. 2011.
[12] A. Toet, “Image fusion by a ratio of low-pass pyramid,” Pattern
Recognit. Lett., vol. 9, no. 4, pp. 245–253, May 1989.
[13] Y. Jin, Y. Ruliang, and H. Ruohong, “Pixel level fusion for multiple
SAR images using PCA and wavelet transform,” in Proc. CIE Int. Conf.
Radar (ICR), 2006, pp. 1–4.
[14] L. Alparone, S. Baronti, A. Garzelli, and F. Nencini, “Landsat ETM+
and SAR image fusion based on generalized intensity modulation,” IEEE
Trans. Geosci. Remote Sens., vol. 42, no. 12, pp. 2832–2839, Dec. 2004.
[15] Y. Byun, J. Choi, and Y. Han, “An area-based image fusion scheme
for the integration of SAR and optical satellite imagery,” IEEE J. Sel.
Topics Appl. Earth Observ. Remote Sens., vol. 6, no. 5, pp. 2212–2220,
Oct. 2013.
[16] F. Nencini, A. Garzelli, S. Baronti, and L. Alparone, “Remote sensing
image fusion using the Curvelet transform,” Inf. Fusion, vol. 8, no. 2,
pp. 143–156, Apr. 2007.
[17] X. Ji and G. Zhang, “Image fusion method of SAR and infrared image
based on Curvelet transform with adaptive weighting,” Multimedia Tools
Appl., vol. 76, pp. 17633–17649, Sep. 2017.
[18] J. J. Lewis, R. J. O’Callaghan, S. G. Nikolov, D. R. Bull, and
N. Canagarajah, “Pixel- and region-based image fusion with complex
wavelets,” Inf. Fusion, vol. 8, no. 2, pp. 119–130, Apr. 2007.
[19] Y.-A. Zheng, C. Zhu, J. Song, and X. Zhao, “Fusion of multi-band SAR
images based on contourlet transform,” in Proc. IEEE Int. Conf. Inf.
Acquisition (ICIA), Aug. 2006, pp. 420–424.
[20] S. Yang, M. Wang, Y. Lu, W. Qi, and L. Jiao, “Fusion of multiparametric
SAR images based on SW-nonsubsampled contourlet and PCNN,”
Signal Process., vol. 89, no. 12, pp. 2596–2608, Dec. 2009.
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 27, NO. 2, FEBRUARY 2018
[21] W. Zhi-She, Y. Feng-Bao, P. Zhi-Hao, C. Lei, and J. Li-E, “Multi-
sensor image enhanced fusion algorithm based on NSST and top-hat
transformation,” Optik-Int. J. Light Electron Opt., vol. 126, no. 23,
pp. 4184–4190, Dec. 2015.
[22] O. Loffeld, H. Nies, S. Knedlik, and W. Yu, “Phase unwrapping for SAR
interferometry—A data fusion approach by Kalman filtering,” IEEE
Trans. Geosci. Remote Sens., vol. 46, no. 1, pp. 47–58, Jan. 2008.
[23] Y. Ye, B. Zhao, and L. Tang, “SAR and visible image fusion based on
local non-negative matrix factorization,” in Proc. 9th Int. Conf. Electron.
Meas. Instrum. (ICEMI), 2009, pp. 263–266.
[24] S. Li, X. Kang, and J. Hu, “Image fusion with guided filtering,” IEEE
Trans. Image Process., vol. 22, no. 7, pp. 2864–2875, Jul. 2013.
[25] J. Xu, X. Yu, W. Pei, D. Hu, and L. Zhang, “A remote sensing image
fusion method based on feedback sparse component analysis,” Comput.
Geosci., vol. 85, pp. 115–123, Dec. 2015.
[26] B. K. S. Kumar, “Image fusion based on pixel significance using
cross bilateral filter,” Signal, Image Video Process., vol. 9, no. 5,
pp. 1193–1204, Jul. 2015.
[27] Y. Liu, S. Liu, and Z. Wang, “A general framework for image fusion
based on multi-scale transform and sparse representation,” Inf. Fusion,
vol. 24, pp. 147–164, Jul. 2015.
[28] L. Grady, “Multilabel random walker image segmentation using prior
models,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern
Recognit. (CVPR), Jun. 2005, pp. 763–770.
[29] L. Grady and G. Funka-Lea, “Multi-label image segmentation for
medical applications based on graph-theoretic electrical potentials,” in
Proc. ECCV Workshops CVAMIA MMBIA, 2004, pp. 230–245.
[30] L. Grady, “Random walks for image segmentation,” IEEE Trans. Pattern
Anal. Mach. Intell., vol. 28, no. 11, pp. 1768–1783, Nov. 2006.
[31] V. Gopalakrishnan, Y. Hu, and D. Rajan, “Random walks on graphs for
salient object detection in images,” IEEE Trans. Image Process., vol. 19,
no. 12, pp. 3232–3242, Dec. 2010.
[32] R. Shen, I. Cheng, J. Shi, and A. Basu, “Generalized random walks for
fusion of multi-exposure images,” IEEE Trans. Image Process., vol. 20,
no. 12, pp. 3634–3646, Dec. 2011.
[33] K.-L. Hua, H.-C. Wang, A. H. Rusdi, and S.-Y. Jiang, “A novel multi-
focus image fusion algorithm based on random walks,” J. Vis. Commun.
Image Represent., vol. 25, no. 5, pp. 951–962, Jul. 2014.
[34] X. Dong, J. Shen, L. Shao, and L. Van Gool, “Sub-Markov random walk
for image segmentation,” IEEE Trans. Image Process., vol. 25, no. 2,
pp. 516–527, Feb. 2016.
[35] J.-F. Cai, H. Ji, C. Liu, and Z. Shen, “Framelet-based blind motion
deblurring from a single image,” IEEE Trans. Image Process., vol. 21,
no. 2, pp. 562–572, Feb. 2012.
[36] Y. Shi, X. Yang, and Y. Guo, “Translation invariant directional framelet
transform combined with Gabor filters for image denoising,” IEEE
Trans. Image Process., vol. 23, no. 1, pp. 44–55, Jan. 2014.
[37] I. Daubechies, B. Han, A. Ron, and Z. Shen, “Framelets: MRA-based
constructions of wavelet frames,” Appl. Comput. Harmon. Anal., vol. 14,
no. 1, pp. 1–46, Jan. 2003.
[38] J.-F. Cai, R. H. Chan, and Z. Shen, “A framelet-based image inpainting
algorithm,” Appl. Comput. Harmon. Anal., vol. 24, no. 2, pp. 131–149,
Mar. 2008.
[39] X. Yang, X. Zhang, and Z. Zhu, “Frame-based image denoising using
hidden Markov model,” Int. J. Wavelets, Multiresolution Inf. Process.,
vol. 6, no. 3, pp. 419–432, May 2008.
[40] Y. Shi, X. Yang, and T. Cheng, “Pansharpening of multispectral images
using the nonseparable framelet lifting transform with high vanishing
moments,” Inf. Fusion, vol. 20, pp. 213–224, Nov. 2014.
[41] F. Fang, G. Zhang, F. Li, and C. Shen, “Framelet based pan-sharpening
via a variational method,” Neurocomputing, vol. 129, pp. 362–377,
Apr. 2014.
[42] G. Zhang, Y. Xu, and F. Fang, “Framelet-based sparse unmixing of
hyperspectral images,” IEEE Trans. Image Process., vol. 25, no. 4,
pp. 1516–1529, Apr. 2016.
[43] A. Ron and Z. Shen, “Affine systems in L 2 (Rd ): The analysis of the
analysis operator,” J. Funct. Anal., vol. 148, no. 2, pp. 408–447, 1997.
[44] M. S. Crouse, R. D. Nowak, and R. G. Baraniuk, “Wavelet-based
statistical signal processing using hidden Markov models,” IEEE Trans.
Signal Process., vol. 46, no. 4, pp. 886–902, Apr. 1998.
[45] D. L. Donoho and J. M. Johnstone, “Ideal spatial adaptation by wavelet
shrinkage,” Biometrika, vol. 81, no. 3, pp. 425–455, Aug. 1994.
[46] D. L. Donoho, “De-noising by soft-thresholding,” IEEE Trans. Inf.
Theory, vol. 41, no. 3, pp. 613–627, May 1995.
[47] Z. Wang and A. C. Bovik, “A universal image quality index,” IEEE
Signal Process. Lett., vol. 9, no. 3, pp. 81–84, Mar. 2002.
YANG et al.: RWs FOR SAR IMAGE FUSION IN FRAMELET DOMAIN 865

[48] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Jingkai Wang received the B.S. degree from the
quality assessment: From error visibility to structural similarity,” IEEE School of Mathematical Sciences, Inner Mongolia
Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004. University, Hohhot, China, in 2013. He is currently
[49] G. Piella and H. Heijmans, “A new quality metric for image fusion,” in pursuing the Ph.D. degree with the School of Math-
Proc. 10th Int. Conf. Image Process. (ICIP), 2003, pp. 173–176. ematics and Systems Science, Beihang University,
[50] C. S. Xydeas and V. Petrović, “Objective image fusion performance Beijing, China. His current research interests include
measure,” Electron. Lett., vol. 36, no. 4, pp. 308–309, Feb. 2000. image processing and remote sensing image fusion.
[51] J. Zhao, R. Laganiere, and Z. Liu, “Performance assessment of combina-
tive pixel-level image fusion based on an absolute feature measurement,”
Int. J. Innov. Comput., Inf. Control, vol. 3, no. 6(A), pp. 1433–1447,
Dec. 2007.
[52] C. Yang, J.-Q. Zhang, X.-R. Wang, and X. Liu, “A novel similarity based
quality metric for image fusion,” Inf. Fusion, vol. 9, no. 2, pp. 156–160,
Apr. 2008.
[53] Z. Liu, E. Blasch, Z. Xue, J. Zhao, R. Laganiere, and W. Wu, “Objective
assessment of multiresolution image fusion algorithms for context
enhancement in night vision: A comparative study,” IEEE Trans. Pattern
Anal. Mach. Intell., vol. 34, no. 1, pp. 94–109, Jan. 2012.
[54] P. Sonneveld, “CGS, a fast Lanczos-type solver for nonsymmetric linear
systems,” SIAM J. Sci. Statist. Comput., vol. 10, no. 1, pp. 36–52,
Jan. 1989.

Xiaoyuan Yang received the B.S. degree from the
Department of Mathematics, Liaoning University,
Shenyang, China, in 1985, the M.S. degree from the
Shenyang Computation Institute, Chinese Academy
of Sciences, Shenyang, in 1988, and the Ph.D.
degree from the Department of Computer Science,
Beihang University, Beijing, China, in 2007. She
is currently a Professor with the Department of
Mathematics and Systems Science, Beihang Uni-
versity. Her research interests include computational
harmonic analysis and image processing.
Ridong Zhu received the B.S. degree in information
and computation science from Tianjin Polytechnic
University, China, in 2014. He is currently pursuing
the Ph.D. degree with the School of Mathematics
and Systems Science, Beihang University, China.
His current research interests include pattern recog-
nition and visual tracking.