Abstract – Remote sensing satellites are limited by the

data transmission rates. As a result the MS (MS) bands are
transmitted at lower resolution and only the PAN band
(PAN) is transmitted at its full resolution. The information
contained in the MS bands is invaluable tool for land use
mapping, urban feature extraction, etc. However, the limited
spatial resolution of the MS images reduces the value of this
information. This paper introduces an image fusion
technique for achieving high spatial quality of the fused MS
image while retaining the spectral quality of the original MS
image. The proposed fusion technique integrates both the
intensity-hue-saturation (IHS) and the Laplacian pyramid
transform (LPT) techniques for combining the advantages of
both. The quality of the fused images has acceptable spatial
characteristics and preserves the spectral characteristics.
The experimental results show that the proposed technique is
superior to the other known fusion techniques.
Keywords – Image fusion, IHS transform, Laplacian
pyramid transform, Remote sensing, Spatial and Spectral
quality
I. INTRODUCTION
Image fusion is the process of combining information
from two or more images of a scene into a single
composite image that is more informative and is more
suitable for visual perception or computer processing. The
objective of image fusion is to reduce uncertainty and
minimize redundancy in the output while maximizing
relevant information particular to an application [14].
In remote sensing applications, the fusion of low
resolution MS and high resolution PAN satellite images is
very important issue for many remote sensing and
mapping applications. To effectively utilize such images,
techniques that can combine high-resolution PAN and
low-resolution MS images into one color image are
demanded.
Many image fusion techniques and software tools have
been developed. The well-known techniques are, for
example, (IHS) transform [2], principal component
analysis (PCA) [4], discrete wavelet transform (DWT) [3]-
[8], discrete wavelet frame transform DWFT [6], LPT
[10]- [12], generalized Laplacian pyramid transform
(GLP) [15] and Fast Fourier transform (FFT)-enhanced
IHS method [1].
The IHS and PCA normally improve the spatial
resolution while distort the color composite so it is still
necessary to investigate how to improve the spatial
resolution of color appearance of the images. Later efforts
take advantage from an underlying multiresolution
analysis, by employing the DWT, DWFT, LPT, and GLP.
Although the DWT improves the quality of the fused
images compared to the other fusion techniques, it suffers
from the underlying down-sampling process so its
multiresolution decompositions and the fusion result are
shift variant. This is particularly undesirable when the
source images are not perfectly registered. The shift
variant problem of the DWT can sometimes cause aliasing
in application like pattern recognition or image fusion.
In this paper, we propose a new image fusion technique
based on IHS and LPT. It is known that the IHS transform
introduces images with high spatial quality, while the LPT
is shift invariant and can avoid ringing and blurring the
edges. Integrating both techniques preserves the spectral
quality and enhances the spatial quality of the fused
image.
The testing images are collected from an IKONOS
dataset with 1m PAN and 4 m MS images [13]. The
results compare the proposed fusion technique to the IHS,
DWT, LPT, and IHS-DWT techniques and show that the
new technique gives better discrepancy and correlation
coefficients and reduces the root mean square error of the
fused image.
II. The Proposed IHS-LPT Fusion technique:
The basic idea behind the IHS-LPT integrated fusion
technique is to modify the intensity component of the
input MS image to look more like the input high-
resolution PAN image. LPT One of the most frequently
studied versions of the pyramid transform. Each level of
the LPT is recursively constructed from its lower level by
the following four basic procedures: blurring (low-pass
filtering), sub-sampling (reduce size), interpolation
(expand in size), and differencing (to subtract two images
pixel-by-pixel) in the order we have given [7]- [11]. IHS
An IHS and Laplacian Pyramid Integrated Fusion Technique to
Improve the Visual Quality of Remote Sensing Satellite Images
Rasha Shoitan*, Zaki Nossair**, Tarek El.Tobely***, Hatem El-Bolok**
* Department of Communications Engineering , Modern University of Technology & Science, Cairo, Egypt
** Department of Electronics, Communications & Computer , Helwan University, Egypt
*** Department of Computer and Automatic Control, Tanta University, Egypt
NxM
) B (A
RMSE
M
1 i
N
1 j
2
j i, K, j i, k,
K
¿¿
= =
÷
=
transform separates the spatial (intensity) and spectral
information (hue& saturation) in a RGB image [9] then
replace the intensity with PAN image. Instead of using a
total replacement of the intensity component as with the
IHS technique [5], the IHS-LPT integrated fusion
technique uses a partial replacement based on LPT. The
detailed steps of this proposed technique are shown in Fig.
4 and can be summarized as follows:
1) The low resolution MS image is co-registered to the
same area as the high resolution PAN image and
resampled to the same resolution as the PAN image.
2) The three resampled bands of the MS image which
represent the RGB space are transformed into IHS
components.
3) Constructing one level of LPT for the PAN and
intensity (I) image according to the above four procedures
(blurring, sub-sampling, interpolating and differencing) to
generate P
HFC
and I
HFC
images, respectively.
4) substituting the I
HFC
with the P
HFC
5) Adding the new I
HFC
which is obtained from step 3
with I
LFC
to generate new intensity(I
new
)
6) Transforming the new intensity together with the hue
and saturation components back into RGB space.
Fig. 4. The integrated IHS-LPT fusion technique
III. Evaluations metrics
The emphasis of this paper is to produce fused image
that retaining the spectral quality of the original MS
images while achieving a high spatial quality. There are
mainly two ways to evaluate the quality of the fused
image -spectral and spatial quality metrics.
A. Spectral Quality Metrics
There are various spectral quality metrics used to
evaluate the spectral characteristics of the fused images
with respect to the original MS images
- Correlation Coefficient (CC
K
)
The correlation coefficient measures the closeness or
similarity between two Images [10]- [6]. It can vary
between –1 to +1. The formula to compute the correlation
between two images A (original MS image) and B (fused
image), both of size NxM pixels is given by
(1)
Where B , A is the mean of the fused and original MS
image respectively,
j i K
B
, ,
and
j i K
A
, ,
are the pixel values
at position ) , ( j i in the Kth band of the fused and original
MS image, respectively
- Discrepancy (D
K
)
The discrepancy measures the spectral quality of the
merged images band-by-band by taking the average
differences between the fused image and the
corresponding original MS band [6], which can be
described by the formula
¿¿
= =
÷ =
M
1 i
N
1 j
j i, K, j i, K, K
A B
NxM
1
D
(2)
Where
j i K
B
, ,
and
j i K
A
, ,
are the pixel values at
position ) , ( j i in the Kth band of the fused and original
MS image, respectively. The lower the value of the
discrepancy, the higher the spectral quality of the fused
images
- Root Mean Square Error (RMSE
K
)
The Root Mean Square Error (RMSE) measures the
amount of change per pixel due to the processing [10] and
is described by:
(3)
The lower the value of the RMSE, the higher the
spectral quality of the fused images
¿ ¿ ¿ ÷ × ¿ ÷
¿ ¿ ÷ ÷
= =
= = =
÷
=
÷
= =
÷ ÷
M
1 i
256
1 i
256
1 j
2
j i, k,
N
1 j
2
j i, k,
M
1 i
N
1 j
j i, k, j i, k,
K
) B (B ) A (A
) B )(B A (A
(A/B) Corr CC
B. Spatial Quality Metrics
The spatial quality evaluation of the fused images is
important since the goal is to retain the high spatial
resolution of the PAN image in the fused image.
- High Pass Correlation Coefficient (HPCC
k
)
The HPCC
k
Measures the amount of edge information
from the PAN image that is transferred into the fused
image. The high spatial resolution information missed in
the MS image is present in the high frequencies of the
PAN image. The fusion process injects the higher
frequencies from the PAN image into the multipectral
image [6]. A higher correlation between the two highpass
filtered images implies that the spatial information has
been retained faithfully. The two images were highpass
filtered using the Laplacian kernel which is described by
the following the equation:
(
(
(
¸
(

¸

÷ ÷ ÷
÷ ÷
÷ ÷ ÷
=
1 1 1
1 8 1
1 1 1
HP
(4)
Let HP(P) be the high pass filtered PAN image and
HP(B) be the high pass filtered fused image. Then the
HPCC
k
is given by:
)) HP(B / Corr(HP(P) HPCC
K K
=
(5)
IV.Experimental Results and Data Discussion
To assess the quality of the fused images, the evaluation
metrics mentioned in the previous sections are employed
to evaluate the proposed IHS-LPT integrated fusion
technique and compare the results of different fusion
techniques such as the IHS, DWT, LPT , and integrated
IHS-DWT techniques.
It is known that the good fusion scheme should preserve
the spectral characteristics of the source MS image as well
as the high spatial resolution characteristics of the source
PAN image. The resulting fused images from the IHS,
DWT, LPT, the Integrated IHS-DWT and the proposed
integrated IHS-LPT fusion techniques are shown in Fig. 5
for IKONOS satellites. The values of spectral quality and
spatial quality metrics are listed in Table I.
It is clear that the IHS technique generally offers a
satisfactory spatial resolution but it tends to produce
distortion in the spectral characteristics of the original MS
images. The DWT approach preserve the spectral
characteristics of MS images compared to the IHS
technique, though images fused by wavelet-based methods
contain much less spatial information than images fused
by IHS methods. Considering the results of the proposed
method, it has lower values for the D
k
, RMSE, and greater
values for the CC
k
. Hence, the spectral quality of the
resulting images is much better than the spectral quality of
the images that are fused by the other techniques. On the
other side, the values of the HPCC
k
in the proposed
techniques are slightly lower than the corresponding
values in the IHS and LPT techniques, but this difference
is insignificant for the quality of the produced images
considering the gained improvements in the other
parameters for the spectral quality.
V.CONCLUSION
This paper introduces a new fusion method based on
integrating Laplacian pyramid transform and IHS
transform by fusing high spatial resolution PAN
IHS DWT LPT IHS+DWT LPT+IHS
R 0.6438 0.8468 0.9219 0.8784 0.9379
G 0.6038 0.8452 0.9208 0.8633 0.9331
C
C
k
B 0.6453 0.7962 0.9203 0.8906 0.9414
R 0.1865 0.1195 0.0842 0.1041 0.0717
G 0.1991 0.1199 0.0845 0.1107 0.0754
D
K
B 0.1819 0.1420 0.0865 0.1018 0.0705
R
0.2383 0.1523 0.1152 0.1363 0.1025
G 0.2527 0.1521 0.1154 0.1435 0.1058
S
p
e
c
t
r
a
l

q
u
a
l
i
t
y
R
M
S
E
K
B 0.231 0.1766 0.1179 0.1326 0.1005
R 0.9327 0.9417 0.9882 0.8944 0.9436
G 0.9482 0.9413 0.9881 0.9053 0.954
S
p
a
t
i
a
l

q
u
a
l
i
t
y
H
P
C
C
k
B
0.9263 0.9385 0.9856 0.8874 0.9398
Table I
IKONOS Image Fusion Results
images and low resolution MS images.
Different metrics are computed to assess the spectral
and spatial qualities of the images that are fused by IHS,
DWT, LPT, integrated IHS-DWT, and the proposed
fusion technique in this paper. The presented results show
that the new technique enhances both the spatial quality
and spectral quality of the fused images. The computed
metrics for the spectral quality of the fused images are
highly improved in the proposed technique compared to
all known fusion techniques, on the other side the spatial
quality is also improved and its computed metric is almost
equal in IHS, and the proposed technique.
It is concluded that the proposed fusion technique is
superior to all know fusion techniques specially IHS and
DWT techniques, where it can produce images with better
spectral quality than IHS and DWT and with the same
spatial quality characteristic as in these two techniques.
REFERENCES
[1] Yangrong Ling , Manfred Ehlers , E. Lynn Usery,
Marguerite Madden,” FFT-enhanced IHS transform
method for fusing high-resolution satellite images”,
ISPRS Journal of Photogrammetry & Remote
Sensing, vol. 61, pp. 381–392, 2007.
[2] Choi M., “A new intensity-hue-saturation fusion
approach to image Fusion with a tradeoff
parameter”, IEEE transactions on geosciences and
remote sensing, vol.44, no. 6, pp.1391-1402, 2006.
[3] Yun Zhang, Gang Hong, “An IHS and wavelet
integrated approach to improve pan-sharpening
visual quality of natural colour IKONOS and
QuickBird images”, Information Fusion 6, pp. 225–
234, 2005.
[4] María González-Audícana, José Luis Saleta, Raquel
García Catalán, Rafael García, “Fusion of MS and
PAN images using improved IHS and PCA merges
based on wavelet decomposition”, IEEE transactions
on geosciences and remote sensing , vol.42, no. 6,
pp.1291-1299, 2004.
[5] Te-Ming Tu, Shun-Chi Su, Heuen-Chyun Shyu, Ping
S.Huang, ”A new look at IHS-like image fusion
methods”, Information fusion 2, pp.177-186.
[6] Shutao Li, James T. Kwok, Yaonan Wang, “Using
the discrete wavelet frame transform to merge
Landsat TM and SPOT PAN images”, Information
Fusion 3, pp.17–23, 2002.
[7] Peter J. Burt, Edward H., “The Laplacian pyramid
transform as a compact image code”, IEEE
transactions on communication, Vol. com-3l, no. 4,
1983.
[8] Pajares G., Jesus M., “A wavelet-based image fusion
tutorial”, Pattern Recognition 37, pp.1855–1872,
2005.
[9] R.C. Gonzalez and R.E. Woods, Digital Image
Processing, 2nd edition, prentice Hall, 2002.
[10] Pushkar pradhan, Multiresolution based,
Multisensor, MS Image Fusion, PhD thesis,
Mississippi State University, 2005.
[11] Rick S. Blum and Zheng Liu, Multi-sensor image
fusion and its applications, CRC Press Taylor &
Francis Group,2006.
[12] Ying-Ying Shih, Analysis of panchromatic and
Multi-spectrum image fusion, M.Sc thesis, Tatung
university, 2004.
[13] The IKONOS web site,
http://studio.gge.unb.ca/UNB/zoomview/examples.h
tml.
[14] A. Ardeshir Goshtasby, Stavri Nikolov,” image
fusion :advances in the state of art”, information
fusion 8, pp.114-118, 2007
[15] Bruno Aiazzi, Stefano Baronti, Andrea Garzelli,
Leonardo Santurri, Massimo Selva,” spatial
enhancement of TIR aster data via VNIR images and
generalized Laplacian decomposition”, Proceedings
of 4th EARSeL Workshop on Imaging Spectroscopy
,Warsaw 2005.
(a) (b) (c) (d) (e) (f) (g)
Fig. 5. IKONOS images. (a) original panchromatic image; (b) original multispectral image with R, G, B bands; (c) IHS fusion result; (d)
DWT fusion result; (e) LPT fusion result; (f) Integrated IHS-DWT fusion result; (g) Integrated IHS-LPT fusion results.

j are the pixel values at position (i. the IHS-LPT integrated fusion technique uses a partial replacement based on LPT. 2) The three resampled bands of the MS image which represent the RGB space are transformed into IHS components. The detailed steps of this proposed technique are shown in Fig. B K . j NxM i 1 j1 (2) Where B K . respectively  Discrepancy (DK) The discrepancy measures the spectral quality of the merged images band-by-band by taking the average differences between the fused image and the corresponding original MS band [6]. mainly two ways to evaluate the quality of the fused image -spectral and spatial quality metrics.i.i .i . Instead of using a total replacement of the intensity component as with the IHS technique [5]. j  B K. Evaluations metrics The emphasis of this paper is to produce fused image that retaining the spectral quality of the original MS images while achieving a high spatial quality.i . j  B) Where B . j  A)(Bk. interpolating and differencing) to generate PHFC and IHFC images. j  A K.i. A. A is the mean of the fused and original MS image respectively. 4) substituting the IHFC with the PHFC 5) Adding the new IHFC which is obtained from step 3 with ILFC to generate new intensity(Inew) 6) Transforming the new intensity together with the hue and saturation components back into RGB space.transform separates the spatial (intensity) and spectral information (hue& saturation) in a RGB image [9] then replace the intensity with PAN image.i . j and AK .i. It can vary between –1 to +1. respectively.i. Spectral Quality Metrics There are various spectral quality metrics used to evaluate the spectral characteristics of the fused images with respect to the original MS images  Correlation Coefficient (CCK) The correlation coefficient measures the closeness or similarity between two Images [10]-[6]. i. j  A)    (Bk. i. the higher the spectral quality of the fused images  Root Mean Square Error (RMSEK) The Root Mean Square Error (RMSE) measures the amount of change per pixel due to the processing [10] and is described by: Fig. 4 and can be summarized as follows: 1) The low resolution MS image is co-registered to the same area as the high resolution PAN image and resampled to the same resolution as the PAN image.i.i. There are RMSE K   i 1 j1 M N (A k. which can be described by the formula DK  1 M N  BK. j ) 2 NxM (3) The lower the value of the RMSE. j are the pixel values at position (i. The formula to compute the correlation between two images A (original MS image) and B (fused image). The lower the value of the discrepancy. both of size NxM pixels is given by   (A k. j and AK . j  B)  256256 i 1 j1  M N   CC K  Corr (A/B)  i 1 j1 M N (1) i 1 j1 2 2   (A k. The integrated IHS-LPT fusion technique III. the higher the spectral quality of the fused images . respectively. j ) in the Kth band of the fused and original MS image. sub-sampling. 3) Constructing one level of LPT for the PAN and intensity (I) image according to the above four procedures (blurring. 4. j ) in the Kth band of the fused and original MS image.

1154 0. DWT.8944 0.  High Pass Correlation Coefficient (HPCCk) The HPCCk Measures the amount of edge information from the PAN image that is transferred into the fused image.1199 0. The high spatial resolution information missed in the MS image is present in the high frequencies of the PAN image.9398 Spectral quality Spatial quality HPCCk RMSEK DK .9482 0.9414 0.0717 0.1865 0.1523 0.9379 0. LPT .1025 0.Experimental Results and Data Discussion To assess the quality of the fused images.9219 0.1195 0.7962 0.1041 0.0845 0.9882 0. Hence.6438 0. The two images were highpass filtered using the Laplacian kernel which is described by the following the equation:  1  1  1 HP   1 8  1     1  1  1  (4) Let HP(P) be the high pass filtered PAN image and HP(B) be the high pass filtered fused image. Then the HPCCk is given by: HPCC K  Corr(HP(P) / HP(B K )) (5) technique and compare the results of different fusion techniques such as the IHS.8633 0. The DWT approach preserve the spectral characteristics of MS images compared to the IHS technique.8468 0.6038 0.1991 0.1766 0.1058 0.1819 0. the spectral quality of the resulting images is much better than the spectral quality of the images that are fused by the other techniques.231 0.954 0. DWT. the values of the HPCCk in the proposed techniques are slightly lower than the corresponding values in the IHS and LPT techniques. LPT.1018 0.9436 0.0865 0.1005 0. It is known that the good fusion scheme should preserve the spectral characteristics of the source MS image as well as the high spatial resolution characteristics of the source PAN image.9053 0. V.9208 0.8784 0. the evaluation metrics mentioned in the previous sections are employed to evaluate the proposed IHS-LPT integrated fusion Table I IKONOS Image Fusion Results IHS R G B R G B R G B R G B 0.1420 0. RMSE.2383 0.9263 CCk DWT 0.9385 LPT 0.2527 0.9203 0.8906 0. On the other side.1326 0.9327 0.B. A higher correlation between the two highpass filtered images implies that the spatial information has been retained faithfully.9413 0. The fusion process injects the higher frequencies from the PAN image into the multipectral image[6].9331 0.CONCLUSION This paper introduces a new fusion method based on integrating Laplacian pyramid transform and IHS transform by fusing high spatial resolution PAN IV.8452 0. though images fused by wavelet-based methods contain much less spatial information than images fused by IHS methods.1152 0.0754 0. It is clear that the IHS technique generally offers a satisfactory spatial resolution but it tends to produce distortion in the spectral characteristics of the original MS images.9856 IHS+DWT 0. 5 for IKONOS satellites.1435 0. The resulting fused images from the IHS. The values of spectral quality and spatial quality metrics are listed in Table I.8874 LPT+IHS 0. and greater values for the CCk.1521 0. the Integrated IHS-DWT and the proposed integrated IHS-LPT fusion techniques are shown in Fig. it has lower values for the Dk. and integrated IHS-DWT techniques.1179 0.6453 0.1107 0. but this difference is insignificant for the quality of the produced images considering the gained improvements in the other parameters for the spectral quality. Considering the results of the proposed method.1363 0.9881 0.0842 0. Spatial Quality Metrics The spatial quality evaluation of the fused images is important since the goal is to retain the high spatial resolution of the PAN image in the fused image.9417 0.0705 0.

Digital Image Processing. Gonzalez and R. vol. [5] Te-Ming Tu. Multisensor. pp. 2005. “Using the discrete wavelet frame transform to merge Landsat TM and SPOT PAN images”. 4. Andrea Garzelli. pp. Ardeshir Goshtasby. Multi-sensor image fusion and its applications. Information fusion 2. Gang Hong. [7] Peter J. [2] Choi M. MS Image Fusion. [8] Pajares G. 2002. (c) IHS fusion result. Proceedings of 4th EARSeL Workshop on Imaging Spectroscopy . Lynn Usery.” image fusion :advances in the state of art”. pp. [3] Yun Zhang. (e) LPT fusion result.Sc thesis. [13] The IKONOS web site. Multiresolution based.h tml. http://studio. 2005. Manfred Ehlers .unb. Yaonan Wang.. Information Fusion 3.2006. Mississippi State University.C. CRC Press Taylor & Francis Group. 5. James T. Massimo Selva. It is concluded that the proposed fusion technique is superior to all know fusion techniques specially IHS and DWT techniques.” spatial enhancement of TIR aster data via VNIR images and generalized Laplacian decomposition”. Raquel García Catalán. images and low resolution MS images. no. com-3l. Different metrics are computed to assess the spectral and spatial qualities of the images that are fused by IHS.1291-1299. IEEE transactions on geosciences and remote sensing . 381–392. and the proposed fusion technique in this paper. Stavri Nikolov. pp. Blum and Zheng Liu. vol. REFERENCES [1] Yangrong Ling .1391-1402. Tatung university. 1983. Marguerite Madden. Woods. 2nd edition.44. (d) DWT fusion result. pp. . 6. ”A new look at IHS-like image fusion methods”. The presented results show that the new technique enhances both the spatial quality and spectral quality of the fused images. 2006. José Luis Saleta. IEEE transactions on communication. Heuen-Chyun Shyu. Leonardo Santurri. pp. (b) original multispectral image with R. DWT. no. no. Edward H. IEEE transactions on geosciences and remote sensing. vol. 225– 234. B bands. ISPRS Journal of Photogrammetry & Remote Sensing. Jesus M.Warsaw 2005. Stefano Baronti. Shun-Chi Su. E..42. [12] Ying-Ying Shih.17–23. [6] Shutao Li. The computed metrics for the spectral quality of the fused images are highly improved in the proposed technique compared to all known fusion techniques. pp. “Fusion of MS and PAN images using improved IHS and PCA merges based on wavelet decomposition”. 6. Rafael García. 2004. Information Fusion 6. “A wavelet-based image fusion tutorial”. information fusion 8.114-118. “An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images”. prentice Hall. M. “A new intensity-hue-saturation fusion approach to image Fusion with a tradeoff parameter”. [11] Rick S. 61.ca/UNB/zoomview/examples. LPT. Vol. [10] Pushkar pradhan. 2007. (g) Integrated IHS-LPT fusion results. 2007 [15] Bruno Aiazzi.. where it can produce images with better spectral quality than IHS and DWT and with the same spatial quality characteristic as in these two techniques. [14] A.. Analysis of panchromatic and Multi-spectrum image fusion. G.1855–1872.(a) (b) (c) (d) (e) (f) (g) Fig. [4] María González-Audícana. 2005. (a) original panchromatic image.” FFT-enhanced IHS transform method for fusing high-resolution satellite images”. 2002. on the other side the spatial quality is also improved and its computed metric is almost equal in IHS. Ping S. Burt. pp.E. PhD thesis.Huang. [9] R. “The Laplacian pyramid transform as a compact image code”. integrated IHS-DWT. 2004. IKONOS images. Pattern Recognition 37. (f) Integrated IHS-DWT fusion result.177-186. Kwok. and the proposed technique.gge.