Professional Documents
Culture Documents
1110 4970 PDF
1110 4970 PDF
Ali A. Al-Zaky
Assistant Professor,
Dept. of Physics, College of Science,
Mustansiriyah Un., Baghdad – Iraq
dr.alialzuky@yahoo.com
Abstract: Various and different methods can be used to produce high-resolution multispectral images from high-resolution panchromatic
image (PAN) and low-resolution multispectral images (MS), mostly on the pixel level. However, the jury is still out on the benefits of a
fused image compared to its original images. There is also a lack of measures for assessing the objective quality of the spatial resolution
for the fusion methods. Therefore, an objective quality of the spatial resolution assessment for fusion images is required. So, this study
attempts to develop a new qualitative assessment to evaluate the spatial quality of the pan sharpened images by many spatial quality
metrics. Also, this paper deals with a comparison of various image fusion techniques based on pixel and feature fusion techniques.
Keywords: Measure of image quality; spectral metrics; spatial metrics; Image Fusion.
image, should increase the spectral fidelity while
I. INTRODUCTION: retaining the spatial resolution of the PAN). They
take into account local measurements to estimate
Image fusion is a process, which creates a new how well the important information in the source
image representing combined information composed images is represented by the fused image. In
from two or more source images. Generally, one addition, this study focuses on cambering that the
aims to preserve as much source information as best methods based on pixel fusion techniques (see
possible in the fused image with the expectation that section 2) are those with the fallowing feature fusion
performance with the fused image will be better than, techniques: Segment Fusion (SF), Principal
or at least as good as, performance with the source Component Analysis based Feature Fusion (PCA)
images [1]. Image fusion is only an introductory and Edge Fusion (EF) in [7].
stage to another task, e.g. human monitoring and The paper organized as follows .Section II gives
classification. Therefore, the performance of the the image fusion techniques; Section III includes the
fusion algorithm must be measured in terms of quality of evaluation of the fused images; Section IV
improvement or image quality. Several authors covers the experimental results and analysis then
describe different spatial and spectral quality analysis subsequently followed by the conclusion.
techniques of the fused images. Some of them enable
subjective, the others objective, numerical definition II. IMAGE FUSION TECHNIQUES
of spatial or spectral quality of the fused data [2-5].
The evaluation of the spatial quality of the pan- Image fusion techniques can be divided into three
sharpened images is equally important since the goal levels, namely: pixel level, feature level and decision
is to retain the high spatial resolution of the PAN level of representation [8-10]. The image fusion
image. A survey of the pan sharpening literature techniques based on pixel can be grouped into several
revealed there were very few papers that evaluated techniques depending on the tools or the processing
the spatial quality of the pan-sharpened imagery [6]. methods for image fusion procedure. In this work
Consequently, there are very few spatial quality proposed categorization scheme of image fusion
metrics found in the literatures. However, the jury is techniques Pixel based image fusion methods
still out on the benefits of a fused image compared to summarized as the fallowing:
its original images. There is also a lack of measures a. Arithmetic Combination techniques: such as Bovey
for assessing the objective quality of the spatial Transform (BT) [11-13]; Color Normalized
resolution of the fusion methods. Therefore, an Transformation (CN) [14, 15]; Multiplicative
objective quality of the spatial resolution assessment Method (MLT) [17, 18].
for fusion images is required. b. Component Substitution fusion techniques: such as
HIS, HIS, HSV, HLS and YIQ in [19].
Therefore, this study presented a new approach to c. Frequency Filtering Methods :such as in [20] High-
assess the spatial quality of a fused image based on Pass Filter Additive Method (HPFA) , High –
High pass Division Index (HPDI). In addition, many Frequency- Addition Method (HFA) , High
spectral quality metrics, to compare the properties of Frequency Modulation Method (HFM) and The
fused images and their ability to preserve the Wavelet transform-based fusion method (WT).
similarity with respect to the original MS image d. Statistical Methods: such as in [21] Local Mean
while incorporating the spatial resolution of the PAN Matching (LMM), Local Mean and Variance
1
Matching (LMVM), Regression variable substitution images. En reflects the capacity of the information
(RVS), and Local Correlation Modeling (LCM). carried by images. The larger En mean high
information in the image [6]. By applying Shannon’s
All the above techniques employed in our previous studies entropy in evaluation the information content of an
[19-21]. Therefore, the best method for each group selected image, the formula is modified as [23]:
in this study as the fallowing: En = − ∑ଶହହ
୧ୀ P(i)log ଶP(i) (2)
Where P(i) is the ratio of the number of the pixels with
a. Arithmetic and Frequency Filtering techniques are High gray value equal to ݅over the total number of the pixels.
–Frequency- Addition Method (HFA) and High c. Signal-to Noise Ratio (ࡿࡺࡾ): The signal is the
Frequency Modulation Method (HFM) [20]. information content of the data of original MS
b. The Statistical Methods it was with Regression variable imageM୩, while the merging ܨ can cause the noise,
substitution (RVS) [21]. as error that is added to the signal. The ܴܵ ܯ of the
c. In the Component Substitution fusion techniques the signal-to-noise ratio can be used to calculate the
IHS method by [22] it was much better than the others signal-to-noise ratio ܴܵܰ, given by [24]:
methods [19].
∑
∑ೕ (ிೖ (,))
మ
To explain the algorithms through this study, Pixels ܴܵܰ = ඨ ∑ ∑ (3)
మ
should have the same spatial resolution from two different ೕ (ிೖ (,)ିெ ೖ(,))
sources that are manipulated to obtain the resultant image.
Here, The PAN images have a different spatial resolution d. Deviation Index (ࡰࡵ): In order to assess the quality
from that of the original multispectral MS images.
of the merged product in regard of spectral
Therefore, resampling of MS images to the spatial resolution
of PAN is an essential step in some fusion methods to bring information content. The deviation index is useful
the MS images to the same size of PAN, thus the resampled parameter as defined by [25,26], measuring the
MS images will be noted by Μ୩ that represents the set of normalized global absolute difference of the fused
DN of band ݇ in the resampled MS image . imageܨ with the original MS image ܯ as follows :
This section describes the various spatial and spectral e. Correlation Coefficient (): The correlation
quality metrics used to evaluate them. The spectral fidelity coefficient measures the closeness or similarity
of the fused images with respect to the original multispectral between two images. It can vary between –1 to +1. A
images is described. When analyzing the spectral quality of value close to +1 indicates that the two images are
the fused images we compare spectral characteristics of very similar, while a value close to –1 indicates that
images obtained from the different methods, with the they are highly dissimilar. The formula to compute
spectral characteristics of resampled original multispectral the correlation between F୩ , M୩ :
images. Since the goal is to preserve the radiometry of the
original MS images, any metric used must measure the ∑
∑ೕ (ிೖ (,)ିிೖ )(ெ ೖ (,)ିெ ೖ)
amount of change in DN values in the pan-sharpened image = ܥܥ (5)
ට ∑ మ
∑ೕ (ிೖ (,)ିிೖ ) ට ∑ ∑ೕ (ெ ೖ (,)ିெ ೖ )
మ
F୩compared to the original imageM୩. Also, In order to
evaluate the spatial properties of the fused images, a
panchromatic image and intensity image of the fused image Since the pan-sharpened image larger (more pixels) than
have to be compared since the goal is to retain the high the original MS image it is not possible to compute the
spatial resolution of the PAN image. In the following correlation or apply any other mathematical operation
F୩ , M୩are the measurements of each the brightness values between them. Thus, the upsampled MS image M୩is used
pixels of the result image and the original MS image of for this comparison.
band݇, M ഥ ୩ and Fത୩are the mean brightness values of both f. Normalization Root Mean Square Error (NRMSE):
images and are of size n ∗ m . BV is the brightness value the NRMSE used in order to assess the effects of
of image data Mഥ ୩ and Fത୩. information changing for the fused image. When
level of information loss can be expressed as a
A. Spectral Quality Metrics: function of the original MS pixel M୩ and the fused
a. Standard Deviation (ࡿࡰ): The standard deviation pixelF୩, by using the NRMSE between M୩ and
(SD), which is the square root of variance, reflects F୩images in band k. The Normalized Root- Mean-
the spread in the data. Thus, a high contrast image Square Error ܴܰܧܵ ܯ between ܨ and ܯ is a point
will have a larger variance, and a low contrast image analysis in multispectral space representing the
will have a low variance. It indicates the closeness of amount of change the original MS pixel and the
the fused image to the original MS image at a pixel corresponding output pixels using the following
level. The ideal value is zero. equation [27]:
∑
సభ ∑ೕసభ((, )ିఓ)
మ ଵ
∑ ∑ (ܨ (݅, ݆) − ܯ (݅, ݆))ଶ (6)
ߪ=ට (1) ܴܰܧܵ ܯ = ට
∗ଶହହమ
×
b. Entropy(ࡱ): The entropy of an image is a measure
of information content but has not been used to B. Spatial Quality Metrics:
assess the effects of information change in fused a. Mean Grades (MG): MG has been used as a measure
of image sharpness by [27, 28]. The gradient at any
pixel is the derivative of the DN values of C. Filtered Correlation Coefficients (FCC):
neighboring pixels. Generally, sharper images have This approach was introduced [33]. In the Zhou’s
higher gradient values. Thus, any image fusion approach, the correlation coefficients between the high-pass
method should result in increased gradient values filtered fused PAN and TM images and the high-pass
because this process makes the images sharper filtered PAN image are taken as an index of the spatial
compared to the low-resolution image. The gradient quality. The high-pass filter is known as a Laplacian filter as
defines the contrast between the details variation of illustrated in eq. (12):
pattern on the image and the clarity of the image [5]. −1 −1 −1
MG is the index to reflect the expression ability of mask = −1 8 −1൩ (12)
the little detail contrast and texture variation, and the −1 −1 −1
definition of the image. The calculation formula is
[6]: However, the magnitude of the edges does not
necessarily have to coincide, which is the reason why Zhou
ଵ ∆ூೣమ ା∆ூ మ
( = ̅ܩ ିଵ ିଵ ට
∑ୀଵ ∑ୀଵ (7) et al proposed to look at their correlation coefficients [33].
ିଵ)(ିଵ) ଶ
So, in this method the average correlation coefficient of the
Where faltered PAN image and all faltered bands is calculated to
∆ܫ௫ = ݂(݅+ 1, ݆) − ݂(݅, ݆) obtain FCC. An FCC value close to one indicates high
∆ܫ௬ = ݂(݅, ݆+ 1) − ݂(݅, ݆) (8) spatial quality.
Where ∆ܫ௫and ∆ܫ௬ are the horizontal and vertical D. High Pass Deviation Index (HPDI)
gradients per pixel of the image fused ݂(݅, ݆). generally, the This approach proposed by [25, 26] as the measuring of
larger ̅ܩ, the more the hierarchy, and the more definite the the normalized global absolute difference for spectral
fused image. quantity for the fused imageܨ with the original MS
b. Soble Grades (SG): this approach developed in this image ܯ . This study developed that is quality metric to
study by used the Soble operator is A better edge measure the amount of edge information from the PAN
estimator than the mean gradient. That by computes image is transferred into the fused images by used the high-
discrete gradient in the horizontal and vertical pass filter (eq. 12). which that the high-pass filtered PAN
directions at the pixel location ݅, ݆of an image ݂(݅, ݆). image are taken as an index of the spatial quality. The HPDI
The Soble operator was the most popular edge wants to extract the high frequency components of the PAN
detection operator until the development of edge image and each ܨ band. The deviation index between the
detection techniques with a theoretical basis. It high pass filtered ܲ and the fused ܨ images would indicate
proved popular because it gave a better performance how much spatial information from the PAN image has been
contemporaneous edge detection operator than other incorporated into the ܵ ܯimage to obtain HPDI as follows:
such as the Prewitt operator [30]. For this, which is
ଵ |ிೖ (,)ି(,)|
clearly more costly to evaluate, the orthogonal ܫܦܲܪ = ∑ ∑ (13)
(,)
components of gradient as the following [31]:
݅(݂{ = ݔܩ− 1, ݆ + 1) + 2݂(݅ − 1, ݆) + ݂(݅ − 1, ݆ − 1)} The smaller value HPDI the better image quality.
− {݂(݅ + 1, ݆ + 1) + 2݂(݅ + 1, ݆) + ݂(݅ + 1, ݆ − 1)} Indicates that the fusion result it has a high spatial resolution
And quality of the image.
݅(݂{ = ݕܩ− 1, ݆ + 1) + 2݂(݅, ݆ + 1) + ݂(݅ + 1, ݆ + 1)}
− {݂(݅ − 1, ݆ − 1) + 2݂(݅, ݆ − 1) + ݂(݅ + 1, ݆ − 1)} IV. EXPERIMENTAL RESULTS
(9)
The above assessment techniques are tested on fusion of
It can be seen that the Soble operator is equivalent to Indian IRS-1C PAN of the 5.8- m resolution panchromatic
simultaneous application of the templates as the following band and the Landsat TM the red (0.63 - 0.69 µm), green
[32]: (0.52 - 0.60 µm) and blue (0.45 - 0.52 µm) bands of the 30
1 2 1 −1 0 1 m resolution multispectral image were used in this work.
ܩ௫ = 0 0 0൩ ܩ௬ = −2 0 2൩ (10) Fig.1 shows the IRS-1C PAN and multispectral TM images.
−1 −2 −1 −1 0 1 Hence, this work is an attempt to study the quality of the
images fused from different sensors with various
Then the discrete gradient ܩof an image ݂(݅, ݆) is given by characteristics. The size of the PAN is 600 * 525 pixels at 6
bits per pixel and the size of the original multispectral is 120
ଵ ீೣ2 +ீ 2 * 105 pixels at 8 bits per pixel, but this is upsampled by
( = ̅ܩ ∑(
݅=1
ିଵ) (ିଵ)
∑݆=1 ට (11) nearest neighbor to same size the PAN image. The pairs of
ିଵ)(ିଵ) 2
images were geometrically registered to each other. The
Where G୶and G୷ are the horizontal and vertical gradients HFA, HFM, HIS, RVS, PCA, EF, and SF methods are
per pixel. Generally, the larger values forGഥ , the more the employed to fuse IRS-C PAN and TM multi-spectral
hierarchy and the more definite the fused image. images. The original MS and PAN are shown in (Fig. 1).
52.79 5.765
R 9.05 0.068 0.08 0.943
3 1
5.783
HFA G 53.57 8.466 0.07 0.087 0.943
3
54.49 5.791
B 7.9 0.071 0.095 0.943
8 5
5.925
R 52.76 8.399 0.073 0.082 0.934
9
53.34 5.897
HFM G 8.286 0.071 0.084 0.94
3 9
54.13 5.872
B 8.073 0.069 0.086 0.945
6 1
41.16
R 7.264 6.583 0.088 0.104 0.915
4
41.98
HIS G 7.293 6.4 0.086 0.114 0.917
6
42.70
B 7.264 5.811 0.088 0.122 0.917
9
47.87 5.196
R 6.735 0.105 0.199 0.984
5 8
49.31 5.248
PCA G 6.277 0.108 0.222 0.985
3 5
51.09 5.294
B 5.953 0.109 0.245 0.986
2 1
Fig.1: The Representation of Original Panchromatic and
51.32 5.884
Multispectral Images R 7.855 0.078 0.085 0.924
3 1
51.76 5.847
V. ANALYSISES RESULTS RVS G 7.813 0.074 0.086 0.932
9 5
52.37 5.816
B 7.669 0.071 0.088 0.938
A. Spectral Quality Metrics Results: 4 6
51.60
From table1 and Fig. 2 shows those parameters for the R
3
5.687 9.221 0.067 0.09 0.944
fused images using various methods. It can be seen that from 52.20 5.704
SF G 8.677 0.067 0.098 0.944
Fig. 2a and table1 the SD results of the fused images remains 7 7
constant for all methods except the IHS. According to the 53.02 5.712
B 8.144 0.068 0.108 0.945
8 3
computation results En in table1, the increased En indicates
the change in quantity of information content for spectral 60
resolution through the merging. From table1 and Fig.2b, it is
50
obvious that En of the fused images have been changed when
compared to the original MS except the PCA. In Fig.2c and 40
SD
table1 the maximum correlation values was for PCA. In 30
Fig.2d and table1 the maximum results of SNR were with
20
the SF, and HFA.
Results of the SNR, NRMSE and DI appear changing 10
PCA
IHS
RVS
HFA
SF
HFM
the fused image, the SF and HFA methods gives the best
results with respect to the other methods. Means that this
method maintains most of information spectral content of the Fig. 2a: Chart Representation of SD
original MS data set which gets the same values presented
8
the lowest value of the NRMSE and DI as well as the high of
7
the CC and SNR. Hence, the SF and HFA fused images for
6
preservation of the spectral resolution original MS image
5
much better techniques than the other methods. En
4
Table 1: The Spectral Quality Metrics Results for the Original MS and Fused
3
Image Methods
2
Meth NRM 1
Band SD En SNR DI CC
od SE 0
51.01 5.209
HFM
IHS
PCA
RVS
HFA
SF
ORG
EF
R
8 3
51.47 5.226
ORG G
7 3 Fig. 2b: Chart Representation of En
51.98 5.232
B
3 6
55.18 6.019
R 6.531 0.095 0.138 0.896
4 6
55.79 6.041
EF G 6.139 0.096 0.151 0.896
2 5
56.30 6.042
B 5.81 0.097 0.165 0.898
8 3
1
0.98 CC
0.96
Table 2: The Spatial Quality Metrics Results for the Original MS and Fused
Image Methods
0.94
IHS
HFA
SF
EF
PCA
RVS
HFM
HFA G 12 52 -0.026 0.21
Fig.2c: Chart Representation of CC B 12 52 -0.028 0.211
R 12 54 0.001 0.205
10
9 HFM G 12 54 0.013 0.204
8
B 12 53 0.02 0.201
7
6 R 9 36 0.004 0.214
SNR
5 IHS G 9 36 0.009 0.216
4
3 B 9 36 0.005 0.217
2 R 6 33 -0.027 0.07
1
0
PCA G 6 34 -0.022 0.08
IHS
B 6 35 -0.021 0.092
HFA
SF
EF
PCA
RVS
HFM
R 13 54 -0.005 -0.058
RVS G 12 53 0.001 -0.054
Fig. 2d: Chart Representation of SNR
B 12 52 0.006 -0.05
0.3 R 11 48 -0.035 0.202
SF G 11 49 -0.026 0.204
0.25 NRMSE DI
B 11 49 -0.024 0.206
0.2
R 6 32 -0.005 0.681
0.15 MS G 6 32 -0.004 0.669
0.1 B 6 33 -0.004 0.657
PAN 1 10 42 ــــ ـــــ
0.05
0 30
IHS
HFA
SF
PCA
EF
RVS
HFM
25
MG
20
15
Fig. 2: Chart Representation of SD, En, CC, SNR, NRMSE & DI of Fused 5
Images 0
Edg HFA HFM IHS PCA RVS SF PAN
F
B. Spatial Quality Metrics Results:
Table 2 and Fig. 4 show the result of the fused images Fig. 3a: Chart Representation of MG
using various methods. It is clearly that the seven fusion 70
all methods except the PCA. from the table2 and Fig.3a the 10
SF
PCA
Edg F
RVS
HFM
PAN
FCC
the spatial quality. The maximum results of MG and SG for 0.15
sharpen image methods was for the EF as well as the results 0.1
of the MG and the SG for the HFA and SF methods have the 0.05
the PAN. Other means the SF added the details of the PAN -0.1
image to the MS image as well as the maximum preservation Fig. 3c: Chart Representation of FCC
of the spatial resolution of the PAN. Continue
0.03
HPDI
0.02
0.01
IHS
HFA
SF
Edg F
RVS
HFM
PCA
-0.01
-0.02
-0.03
-0.04
Fig..4c: HIS
Fig..4a: HFA
Fig.4d: PCA
Contenue Fig. 4
VI. CONCLUSION
VII. REFERENCES