Professional Documents
Culture Documents
Shaode YUa , Wentao ZHANGa,b , Shibin WUa , Xiaolong LIa,b , Yaoqin XIEa∗
a
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China
b
College of Electrical and Information Engineering, Hunan University, China
{sd.yu, wt.zhang, sb.wu, xl.li, yq.xie}@siat.ac.cn
1. INTRODUCTION Two edge maps are primary for EPR calculation. One is from
the reference image and the other is from its distorted image.
Image processing is indispensable in many visual tasks, such Edge maps preserve main structures and low-level features.
as object segmentation [1, 2], feature enhancement [3] and These structures and features are useful in object segmenta-
image registration [4, 5, 6]. After processing, image quality is tion and image registration. Canny detector [11] is adapted
inevitably changed. Image quality assessment (IQA) metrics for edge map extraction because of its good detection, good
are thus proposed to evaluate the performance of these im- localization and one-pixel edge width [12]. Edge maps from
age processing methods and guide them toward better visual the reference (EMr) and the distorted image (EMd) are used
quality. Based on the accessibility of “gold standard”, IQA in Equation (1) and (2) to evaluate the EPR from accuracy
metrics can be divided into full-, reduced- and no-reference and robustness, respectively. The num denotes to count the
categories [7]. number of points with true value in edge maps.
To full-reference metrics, IQA metrics have been evolved ∩
from pixel-level analysis to structure- and feature-level anal- num(EM r EM d)
EP Ra = (1)
ysis [7, 8, 9, 10]. Subjective evaluation may be the only num(EM r)
“correct” way to assess image quality, but it is troublesome ∩
num(EM r EM d)
in practice because of inconvenience, operator-dependance, EP Rr = (2)
huge time consumptions and biased quantification. Error- num(EM d)
sensitivity approaches, such as mean squared error (MSE), If structures and low-level textures are well maintained,
are widely used for their simplicity and concise physical EPRa value will be high; if false edges are introduced, EPRr
meanings. However this kind of approaches do not always value decreases. In other words, higher EPRa shows better
match well with visual quality. Since human beings are high- edge preservation and higher EPRr indicates less false struc-
ly adapted to extract structural messages from visual scenes, tures. A color map∩ is synthesized with EMr, EMd and the in-
Structural Similarity (SSIM) is designed by averaging local tersection (EMr EMd). In the color map, preserved edges
scores which are quantified from signal correlation coeffi- are in white, primary structures in the reference image are in
cients, relative distortions and mean luminance [8]. Feature red if they are not lost, and false edges are in green. The EPR
4. RESULTS
Fig. 1. The EPR map. (D) is from (A) and (E) is from (B) with
a same threshold from edge detection. (C) shows absolute 4.1. Image magnification
difference of (A, B), and (F) is the synthesized color map. Unbiased IQA scores are shown in Table 1. Best scores are
in bold. It is observed that DCCI outperforms the other five
methods with higher IQA scores in all five metrics. SSIM
3. EXPERIMENTS and FSIM show no much difference among these six meth-
ods. In other words, SSIM and FSIM fail to distinguish EDI
This section presents reference images and experimental de- methods from traditional methods. In contrast, the proposed
signs in image magnification and noise reduction. Three other EPR metrics highlight these EDI methods with much better
metrics, MSE, SSIM [8] and FSIM [9], are also used. On one edge preservation. Compared to Bi-linear and Bi-cubic, the
hand, the feasibility of EPR is validated in image process- average EPRa of EDI methods is improved up to 60%, and
ing. On the other hand, the performance of image processing the average EPRr is increased more than 72% [10]. In fact,
methods are compared with respect to these four IQA metrics. EDI methods are able to maintain more than 66% edge struc-
tures in reference images (EPRa) and mislead less than 33%
structures (EPRr).
3.1. Reference images
Two “gold standard” groups, STILL and KODAK, are used MSE SSIM FSIM EPRa EPRr
as the reference [10]. Each group contains 20 images. Image Bi-linear 24.61 0.93 0.95 0.46 0.49
resolution in STILL is 512×512, and in KODAK is 768×512. Bi-cubic 24.44 0.92 0.95 0.49 0.48
All algorithms are implemented in MATLAB. NEDI 17.72 0.94 0.97 0.66 0.69
DFDF 16.61 0.94 0.97 0.67 0.68
3.2. Image magnification ICBI 16.77 0.94 0.97 0.70 0.67
DCCI 16.40 0.95 0.97 0.69 0.69
Image magnification aims to generate high-resolution im-
ages for better visualization, because high resolution direct- Table 1. Average IQA scores in image magnification
ly improves image readability and scene analysis. Traditional
methods, such as Bi-linear and Bi-cubic, overcome blocking Perceptive analysis of an instance in image magnification
with image smoothing. But these methods also soften sharp is illustrated with respect to a region of interest in Figure 2.
edges and blur minor structures. As edges account for struc- Image (A) is downsampled to image (B) and definitely loses
tural messages and low-level features, edge-directed interpo- some small or short edges for its low resolution. Sometimes,
lation (EDI) methods become a research center. Four EDI insufficient pixels leads to mistakes and misunderstanding,
methods, NEDI [13], DFDF [14], ICBI [15] and DCCI [16], such as texture orientations directed by red and green arrows.
are involved in this paper. Image magnification aims for regenerating these lost minor
699
messages. From visual perception, Bi-linear (C) and Bi-cubic pixel intensities. SSIM and FSIM validate that BM3D is the
(D) cause blurring, (E, G) result in weird textures directed best to retain image structures and low-level features. While
by green arrow, and (F, H) show clear image structures. It EPRa presents that Wiener outperforms other methods with
must be admitted that all magnification methods fail to recov- right preserved edges, and EPRr shows NLM is with the su-
er right textures compared to the reference (A), because the perior robustness in image regeneration.
source information in image (B) is with wrong texture orien-
tation (directed by arrows), and subsequently causes mislead-
MSE SSIM FSIM EPRa EPRr
ing propagation in image magnification.
Wiener 51.17 0.81 0.92 0.61 0.46
Bilateral 37.56 0.84 0.92 0.47 0.66
BaysShrink 47.92 0.84 0.93 0.57 0.50
NLM 52.03 0.85 0.91 0.39 0.70
BM3D 51.24 0.87 0.93 0.45 0.68
MTV 56.56 0.80 0.88 0.29 0.62
700
ful and interesting in structural hierarchy interpretation and
multi-scale analysis with respect to image structures.
As a full-reference metric, EPR quantifies assessment of
image quality from accuracy (EPRa) and robustness (EPRr).
In practice, EPRa is more important and preferable than EPRr
in IQA analysis, since accurate edge preservation is critical
for follow-up visual tasks, such as object segmentation [1, 2],
feature extraction [3] and image registration [4, 5, 6]. On the
other hand, higher EPRr values validate less introduced struc-
tures (in Figure 5). In addition, EPR synthesizes a color map
to demonstrate image structure changes. This map visualizes
edge maintenance and alias introduction with different colors.
Fig. 5. EPR maps of noise reduction. Image (A) shows con- Edge preservation is decisive in selecting precise algorithm-
cise structures, and is destroyed by noise (B). (C, E) keep the s for specific purposes in visual tasks. Two issues should be
general structures. (D, F-H) reduce noise but fail to regenerate concerned for further EPR improvement. One is about how to
fine structures (red edges). precisely detect edges and determine their orientation. Even
as a standard edge detector, Canny edge detector [11] is not
perfect [23, 24, 25]. Meanwhile, edge orientation is still hard
5. DISCUSSION AND CONCLUSION
to determine, and existing methods are quite rough [12, 26]
Image processing is a fundamental visual task in computer or too complicate [27]. Then the subsequent issue is how to
vision and pattern analysis. Consequently, IQA is useful and properly merge information of edge orientation into the EPR
necessary. It identifies proper algorithms for specific appli- calculation.
cations, or optimizes precise parameters for better results, or In conclusion, accurate objective IQA is still a challenging
automatically monitors visual systems for high visual quali- task. This paper validates the feasibility of EPR metrics and
ty. Among these four involved metrics, MSE evaluates image EPR map in image processing via image magnification and
quality from error sensitivity based model [7]; SSIM [8] and noise reduction. It suggests that EPR is useful in evaluating
FSIM [9] measure image similarity from structures and low- image structure preservation and provides a new way in IQA.
level features, respectively; and EPR [10] assesses the essence
for human visual systems from image structures and provides 6. ACKNOWLEDGEMENT
a synthesized color map for visual analysis.
The feasibility of EPR metrics in image processing is val- This study was financed partially by the National 863 Pro-
idated via image magnification and noise reduction in this gram of China (Grant No. 2012AA02A604), the Next gen-
paper. In image magnification, EPR metrics verify that E- eration communication technology Major project of National
DI methods (NEDI [13], DFDF [14], ICBI [15] and DCCI S&T (Grant No. 2013ZX03005013), the Key Research Pro-
[16]) are much better than traditional methods (Bi-linear and gram of the Chinese Academy of Sciences, and the Guang-
Bi-cubic) for their better edge preservation with less blurring dong Innovation Research Team Funds for Image-Guided
[10]. Compared to reference images, low-resolution images Therapy and Low-cost Healthcare.
definitely lose small textures, or even worse, they may misin-
terpret image contents, such as texture orientations (directed
7. REFERENCES
by arrows in Figure 2). That is because low-resolution im-
ages are with insufficient pixels to depict fine structure. Im- [1] W. Zhou and Y. Xie, “Interactive medical image seg-
age magnification methods are able to enhance image quality, mentation using snake and multiscale curve editing,”
improve image readability with enriched textures in high res- Computational and Mathematical Methods in Medicine,
olution, and regenerate part of lost image information. But, it vol. 2013, 2013.
is still challenging to rectify these wrong messages nor to gen-
erate all lost structures during image magnification. In noise [2] Q. Luo, W. Qin, T. Wen, J. Gu, N. Gaio, S. Chen, L. Li,
reduction, we find that good image contrast may be derived and Y. Xie, “Segmentation of abdomen mr images us-
from concise and distinct image structures. Noise imposes ing kernel graph cuts with shape priors,” Biomedical
many trivial textures and degrades image contrast. Wiener Engineering Online, vol. 12, no. 1, pp. 124, 2013.
[17] and BaysShrink [19] maintain image structures well but
fail to remove the general noise, while other four methods [3] S. Wu, S. Yu, Y. Yang, and Y. Xie, “Feature and contrast
(Bilateral [18], NLM [20], BM3D [21] and MTV [22]) have enhancement of mammographic image based on mul-
exaggerated image contrast at the cost of discarding these fine tiscale analysis and morphology,” Computational and
structures (red edges in Figure 5). This phenomenon is use- Mathematical Methods in Medicine, vol. 2013, 2013.
701
[4] Y. Xie, M. Chao, P. Lee, and L. Xing, “Feature-based [17] J.S. Lim, “Two-dimensional signal and image process-
rectal contour propagation from planning ct to cone ing,” Englewood Cliffs, NJ, Prentice Hall, vol. 1, 1990.
beam ct,” Medical Physics, vol. 35, no. 10, pp. 4450–
4459, 2008. [18] C. Tomasi and R. Manduchi, “Bilateral filtering for gray
and color images,” in IEEE ICCV, 1998, pp. 839–846.
[5] Y. Xie, M. Chao, and G. Xiong, “Deformable image
registration of liver with consideration of lung sliding [19] S.G. Chang, B. Yu, and M. Vetterli, “Adaptive wavelet
motion,” Medical Physics, vol. 38, no. 10, pp. 5351– thresholding for image denoising and compression,”
5361, 2011. IEEE Transactions on Image Processing, vol. 9, no. 9,
pp. 1532–1546, 2000.
[6] R. Zhang, W. Zhou, Y. Li, S. Yu, and Y. Xie, “Nonrigid
registration of lung ct images based on tissue features,” [20] A. Buades, B. Coll, and J.M. Morel, “A non-local al-
Computational and Mathematical Methods in Medicine, gorithm for image denoising,” in IEEE CVPR, 2005,
vol. 2013, 2013. vol. 2, pp. 60–65.
[7] Z. Wang, A.C. Bovik, and L. Lu, “Why is image quality [21] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Im-
assessment so difficult?,” in IEEE ICASSP, 2002, vol. 4, age denoising by sparse 3-d transform-domain collabo-
pp. 3313–3316. rative filtering,” IEEE Transactions on Image Process-
ing, vol. 16, no. 8, pp. 2080–2095, 2007.
[8] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncel-
li, “Image quality assessment: From error visibility to [22] Y. Wang, W. Chen, S. Zhou, T. Yu, and Y. Zhang, “Mtv:
structural similarity,” IEEE Transactions on Image Pro- modified total variation model for image noise removal,”
cessing, vol. 13, no. 4, pp. 600–612, 2004. IET Electronics Letters, vol. 47, no. 10, pp. 592–594,
2011.
[9] L. Zhang, L. Zhang, X. Mou, and D. Zhang, “Fsim: a
feature similarity index for image quality assessment,” [23] K. Bowyer, C. Kranenburg, and S. Dougherty, “Edge
IEEE Transactions on Image Processing, vol. 20, no. 8, detector evaluation using empirical roc curves,” in IEEE
pp. 2378–2386, 2011. CVPR., 1999, vol. 1.
[10] S. Yu, R. Li, R. Zhang, M. An, S. Wu, and Y. Xie, [24] L. Ding and A. Goshtasby, “On the canny edge detec-
“Performance evaluation of edge-directed interpolation tor,” Pattern Recognition, vol. 34, no. 3, pp. 721–725,
methods for noise-free images,” in ACM ICIMCS, 2013, 2001.
pp. 268–272.
[25] P. Bao, D. Zhang, and X. Wu, “Canny edge detection
[11] J. Canny, “A computational approach to edge detection,” enhancement by scale multiplication,” IEEE Transac-
IEEE Transactions on Pattern Analysis and Machine In- tions on Pattern Analysis and Machine Intelligence, vol.
telligence, vol. 1986, no. 6, pp. 679–698, 1986. 27, no. 9, pp. 1485–1490, 2005.
[12] S. Yu, R. Zhang, S. Wu, J. Hu, and Y. Xie, “An edge- [26] F. Porikli, “Accurate detection of edge orientation for
directed interpolation method for fetal spine mr images,” color and multi-spectral imagery,” in IEEE ICIP, 2001,
Biomedical Engineering Online, vol. 12, no. 1, pp. 102, vol. 1, pp. 886–889.
2013.
[27] W.T. Freeman and E.H. Adelson, “The design and use of
[13] X. Li and M.T. Orchard, “New edge-directed interpola- steerable filters,” IEEE Transactions on Pattern analysis
tion,” IEEE Transactions on Image Processing, vol. 10, and machine intelligence, vol. 13, no. 9, pp. 891–906,
no. 10, pp. 1521–1527, 2001. 1991.
[14] L. Zhang and X. Wu, “An edge-guided image interpola-
tion algorithm via directional filtering and data fusion,”
IEEE Transactions on Image Processing, vol. 15, no. 8,
pp. 2226–2238, 2006.
[15] A. Giachetti and N. Asuni, “Real-time artifact-free im-
age upscaling,” IEEE Transactions on Image Process-
ing, vol. 20, no. 10, pp. 2760–2768, 2011.
[16] D. Zhou, X. Shen, and W. Dong, “Image zooming using
directional cubic convolution interpolation,” IET Image
Processing, vol. 6, no. 6, pp. 627–634, 2012.
702