LOCALIZED FILTERING FOR ARTIFACT REMOVAL IN COMPRESSED IMAGES

D ung T. V o

Digital Media Solutions Lab, SISA

(Samsung US R&D Center), Irvine, CA, USA
Truong Q. Nguyen
Video Processing Lab, ECE Department
UC San Diego, La Jolla, CA, USA
ABSTRACT
The paper proposes a novel method for coding artifact reduction
in compressed images. For removing blocking artifacts, a local-
ized DCT-based lter with condition on the similarity between
surrounding blocks is considered. To reduce ringing, a localized
fuzzy lter is utilized to avoid the blurry effect of linear lter and
painting-like effect of conventional fuzzy lter. To enhance chroma
components and reduce the color bleeding, the localized lter for
luma component are implemented for the chroma components. Sim-
ulations on a wide range of compressed images are performed to
verify the effectiveness of the algorithm.
Index Terms coding artifact reduction, fuzzy lter, localized
ltering.
1. INTRODUCTION
Visual quality of block-based compressed images is degraded by
blocking artifacts, ringing artifacts and color bleeding, especially
at low-bit-rate coding. Blocking artifacts occur at the border of
neighboring blocks when each block is processed independently
with coarse quantization of discrete cosine transform (DCT) coef-
cients. In natural images, high frequency components tend to have
smaller values than the low frequency components. But they are
quantized with higher quantization step size than the low frequency
components. This makes the high frequency components more
distorted or even rounded to zero value. Loss in high frequencies
will cause ringing artifacts, which affect the visual quality seriously
in detail areas. For chroma components, they are downsampled
and then compressed with even higher quantization matrix than the
luma components. Color becomes blocky and mismatchs to its luma
information. These lead to color bleeding.
Visual effects of coding artifacts vary fromone codecs to another
but these artifacts always have directional and data-dependent prop-
erties. Because of the block-based compression, blocking artifacts
in luma and chroma components occur at the horizontal and verti-
cal directions at pixels between 2 blocks. Furthermore, due to the
loss of high frequency components during the coarse quantization,
the ringing artifacts appear along the strong edges. The data depen-
dent property is from the usage of the same quantization step size
matrix for every blocks. Coding artifacts degrade the visual quality
more in detail areas than in at areas. These characteristics make the
coding artifacts different than the recording noise, where it is usu-
ally assumed as additive zero-mean white Gaussian noise [1]. Fig. 1
shows the differences between additive Gaussian noise and coding
artifacts in MJPEG compression. The noisy image in Fig. 1.(a) is
degraded by additive Gaussian noise with variance of 0.01 while the
compressed image in Fig. 1.(b) is encoded with scaling factor of 4

D ung T. V o performed this work while at UC San Diego

(a) Noisy image (b) Compressed image
Fig. 1. Comparison on visual effect between coding artifacts and
Gaussian noise.
for the quantization step size matrix. As shown in this example,
Gaussian noise is uniformly distributed over pixels while the coding
artifacts is non-uniformly distributed due to their special character-
istics. This implies that these coding artifacts should be treated in a
different way other than methods for denoising in pre-processing.
Many pixel-domain and DCT-domain post-processing approa-
ches have been considered for blocking artifact reduction . These in-
clude low-pass ltering [2] and adaptive median ltering [3] which
were applied to remove the high frequencies caused by sharp edges
between adjacent blocks. In H.264/AVC, an adaptive deblocking l-
ter [4] was proposed to selectively lter the artifacts in the coded
block boundaries. With assumption on the small changes of neigh-
boring DCT coefcients at the same frequency in a small region, a
xed low pass lter in [5] or adaptively weighted low pass lters in
[6] and [7] are applied to the transform coefcients of the shifted
blocks. Although effectively reducing the blocking artifacts, these
mentioned methods also blur the output images.
To reduce ringing artifacts, edges or edgy areas which contain
ringing are detected and then are processed with an adaptive lter
as in [8] or a gray-level morphological nonlinear smoothing lter
as in [9]. These lters remove the ringing artifacts, but at the same
time also reduce the details of the images. To avoid the blurry effect,
isotropic fuzzy lters [10] lters are used, but their output sometime
have painting-like effects with very strong edges above smooth ar-
eas.
This paper proposes a novel method to reduce the blocking, ring-
ing and color bleeding artifacts using localized ltering. For de-
blocking, a conditionally lowpass lter is applied for local shifted
blocks around the block of interest. To avoiding blurry effect, a novel
fuzzy lter is proposed for deringing. Filter coefcients of this l-
ter are weightedly locally to reduce the painting-like effect of fuzzy
lters. The paper is organized as follows. Section 2 introduces the
conditionally localized DCT-based lters for deblocking. Section 3
describes the localized fuzzy lter for deringing. Simulations and
1269 978-1-4577-0539-7/11/$26.00 2011 IEEE ICASSP 2011
T
c
m
i
'E
n
i
1
2
3
4
xq
xq,i
Fig. 2. Translation between blocks of image xs and x.
comparison results are presented in Section 4. Finally, Section 5
gives the concluding remarks.
2. CONDITIONALLY LOCALIZED DCT-BASED
DEBLOCKING FILTERS
Assume the image Y (m, n) is compressed to Yq(m, n) by N N
blockwise processing with a transform T followed by a quantization
matrix kQ where Q is the standard quantization matrix and k is the
scaling factor.

Y and

Yq are the transformed coefcients of Y and its
quantized value, respectively. Because of the block-size processing,
the correlation between the N N block of pixels are destroyed. To
increase this cross-block correlation, a DCT-based domain low pass
lter [6] is applied through the surrounding N N blocks Yq(m+
mi, n+ni) together with the block of interest Yq(m, n), as shown
in Fig.2. Assume that the DCT transform (type II) of block Yq(m+
mi, n+ni) is

Yq,m
i
,n
i

Yq,m
i
,n
i
(u, v) =
2
N
kukv
N1

m=0
N1

n=0
Yq(m+mi, n+ni)C
m
u
C
n
v
for m, n = 0, . . . , N1 (1)
and

Yq,0,0 =Xq where the shifted motion vector is considered equal
to 0. The localized DCT-based deblocking lter is applied to the
DCT coefcients at the same coefcient of these local DCT coef-
cients to form the enhanced DCT coefcients

Y

q
(u, v) =

(m
i
,n
i
)
DB
WDB(mi, ni)

Yq,m
i
,n
i
(u, v)
for u, v = 0, . . . , N1. (2)
where WDB is the lowpass lter coefcients for deblocking and DB
is the local window centered at location (m, n) which consists all
(mi, ni). To avoid the reconstructed output

Y

q
biasing from

Yq, the
lter coefcient must satisfy

(m
i
,n
i
)
DB
WDB(mi, ni) = 1 (mi, ni) DB (3)
Normally
WDB(0, 0) WDB(mi, ni). (4)
so that the DCT coefcients of the center blocks have highest con-
tribution to the output. The enhanced N N block Y

q
is recon-
structed by inverse DCT transform. For highly compressed images,
the high DCT frequencies tend to have zero value. As can be seen
in (2), these coefcients are recreated because of the non-zero val-
ues of

Yq,m
i
,n
i
(u, v). In [11], a condition of quality enhancement
was considered for the linear temporal lter over many frames. It
is shown that the difference between the block of interest and the
temporally surrounding blocks must be small enough so that these
blocks are rather related to each other to achieve the quality improve-
ment. Similarly in this paper for image enhancement, a condition of
the small difference between the block of interest and the spatially
surrounding blocks is implemented as follows
N1

m=0
N1

n=0

Yq(m+mi, n +ni) Yq(m, n)

Th (5)
where Th is a threshold. Only surrounding blocks which satisfy this
condition will be locally ltered using coefcients WDB(mi, ni).
3. LOCALIZED FUZZY DERINGING FILTERS
Although the localized deblocking lters as described in Section 2
can reduce the blocking artifacts and ringing artifacts, it will blur
the images due to its lowpass ltering. The deblocking lter should
be only applied to pixels near to block borders, not all pixels in the
blocks
YDB(m, n) =
_
Y

q
(m, n) if (m, n) near to block borders
Yq(m, n) otherwise.
(6)
To reduce the ringing artifact, this section will discuss a novel local-
ized fuzzy lters. Fuzzy lters are non-linear lters which are based
on the real-valued spatial-rank relation, as described in [10][12].
Assume that a normalized fuzzy lter hDR,n is applied to a set
DR of neighboring samples YDB(m+mi, n+ni) around the input
YDB(m, n) to form the output
YDR(m, n) =

(m
i
,n
i
)
DR
hDR,n(mi, ni)YDB(m+mi, n+ni) (7)
where hDR(mi, ni, n) is calculated by
hDR,n(mi, ni) =
hDR
_
YDB(m+mi, n+ni), YDB(m, n)
_

(m
i
,n
i
)
DB
hDR
_
YDB(m+mi, n+ni), YDB(m, n)
_
(8)
where hDR is the lter coefcient which controls the contribution of
the input YDB(m+mi, n+ni) to the output. Fuzzy lters are signal
dependent and help removing the ringing artifact while preserving
the details. They depend on the distance between the surrounding
pixel YDB(m+mi, n+ni) and the pixel of interest YDB(m, n). Fol-
low the spatial-rank relation, the lter coefcients hDR must fulll
the constraints
lim
|Y
DB
(m+m
i
,n+n
i
)Y
DB
(m,n)|0
hDR
_
YDB(m+mi, n+ni), YDB(m, n)
_
=1
(9)
lim
|Y
DB
(m+m
i
,n+n
i
)Y
DB
(m,n)|
hDR
_
YDB(m+mi, n+ni), YDB(m, n)
_
=0
(10)
and
h
_
YDB(m+mi,1, n+ni,1), YDB(m, n)
_
h
_
YDB(m+mi,2, n+ni,2), YDB(m, n)
_
if | YDB(m+mi,1,n+ni,1)YDB(m, n) |
| YDB(m+mi,2, n+ni,2)YDB(m, n) | . (11)
1270
c
c
c
c
c
c
c
c
c
c
c
c
c
c
a
a
a
c
c
c
c
c
c
c
a
a
a
c
c
c
a
a
a
s
s
s
c
c
c
c
s
s
s
c
c
c
c
s
s
s
c
c
c
c
c
c
c
c
(1,1)
(m
i
,n
i
)=(1,1)
(0,0)

'
'
'
'
'
'
'
'
'
(1,2)
(m
i
,n
i
)=(1,0)
Fig. 3. Localized coefcients of fuzzy lters.
hDR is referred to as the membership function and there are
many functions which fulll these requirements. An example is the
Gaussian membership function,
hDR
_
YDB(m+mi, n+ni), YDB(m, n)
_
=exp
_

_
Y
DB
(m+m
i
,n+n
i
)Y
DB
(m,n)
_
2
2
2
_
(12)
where represents the spread parameter of the input and controls
the strength of the fuzzy lter. Note that the contribution of the input
YDB(m, n) to the output is always highest compared to the contri-
bution of other samples
h
_
YDB[m, n], YDB[m, n]
_
=1
h
_
YDB[m+m

, n+n

], YDB[m, n]
_
(m

, n

). (13)
For the same | YDB[m+m

, n+n

]YDB[m, n] |, the higher the

value, the higher the contribution of YDB[m+m

, n+n

] relatively
compared to the contribution of YDB[m, n] to the output. This im-
plies that YDB[m, n] will be more averaged to YDB[m+m

, n+n

].
Smaller values will keep the signal YDB[m, n] more isolated from
its neighboring samples.
An example of calculating the coefcients of the localized fuzzy
lters is shown in Fig.3. The top right area visualizes the way cal-
culating fuzzy coefcient at pixel (m, n) = (1, 1). To estimate
the spatial-rank relation between pixel of interest (m, n) = (1, 1)
and its surrounding pixel at (2, 2), the conventional fuzzy lter
is based only on the value of these two pixels using (12). If these
two pixels represent an edge with similar values, its small differ-
ence leads to coefcients having values closed to unity (as observed
from (9)). They would dominate the coefcients in other directions
which are different than the edge direction. This will cause the
painting-like effect. To reduce this effect, the spatial-rank relation
is proposed to be a weighted combination of the spatial-rank rela-
tion of surrounding pairs of pixels where their position difference are
(mi, ni) = (1, 1). These relations are shown as the 45
o
arrows in
the top right of Fig.3. The pairs of pixels are determined by the shift
positions (mii, nii) V , where V is the 3 3 local window cen-
tered by pixel at (m, n) = (1, 1) in this example. Another exam-
ple with (m, n) = (1, 2) and direction with (mi, ni) = (1, 0)
is shown in the bottom left area of Fig.3.
hDR in this case is not calculated using (12) but is a lowpass
averaging of the lter coefcient inside the window V which is cen-
tered by the pixel of interest at (m, n)
hDR
_
YDB(m+mi, n+ni), YDB(m, n)
_
(14)
=

(m
ii
,n
ii
)V
_
WDB(mii, nii)
hDR,m
ii
,n
ii
_
YDB(m+mi +mii, n+ni +nii), YDB(m+mii, n+nii)
_
_
E
U
V
DCT
E
Deblocking
E
IDCT
E
2 T
E
Deringing
E
U
DR
V
DR
Y
E
DCT
E
Deblocking
E
IDCT
E
Deringing
c
E
Y
DR
Fig. 4. Block diagram of the proposed localized ltering scheme.
where WDR(mii, nii) is the weighting factor of each hDR,m
ii
,n
ii
to
the lter coefcient hDR. hDR,m
ii
,n
ii
is calculated by
hDR,m
ii
,n
ii
_
YDB(m+mi +mii, n+ni +nii), YDB(m+mii, n+nii)
_
=exp
_

_
YDB(m+mi +mii,n+ni +nii)YDB(m+mii,n+nii)
_
2
2
_
.
(15)
For chroma components, the localized algorithm cannot base on
U and V because they dont have enough information. In this case,
the algorithm is extended for chroma components by using the same
localized fuzzy lter of luma component. This helps match the color
to the structure in the luma component and reduce the color bleed-
ing. The whole algorithm is shown in Fig.4. At rst, all components
are transformed to DCT domain and are deblocked using the local-
ized linear lter. They are then converted back the pixel domain by
IDCT transform. Next, the deblocked Y component is implemented
to form the coefcients at the localized fuzzy lter, which is later
used for deringing. For U and V components, they are upsampled
to the same size of Y component, then are deringed by the localized
fuzzy lter from luma.
4. SIMULATION RESULTS
Simulations are performed to verify the effectiveness of the local-
ized lters in artifact reduction. The simulated database includes
different types of images from video sequences: Foreman, News,
Mother, Silent, Soccer, Bus. They are compressed using JPEG com-
pression with different scaling factors k of the quantization matrix.
For both deblocking and deringing localized lters, the weighting
matrices WDB and WDR are [1, 1, 1; 1, 3, 1; 1, 1, 1] /11. The local
windows DB, DR and V are all set as 3 3 pixels. The thresh-
old Th in the deblocking part is set to 250k
2
. Only 2 pixels at each
side of the block borders are replaced by the deblocked pixels to
avoid the blurry effect which is mentioned in Section 3. The spread
parameter of the fuzzy lter is also proportional to the scaling fac-
tor as 5k. For comparison, other methods of artifact reduction such
as Chens method [6], Lius method [7] and Kongs method [10] are
also simulated. Fig.5 shows the zoomed-in result for the 6
th
frame of
News sequence which is compressed with scaling factor of 3. Com-
paring to the compressed image in Fig.4(b), the enhanced images
using Chens method (Fig.4(c)), Lius method (Fig.4(d)) and Kongs
method (Fig.4(e)) can reduce most of blocking artifact and ringing
artifact. However they also introduce other artifacts such as blurri-
ness (Chens and Lius methods) or painting-like effect with dom-
inant edges and very few details (Kongs method). The enhanced
images using Chens method and Lius method are rather blurry and
still have some ringing artifacts at strong edge areas. The enhanced
image using Kongs method is sharper with less ringing artifacts than
the enhanced image using Chens method or Lius method. But it has
the painting-like effect with cleared out details, as can be seen at the
face of the spokeswoman. The enhanced image using the proposed
localized lter in Fig.4(f) achieves the best visual quality comparing
1271
(a) Original (b) Compressed (28.45dB)
(c) Chens method (28.32dB) (d) Lius method (28.46dB)
(e) Kongs method (28.90dB) (f) Proposed method (29.25 dB)
Fig. 5. Comparison for 6
th
frame of News sequence.
Table 1. Comparison of PSNR in units of dB for different methods.
Images 3Q Chen Liu Kong Localized Filters
News 28.45 28.32 28.46 28.90 29.15
Silent 28.85 29.40 29.32 29.67 29.76
Foreman 29.09 29.45 29.44 30.09 30.15
Soccer 27.35 27.54 27.50 27.58 27.92
Mother 32.36 32.95 32.81 33.16 33.39
Bus 26.29 25.80 25.93 26.52 26.64
Average gain 3Q +0.1809 +0.1794 +0.5873 +0.7694
4Q 0.0553 +0.0008 +0.4074 +0.5456
2Q +0.0407 0.0391 +0.3319 +0.4955
to other methods. It both effectively removes blocking and ring-
ing artifacts and still keeps the details and edge sharpness. A close
inspection at the dress of the dancing woman concludes that the pro-
posed method is more effective in color bleeding reduction, com-
paring to other methods. The PSNR of enhanced image using the
proposed method also achieves highest value (29.25 dB), compar-
ing to the compressed image (28.45dB), Chens method (28.32dB),
Lius method (28.46dB) and Kongs method (28.90dB).
The comparison in term of PSNR values is shown Table.1.
Over many types of images which are compressed with scal-
ing factor of 3, the proposed localized lters provide the highest
PSNR improvement of +0.7694 dB comparing to Chens method
(+0.1809 dB), Lius method (+0.1794 dB) and Kongs method
(+0.5873 dB). It also outperforms other methods in different scal-
ing factors, as also shown in Table.1. The localized fuzzy lter
can be also applied to JPEG2000 compressed images or H.264
sequences to reduce the ringing artifacts.
5. CONCLUSIONS
The paper proposes a novel method for coding artifact reduction
using a conditionally localized DCT-based lter for deblocking and
a localized fuzzy lter for deringing. These lters effectively remove
the coding artifacts while avoiding the blurry effect from linear l-
tering and painting-like effect from non-linear fuzzy lters. Future
works will focus on adaptive localized lters and advanced methods
for color bleeding reduction.
6. REFERENCES
[1] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Im-
age Denoising by Sparse 3-D Transform-Domain Collabora-
tive Filtering, IEEE Trans. Image Process., vol. 16, pp. 2080
2095, August 2007.
[2] T. Jarske, P. Haavisto, and I. Defee, Post-ltering methods for
reducing blocking effects from coded images, IEEE Trans.
Cosumer Electronics, vol. 40, pp. 521526, August. 1994.
[3] Y.F. Hsu and Y.C. Chen, A New Adaptive Separate Median
Filter for Removing Blocking Effects, IEEE Trans. Cosumer
Electronics, vol. 39, pp. 510513, August 1993.
[4] P. List, A. Joch, J. Lainema, G. Bjontegaard, and M. Kar-
czewicz, Adaptive Deblocking Filter, IEEE Trans. Circuits
Syst. Video Technol., vol. 13, pp. 614?619, July 2003.
[5] A. Nosratinia, Embedded post-processing for enhancement
of compressed images, Proc. IEEE Data Compression Conf.,
pp. 6271, 1999.
[6] T. Chen, H.R. Wu, and B. Qiu, Adaptive postltering of trans-
form coefcients for the reduction of blocking artifacts, IEEE
Trans. Circuits Syst. Video Technol., vol. 11, pp. 594602, May.
2001.
[7] S. Liu and A.C. Bovik, Efcient DCT-domain blind measure-
ment and reduction of blocking artifacts, IEEE Trans. Circuits
Syst. Video Technol., vol. 12, pp. 11391149, December. 2002.
[8] J. Hu, N. Sinaceur, F. Li, K.W. Tam, and Z. Fan, Removal of
Blocking and Ringing Artifacts in Transform Coded Images,
Proc. IEEE Int. Conf. Acoustics, Speech and Signal Process-
ing, vol. 4, pp. 25652568, April 1997.
[9] S.H. Oguz, Y.H. Hu, and T.Q. Nguyen, Image coding ringing
artifact reduction using morphological post-ltering, Proc.
IEEE Int. Work. Multi. Signal Process., pp. 628633, 1998.
[10] H.S. Kong, Y. Nie, A. Vetro, H. Sun, and K. Barner, Adap-
tive fuzzy post-ltering for highly compressed video, Prof. of
IEEE Int. Conf. Image Proc., pp. 18021806, 2004.
[11] D.T. Vo and T.Q.Nguyen, Quality Enhancement for Motion
JPEG using Temporal Redundacies, IEEE Transactions on
Circuits and Systems for Video Technology., vol. 18, pp. 609
619, May 2008.
[12] D.T. Vo, T.Q.Nguyen, S. Yea, and A. Vetro, Adaptive Fuzzy
Filtering for Artifact Reduction in Compressed Images and
Videos, IEEE Transactions on Image Processing., vol. 18,
pp. 11661178, May 2009.
1272