High-Resolution Image Restoration from Low-Resolution Images
-Preprocessing for Pattern Recognition-
Nobuhide Okabayashia, Hideya Takahashia,
Tahito Aidaa
a
Department of Electrical Engineering,
Graduate School of Engineering,
Osaka City University, Japan
nobuhide@ec.elec.eng.osaka-cu.ac.jp
Abstract
In this paper, we study the problem of restoring a
high-resolution image from several blurred low-
resolution images. The images consist of decimated,
blurred and noisy versions of the high–resolution
image. The high-resolution image is modeled as a
maximum a posteriori. We show a projection onto
convex sets method with criterion. Computer
simulations are given to confirm the effectiveness of
the proposed method. The character recognition was
experimentally demonstrated using real images with
low-resolution. Using the restored high-resolution
image, a high recognition rate (100) was got as a
result.
1. Introduction
Reconstructing a high-resolution image refers to
methods that increase spatial resolution by fusing
information from a sequence of images, acquired in
one or more of several possible ways. High-resolution
is used in the meaning of the algorithm that produces
an increase in resolution from multiple low-resolution
degraded images [1][2].
Due to hardware cost, size, and fabrication
complexity limitations, imaging systems like CCD
(charge coupled device) detector arrays often provide
only multiple low-resolution degraded images.
However, a high-resolution image is indispensable in
applications including health diagnosis and monitoring,
military surveillance, and terrain mapping by remote
sensing. Other intriguing possibilities include
substituting expensive high-resolution instruments like
scanning electron microscopes by their cheaper
counterparts and then applying technical methods for
0-7695-2882-1/07 $25.00 ©2007 IEEE
Hiromitsu Hamab
b
Department of Information and
Communication Engineering,
Graduate School of Engineering,
Osaka City University, Japan
increasing the resolution to that derivable with much
more costly equipment [3][4][5].
In this paper, we propose a basic algorithm that
reconstructs a high-resolution image from multiple
low-resolution images. Additionally, in order to
evaluate the proposed method of high-resolution image
restoration, the proposed method was applied to the
character recognition.
The outline of this paper is as follows. In Section 2,
we propose the method that restores the high-
resolution from low-resolution images. In Section 3,
the experimental result of the character recognition
using the high-resolution image is shown. Finally,
concluding remarks are given in Section 4.
2. High-resolution image from multiple
low- resolution images
We propose the technique for restoring high-
resolution image using multiple low-resolution images
that are the deterioration one of the original one. The
low-resolution images are obtained from an original
image according to Eq. (1),
­°M1 M2 ½
Li, j d1,d2 ®¦¦HM1(i1)d1 k,M2 ( j1)d2 l ¾/ M1 uM2 , (1)
°̄k 1 l 1 ¿
i 1,2,, N1 / M1 , j 1,2,, N2 / M2 ,
where M1and M2 mean resolution ratios of horizontal
and vertical directions, respectively [3],[6]-[10]. The
resolution of the original image is N1×N2. Hi,j and
Li,j(d1,d2) are pixel values of the original and low-
resolution images, respectively. Figure 1 illustrates the
relationship between original image and low-resolution
images. For the sake of simply, one-dimensional

Figure 1.Example of low-resolution images.
notations are used in Figure 1, but generalization to
two dimensions is straightforward.
As for multiple low-resolution images, it is easily
understood that information is lost compared with the
original image. Then, some criterion is necessary to
restore the original image. In some cases, it can be
considered as a criterion F to minimize the difference
between adjacent pixel values,
A1 B 2 A B1
' ¦¦ X
i 1 j 1
i1, j  Xi, j ¦¦ Xi, j1  Xi, j ,
i 1 j 1
F min '.
X1, X2 ,, XZ
Figures 2 and 3 show the result of restoring the
original image by the proposed method. In Figure 2,
the resolution of original image is 64×64. And, low-
resolution images are 16×16. Periodic noise is
observed in the restored image in Figure 2(c). This is
(a) (b)
(c) (d)
Figure 2. Restoration example 1;
M1=4,M2=4,N1=64,N2=64.
(a) Original image,
(b) One of low-resolution images,
(c) Restored image with quantization noise,
(d) Restored image after removing QE.
due to quantization error (QE) when the low-resolution
images are made.
To remove the QE, the original and restored images
are compared in the frequency domain. For example,
Fourier power spectrums of the same row of Figure
2(a) and Figure 2(c), are shown in Figure 4. It can be
seen that there are some extra spectrums in the restored
image in Figure 4. The inverse-Fourier transform after
removing these spectrums can remove the QE. Figure
2(d) and Figure 3(c) show the restored images by the
proposed technique. The image quality is fairly good.
We described that the low-resolution images were
made from the original image, and proposed the
technique for obtaining high-resolution image from
multiple low-resolution images. Moreover, the
experimental restoration is demonstrated using low-
resolution images generated from the original image,
and the effectiveness of the technique was confirmed.
2
(2)
(a) (b)
(c)
Figure 3. Restoration example 2;
M1=4,M2=4,N1=128,N2=128
(a) Original image,
(b) One of low-resolution image,
(c) Restored image after removing QE.
Figure 4. Fourier power spectrum.
3. Character recognition
The character recognition is one of applications
using the restoration method. It is not likely to be able
to read by the character's collapsing in the low-
resolution image. So it is necessary to restore the high-
resolution image.
An experiment on the character recognition was
carried out, and the accuracy was evaluated. The low-
resolution image was got by using pictures of
characters taken by a Web camera. In this experiment,
the camera was fixed, and the paper on which
characters are written was moved to horizontal by 1/4
pixel. Figure 5(a) shows four images taken like this.
The photographic images were taken at the shifted
positions from X1 by 0, 1/4, 2/4, and 3/4 pixels. These
four images were rearranged, as shown in Figure 5(b):
of Eq (2), X1 1,j, X2 1,j, X3 1,j, X41,j, X12,j …X4M1,j.
(a)
(b)
(c)
(d)
(e)
Figure 5. Images used in the experiment: (a)
Input images with low-resolution, (b)
Rearranged image of X1 Ξ X4, (c) Restored
image, (d) Restored image using ray-space
data, (e) Reference image as templates.
Figure 5(b) is a high-resolution image by burying the
pixels of low-resolution images. Figure 5(c) is the
median of pixels taken vertically and horizontally in
the image of Figure 5(b).
Figure 5(d) shows the restored image by another
method. This method uses ray-space data [11]. Since
the pixels corresponding to the same point on the
target paper of images X1, X2, X3, and X4 are aligned
in the ray-space, we can correct the displacement of
camera positions. Figure 6(a) shows the ray-space data
made from four input images. Figure 6(b) shows the
ray-space data corrected from Figure 6(a). Figure 5(d)
was rearranged from Figure 6(b).
Next, we took a picture of the similar image by the
high-resolution camera to get a reference image of
each character as a template for character recognition.
(Figure 5(e)). In Figure 5(a), (b), (c), and (d), the
region of each character was segmented using
histograms and it was recognized by template
matching.
In this experiment, the image was made gray-scale
to recognize easily. As a template matching method,
‘Rate1’ and ‘Rate2’ were used. The former shows the
possibility that a white area of the template consists
one of an input image, and the latter shows the
possibility that a black area of the template consists
one of segmented character. When a template and a
segmented character match each other, both ‘Rate1’
and ‘Rate2’ may become the maximum values. When
matching template and segmented character each other
and either ‘Rate1’ or ‘Rate2’ is more than 0.7, the
possibility is defined as the sum of ‘Rate1’ and ‘Rate2’.
The template at the highest possibility is selected as a
correct character.
Table 1 shows the experimental result. The yellow
color as indicate the recognition failed. ‘Ra’ means
success rate of character recognition. ‘Ave’ means the
averages of the difference between a possibility of the
recognized correct character and the maximum
possibility except for the correct character. The higher
the value of ‘Ave’ is, the more certain the recognition
result become. From Table 1, misidentification
disappeared by making it to a high-resolution though
there was misidentification in the photographic image.
(a)
(b)
Figure 6. Restoration using ray-space data: (a)
Ray-space data, (b) Corrected data.
 Table 1. Result of character recognition.
Correct character
Input % Ave
B R E A K F A S T
X1 E K E A F F A S T 67 -0.113
X2 B K E A T F A E T 67 -0.271
X3 B K E A T F A E T 67 -0.271
X4 K R E A K F A S T 89 0.081
Y B R E A K F A S T 100 0.109
Z B R E A K F A S T 100 0.097
R B R E A K F A S T 100 0.114
Moreover, restored image using ray-space data does
the character recognition with stability. It was able to
be confirmed to obtain the high-resolution image by
the above-mentioned result.
4. Conclusions
We have proposed the restoration method of the
high-resolution image from multiple low-resolution
images. Some experiments were carried out to confirm
the effectiveness of the proposed method. The low-
resolution images were taken by a Web camera, and
the results were fairly good. The new method for
removing QE in the frequency domain was proposed.
As one of applications, the restored high-resolution
image was used as the pre-processed image for the
character recognition. We experimented the pattern
matching between the character segmented from the
restored high-resolution image and the template made
of the image taken directly by a high-resolution camera.
The restored high-resolution image matched rather
than any other low-resolution images taken. The
experiment in the character recognition showed perfect
result that is 100% recognition rate. But we had better
use much more images get, and improve the restoration
method more robust.
The high-resolution image could be restored by the
proposed method. If the accuracy of it will be
improved in near future, some problems for practical
use will be solved and the technique will be put to
practical use. For example, the image like digital
camera with millions pixels may be got from images of
camera of mobile phone with a half million pixels.
And when a high-resolution image is needed, the
camera size can be minimized, as vision system for
micro robot. That may lead to decrease in cost or in
size.
5. References
[1] Sung C.P., Min K.P. et al., ϘSuper-Resolution Image
Reconstruction: A Technical Overview, ϙ IEEE Signal Proc.
Magazine, Vol. 26, No. 3, pp.21-36Δ2003.
[2] Joohyum Kim, Wonwoo Jang, et, ”A New Image-scaling
Algorithm Eradicating Blurring and Ringing to Apply to
Camera Phones”, IEEE 5.4-1 2007.
[3] Michael K.NG and Andy C.YAU, “Super-Resolution
Image Restoration form Blurred Low-Resolution Images”,
Journal of Mathematical Imaging and Vision 23: 367-378,
2005.
[4] J.Tanida, T.Kumagai, K.Ymada, S Miyatake, K.Ishida,
T.Morimoto, N.Kondou, D.Miyazaki, and Y.Ichioka, “Thin
observation module by bound optics (TOMBO): concept and
experimental verification”, APPLIED OPTICS, Vol.40,
No.11, 10 April 2001.
[5] Tomomasa Gotoh and Masatoshi Okutomi, “Direct
Super-Resolution and Registration Using Raw CHA Images”,
IEEE 2004.
[6] H. Stark and P. Oskoui, Ϙ High resolution image
recovery from image-plane arrays, using convex
projections, ϙ J. Opt. Soc. Am. A, vol6, pp.1715-1726Δ
1989.
[7] Yan Chang, XiangZhong Fnag and Xiaokang Yang,
“Edge-image-based approach for stable super-resolution
reconstruction” Optical Engineering 46(2), 027004(February
2007)
[8] Vincent Laude and Carine Dirson, “Liquid-crystal active
lens: application to image resolution enhancement”, Optics
Communication 163, 72-78, 1999.
[9] Deepu Rajan, Subhasis, and Chaudhuri, “An MRF—
Based Approach to Generation of Super-Resolution Images
from Blurred Observations”, Journal of Mathematical
Imaging and Vision 16: 5-15, 2002.
[10] T.S.Huang and P.M.Narendra, “Image restoration by
singular value decomposition”, APPLIED POTICS, Vol.14,
No.9, September 1975.
[11] T.Kobayashi, T.Fujii, T.Kimoto, and M.Tanimoto,
“Interpolation of ray-space data by adaptive filtering,”
proceedings of SPIE, 3958, p.252-259, 2000.