Content-Based Image Retrieval Method and its Application to Shoeprint

Identification

Xuejing Dai
Department of Forensic Science
China Criminal Police College
Shenyang, China
Dai-xuejing @tom.com

Abstract—In this paper, content-based image retrieval method
is applied to develop a system for automatic retrieval of
questioned shoeprint images from a reference database of
shoeprint images. The organizational structure, functional
features, shoeprint query of the system and its application to
footwear impression examination and identification is
introduced in more detail. Comparison experiment is made to
demonstrate its ability of retrieving similar images and
identifying shoeprints. Experimental results show that the
retrieval speed and performance of the system are satisfying.
Keywords-content-based; shoeprint image; identification;
case linking
I. INTRODUCTION
Footwear marks provide valuable forensic evidence. In
many instances, shoe marks can be positively identified as
having been made by a specific shoe to the exclusion of all
other shoes. In other instances, details retained in a shoe
mark may be insufficient to uniquely identify an individual
shoe but they are still very valuable. Due to the wide variety
of shoes available on the market, with most having
distinctive outsole patterns, if the model of a shoe can be
determined from its mark, then this can significantly narrow
down the search for a particular suspect [1].
The photograph of the impression or of the lifted
impression or cast can be subsequently scanned and a digital
image produced. Forensic analysis requires comparison of
this image against specific database which includes (i) marks
made by shoes currently and previously available on the
market and (ii) marks found at crime scenes. The comparing
process is a laborious task and commonly conducted
manually by searching paper catalogues or computer
database. Thus, computer-based methods that reduce the
operator’s effort for this task offer great benefit to forensic
scientists.
Automatic matching of footwear patterns has been little
explored. Early work [2-6] involves semi-automatic methods
of manually annotated footwear impression descriptions
using a codebook of shape primitives, e.g. wavy patterns,
geometric shapes and logos. The query print needs to be
encoded in similar manner. The process is laborious and of
poor performance due to inconsistent user encoding.
Reference [7] employs shapes generated from footwear
impressions using image morphology operators. Spatial
positioning and frequencies of shapes are used for
classification with a neural network. Reference [8,9] use
fractals to represent prints and mean square noise error
classification. Fourier Transforms (FT) have been used for
classification of full and partial prints [10,11]. The FT is
invariant to translation and rotation and this approach shows
that although footwear impressions are processed globally,
they are encoded in terms of the local information evident in
them. In [12] pattern edge information is employed for
classification. The approach deals well with variations,
however query examples originate from the learning set and
no performance is given for partial prints. Most recently [13]
invariant local feature descriptors and spectral matching have
been used.
Due to the fact that large volumes of images are being
produced and that it is expensive or impossible to annotate
each of them by type, content-based image retrieval (CBIR)
method is widely applied [14,15]. The queries to a CBIR
system are such as “find the K most similar images to this
query image,” or “find the K images which best match this
set” on the basis of the simple visual characteristics of
images [16]. In this paper, a CBIR system is developed and
its application in forensic science is demonstrated.
II. FUNCTIONAL FEATURES OF THE SYSTEM
A. Construction of the system
This system adopts a C/S mode with 3-layer construction,
in which the user interface layer includes the system frame,
image file correlation and function control; the operation
logic interface takes on database operations such as import,
search, statistics and maintenance; the data interface holds
the management data of the system, see Fig. 1.
Figure 1. Main interface of the system

978-1-4244-3709-2/10/$25.00 ©2010 IEEE

B. Shoeprint collection are drawn and part of the image is erased to mend the
Several approaches are provided to capture shoeprint excessive or incorrect regions of the image, see also Fig. 3-d.
images, while digital camera and scanner are the most
commonly used equipment in collecting shoeprints. For
digital camera, the standard PNP ports are adopted such as
USB 2.0 or 1394, by which the pictures can be imported the
same way as from local disk. For scanner, custom software
and driver are needed. Besides, a special collecting
instrument is often used to collect shoe sole pattern images
of the suspect and entered directly into the system, see Fig. 2.

Figure 2. Shoe sole pattern image collection of suspect

C. Shoeprint entry
1) Pretreatment of the shoeprint image
The aim of pretreatment operation is to trim the shoeprint
image so as to be ready for the entry of characteristic
information of the footwear impression collected at the crime
scene in a relatively easy way by the operator. Main
pretreatment operations consist of improving the image
quality, adjusting the ratio of shoeprint to the whole image,
scale setup of the image and saving of information for
characteristic entry etc. see Fig. 3-a.
2) Information import
Text information import is executed to complete data
operation and communication with the database, which
includes data display control, adding, deleting and modifying Figure 3. Shoeprint entry
of the data and image file upload etc.
Characteristic patterns of shoeprints are defined as those
raised ones on the shoe sole, fine and easily worn patterns in III. SHOEPRINT QUERY AND CASE LINKING
combinatory patterns, see Fig. 3-b. Whereas non knaggy
A. Shoeprint image query
patterns are ignored, see Fig. 3-c.
In order to improve the accuracy and integrality of Once the treated image is satisfactory, click the
shoeprint query, it is desired that shoeprints should be characteristic extraction button and a dialogue frame pop-ups
separated from the background, debris or shadows etc. with showing “successfully saved”, which means the extracted
the outsole patterns shown in the foreground color(white) characteristics have been imported to the database and it is
and the other parts in the background color(black), see Fig. ready for shoeprint query. The interface of shoeprint image
3-d. query is shown in Fig. 4.
Interactive image enhancement operations are available
and necessary to describe or trim the shoeprint images again
so that the valid characteristic information can be managed
and imported to the database by background process of the
system program. Commonly used operations include image
partition, binarization, denoise and reverse. Sometimes lines

Figure 4. Shoeprint image query
In shoeprint searching, the query print is compared with
all corresponding shoeprints in the database one by one.
Then each partition of the questioned shoeprint is compared
with that of the shoeprint found to calculate the similarity
respectively. The similarity of partition i is
1 n | Aij − Bij |
Li = 1 − ∑ σ j , (1)
n j =1 max( Aij , Bij )
and the total similarity of the whole image is known as
1 m
L = ∑ δ i Li .
m i =1
Where δi is the proportion coefficient of partition
numbered i (i ranges from 1 to m),
σj is the proportion coefficient of characteristic numbered
j (j ranges from 1 to n),
Aij is the characteristic numbered j of the partition
numbered i in the questioned footprint,
Bij is the characteristic numbered j of the partition
numbered i in the known footprint,
m is the number of footprint partitions,
n is the number of characteristics extracted from each
partition.
B. Linking cases with shoeprints
Shoeprints collected from different crime scenes are
usually compared to link cases or/and suspect(s). This
operation is limited to the manager of the system, which
means the case linking operation is not only the comparison
but also the management of similar shoeprint images that are
becoming bigger and bigger in number. Case linking records
and related information are provided by the system based on
shoeprint query with shoeprint characteristics and auxiliary
text information, see Fig. 1.
IV. COMPARISON EXPERIMENTS
A. Shoeprint database
The research involves developing a large database of
3000 representative images consisted of footwear
impressions found at the scenes of crime and of shoe sole
designs from suspects out of which one hundred images of
questioned shoeprints and suspect shoeprints each are
randomly taken so that an experimental search can be
conducted to test the performance of the system.
B. Results
It is not all the case that the more partitions selected the
better the search result. Generally it is advisable to choose 2
to 5 partitions where the shoeprint characteristics are quite
obvious. The images found are displayed according to the
number given and the sort rules for search, with the best
matched shoeprints in the forefront, see Fig. 5.
(2)
Figure 5. Search results
C. Discussion
A quantitative measurement of the search result is listed
in table Ⅰ, with the average search time of one shoeprint
about 60s to 80s, and the time for one by one comparison
about 0.01s to 0.015s under the conditions of ordinary
computer as the server. In general, the speed of retrieving a
query print is satisfactory and high accuracy has been
achieved. Nevertheless, further study is needed to design
algorithms for not only the automatic recognition of
extraordinarily complicated texture and logo pattern, but also
the improvement of algorithm accuracy in recognition of
blurred crime scene impressions.
V. APPLICATIONOF THE SYSTEM
The system is widely used in many police stations
through out the country. Generally, the system is applied in
four modes. Firstly examine from case to case, which means
to search the questioned footwear impressions in crime scene
shoeprint database. Secondly search suspect from cases,
which means to find out the suspect by searching the
questioned footwear impressions in the sample shoeprint
database. Thirdly search cases related to the suspect, which
means to search the known shoeprints in the crime scene
shoeprint database. Fourthly search from suspect to suspect,
which means to search and identify the suspect in the known
sample shoeprint database. The system proofs to be quite
effective from case solving practice and the utility of
footwear impressions increases greatly as convincing
physical evidence.
REFERENCES
[1] W. Bodziak, Footwear Impression Evidence Detection, Recovery and
Examination, second ed. CRC Press, 2000.
[2] A. Girod, Computer classification of the shoeprint of burglars’ shoes,
Forensic Science Int. 82 (1996) 59–65.
[3] M. Phillips, Shoeprint image coding and retrieval system,
Proceedings of the European Convention on Security and Detection.
1995, pp. 267-271.
[4] N. Sawyer, SHOE-FIT: A computerized shoe print database, Proc.
European Convention on Security and Detection,1995.
[5] W.Ashley, What shoe was that? the use of computerized image
database to assist in identification, Forensic Science Int. 82(1) (1996)
7–20.
[6] S. Mikkonen, T. Astikainenn, Databased classification system for
shoe sole patterns-identification of partial footwear impression found
at a scene of crime, Journal of Forensic Science 39(5) (1994) 1227–
1236.
[7] Z. Geradts, J. Keijzer, The image-database REBEZO for shoeprints
with developments on automatic classification of shoe outsole
designs, Forensic Science Int. 82(1996) 21–31.
[8] A. Alexander, A. Bouridane, D.Crookes, Automatic classification and
recognition of shoeprints, Proc. Seventh International Conference
Image Processing and Its Applications 2 (1999) 638–641.
[9] A. Bouridane, A. Alexander, M. Nibouche, D. Crookes, Application
of fractals to the detection and classification of shoeprints, Proc. 2000
International Conference Image Processing 1 (2000) 474–477.
[10] C. Huynh, P. de Chazal, D. McErlean, R. Reilly, T. Hannigan, L.
Fleud, Automatic classification of shoeprints for use in forensic
science based on the fourier transform, Proc. 2003 International
Conference Image Processing 3 (2003) 569–572.
[11] P. de Chazal, J. Flynn, R. B. Reilly, Automated processing of
shoeprint images based on the fourier transform for use in forensic
science, Pattern Analysis and Machine Intelligence, IEEE
Transactions on 27 (2005) 341–350.
[12] L. Zhang, N. Allinson, Automatic shoeprint retrieval system for use
in forensic investigations, UK Workshop On Computational
Intelligence (UKCI05) ,2005.
[13] M. Pavlou, N. M. Allinson, Automatic extraction and classification of
footwear patterns, Lecture Notes in Computer Science, Proc.
Intelligent Data Engineering and Automated Learning, Burgos, Spain
(2006) 721–728.
[14] Y. A. Aslandogan, C. T. Yu, Techniques and systems for image and
video retrieval, IEEE Trans on Knowledge and Data Engineering,
1999,11(1):56-63.
[15] M. T. Meharga, C. Plazanet, S. Spaccapietra, A Content-Oriented
Shoeprint Identification System. Eighth Working Conference on
Database Semantics(DS-8),1999, pp.109-22.
[16] A. K. Jain, A. Vailaya, Image Retrieval Using Color and Shape,
Pattern Recognition, 1997,29(8):221-228.