Engineering Science and Technology, an International Journal 23 (2020) 264–273

Contents lists available at ScienceDirect

Engineering Science and Technology,

an International Journal
journal homepage: www.elsevier.com/locate/jestch

Full Length Article

Anti-spoofing method for fingerprint recognition using patch based deep

learning machine
Diaa M. Uliyan a,⇑, Somayeh Sadeghi b, Hamid A. Jalab b
a
Middle East University, Faculty of Information Technology, Amman 11831, Jordan
b
University of Malaya, Faculty of Computer Science and Information Technology, Kuala Lumpur 50603, Malaysia

a r t i c l e i n f o a b s t r a c t

Article history: Today’s with increasing identity theft, biometric systems based on fingerprints have a growing impor-
Received 7 January 2019 tance in protection and access restrictions. Malicious users violate them by presenting fabricated
Revised 30 April 2019 attempts. For example, artificial fingerprints constructed by gelatin, Play-Doh and Silicone molds may
Accepted 16 June 2019
be misused for access and identity fraud by forgers to clone fingerprints. This process is called spoofing.
Available online 4 July 2019
To detect such forgeries, some existing methods using handcrafted descriptors have been implemented
for assuring user presence. Most of them give low accuracy rates in recognition. The proposed method
Keywords:
used Discriminative Restricted Boltzmann Machines to recognize fingerprints accurately against fabri-
Biometric systems
Deep learning
cated materials used for spoofing.
Discriminative Restricted Boltzmann Ó 2019 Karabuk University. Publishing services by Elsevier B.V. This is an open access article under the CC
Machines BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Fingerprint authentication

1. Introduction detection techniques have been developed [8–10]. Thus, misrepre-

Today, one of major challenges confronting biometric systems is
the rapid threat of malicious actions. The most of malicious actors
use a common type of presentation attack, known as ‘‘spoofing” to
defeat biometric systems [1]. The main goal of presentation attack
is impersonating target victims that have the desired authoriza-
tion. It happens when intermediate spoofing forgers intentionally
guess that identity of unsuspecting individuals via steal victim’s
fingerprints, tampering them with a certain legal material to
defraud fingerprint recognition based systems [2].
Failure to prevent fingerprint spoofing forgeries on devices [3]
may compromise confidential information in many applications
such as video surveillance [4], biometric identification [5] and face
indexing in social media [6]. This issue motivates researchers to
employ countermeasure techniques and combined them into the
biometric based systems to beat such forgery. Fig. 1 shows some
examples of fingerprints which gained from real and fake fingers.
Visually by the eyes, it cannot distinguish between real and fake
ones.
The first row in Fig. 1 shows the real fingerprints. The second
row shows fake ones were acquired from artificial fingers which
made by different fabricate materials. Many spoof fingerprint
⇑ Corresponding author.
E-mail address: duliyan@meu.edu.jo (D.M. Uliyan).
Peer review under responsibility of Karabuk University.
sentation of fingerprints and detection of forged fingerprints is still
an open issue [11].
There are many countermeasure methods have been proposed
[12] that employed multiple factors in securing the information.
Spoof forgeries [13,14] can be defined as the techniques used to
deceive biometric based systems by giving a forged identity of
the user to have authentication. The authentication system mainly
considers individuals of users based on behavioral and biological
features such as face recognition, iris features, voice signals, finger-
print and palm veins [15]. In case of fingerprint spoofing forgeries,
the intermediate spoof forger uses finger molds to deceive the bio-
metric authentication system. For instance, palm images have been
printed and easily defeat the biometric system [16].
Thus, fingerprint spoofing detection methods are classified into
two classes depending on whether additional sensors are used or
not:
1) Hardware-based (exploring extra finger readers) [17]: the
state of art fingerprint readers can scan the input finger-
prints with high resolutions which are suitable for matching.
The fingerprint reader executes misrepresentation detection
on two types of finger images: non processed and raw fin-
gerprint images [15]. While these raw images contain a clear
information that provides discriminate information for spoof
detection, these raw images have to be processed such as
color conversion, normalization or filtering to extract

https://doi.org/10.1016/j.jestch.2019.06.005
2215-0986/Ó 2019 Karabuk University. Publishing services by Elsevier B.V.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
 D.M. Uliyan et al. / Engineering Science and Technology, an International Journal 23 (2020) 264–273
Fig. 1. Fingerprint samples using Cross Match 2013 reader [7].
features for matching. The main objective of hardware based
methods is to reveal the features of liveness, such as blood
flow [18], skin distortion [19], odor [13]. As pointed in
[20], more details about Liveness Detection methods were
tested in the LivDet datasets.
2) Software-based methods rely on the image information
scanned by intent sensor without requiring more hardware
cost, to classify images into: live and spoof fingers [10].
Among these methods, the software based methods has a virtue
it can be adapted in a common fingerprint scanners to recognize
and examine the fingerprint if it is forged by using the fake mate-
rials in fingerprint image. In the paper, the software based method
is considered in research to detect fingerprints under spoof mate-
rials. Software methods can be categorized into two main classes
[21]: feature based [22] and deep learning based. The feature based
recognition method is mainly extracts discriminate fingerprint
liveness with a single feature point in early stage, and it has not
been shown good performance for various fake materials. It can
be noticed, that deep learning methods are used to learn fake fin-
gerprints for the various types of spoof materials [23].
2. Related works
With the rapid use of authentication systems using fingerprint
recognition for many applications in recent years, the task of
observing fake fingerprint becomes a vital due to fingerprint can
be easily forged by using different types of instruments, for
instance, wood glue, gelatin, silicone or printed fingerprints [1].
The texture of a real finger may have lasting residual effects on sur-
face and then, is transferred into the material to counterfeit any
biometric authentication system. Due to the identical texture
between real and fake fingerprint, this issue will easily can deceive
the authentication fingerprints based systems. Various fingerprint
spoofing materials have been explored in [24]. The real scenario is
that the forger tries to bypass a fingerprint recognition sensor by
replicating a certain fingerprint image. The cloned fingerprint has
an artifact which is employed to do presentation attack.
The main challenge in research is how to discriminate genuine
living fingerprints from fake ones [25] which is inspired by The ISO
standard IEC 30107-3 (E). It gives a foundation for presentation
attack detection by establishing a common framework that specify
presentation attack events and detect them [26]. Two well known
methods are used fabricate and replicate a fingerprint:
1) Cooperative methods: the finger of user must be placed into
certain elastic material [27]. These materials help to create a
mold of a fingerprint using for example, silicone, gelatine, or
PlayDoh.
2) Non-cooperative method: it is made when the user acci-
dently, left a trace of fingerprint on a surface and it required
265
to enhance these traces. This process can be done by captur-
ing fingerprint traces and save it into image. The fingerprint
image is enhanced and then, molds are created materials like
silicone, gelatine, or PlayDoh over the printed image [28].
For instance, researchers at Michigan State University help
police in July 2016 to unlocked a fingerprint secure smart
phone with homicide case1 based on a 2D printed fingerprint
spoof method proposed by [29].
Various materials can reveal such features: gradients, ridge and
intensity in forged fingerprints. Therefore, the task of the existing
anti spoof fingerprint recognition systems extensively depends
on the type of fabricated materials used to create fake fingerprints
via training step [30]. For instance [31], mentioned that single low
level feature-based methods struggles to carry out various spoof-
ing fingerprint materials. Features in fingerprint images are the
core of discriminating forged fingerprints for different kinds of fake
fingerprints. Multiple features become a practical way to charac-
terize the properties of forged fingerprints and the real ones when
the sole of the authentication system does not know what the
types of materials used for forgery. For instance [10], have com-
bined multiple features: 1-gradient features from the input image
where local interest points detected by SURF method. 2-pyramid
multiscale features of the Histograms of Oriented Gradient. 3- Tex-
ture Gabor features. After combining the three features, the
dynamic score level fusion approach is employed to decide if the
fingerprint is fake or real. Their method is tested on LivDet 2011
dataset with an Average Equal Error Rate of 3.95% and also it
decreased the Average Classification Error rate to 2.27%.
Rattani and Ross [32] proposed a fingerprint spoof detector
using combined multiple features: Grey Level Co-occurrence
Matrix (GLCM) with Histogram of Oriented Gradients (HOG) for
exposing liveness fingerprints.
Xia et al. [33] developed a fingerprint recognition method based
on intensity variance features and local binary gradient orienta-
tion. These features are combined in a feature vector using co-
occurrence probability method which is finally, classified by sup-
port vector machine (SVM) classifiers.
In fingerprint spoofing recognition systems, two common ways
have been identified for detecting the spoof fingerprint: 1) the
active liveness features of fingerprint detectors, for instance, by
examining pulse, perspiration patterns and blood pressure [34];
2) Passive pattern analyzer for fabricating materials [35] for
instance, the lack of details of spoofed fingerprints according to
the real ones, fingerprint pattern differences. The last type of
spoofing problem is the scope of our proposed method discovers
a threat of using material and sensor.
Several studies spoofing methods in fingerprint recognition
have been introduced in [5]. Among the most general methods
for static extracted features: texture based anti-spoofing tech-
niques involves statistical feature analysis [36], Ridge based fea-
tures [37], curvelet transform [38], Power Spectrum Fourier
based features [39], Local Phase Quantization patterns [36] and
Local Binary Patterns [40].
However, recent methods focused on combining multiple fea-
tures and probably even multiple liveness detectors. For instance
[41], suggest a fusion of liveness detector with a single modality
for anti-spoofing capabilities. Galbally et al. [42] employed multi-
ple features of spoof fingerprints such as, orientation certainty
level and local clarity score to discover its liveness.
Recently, some existing methods inspired by its deep learning
feature extraction, such as a Convolution Neural Network (CNN)
1
https://statenews.com/article/2016/08/how-msu-researchers-unlocked-a
fingerprint-secure-smartphone-to-help-police-with-homicide-case
266 D.M. Uliyan et al. / Engineering Science and Technology, an International Journal 23 (2020) 264–273

which can be trained to distinguish a live finger from a spoof shown in Fig. 2 arrange two types of layers: a visible layer v with
[6,43]. Deep learning has been adopted to generate a robust finger- stochastic m neurons are direct relations with a hidden layer h
print spoofing system. with n neurons. However, there are no connections between neu-
For instance [31], shows that convolution neural network can rons in the same layer, and this restriction gives the RBM its name.
identify the forged fingerprints with known fabricate materials. A weight matrix is constructed asWmXn . It has the weights between
Chugh et al. [44] proposed a method for fingerprint recognition the connected, visible and hidden neurons, where wij represents
by extracting local regions from the image itself. These patches symmetric weights between the ith visible neuron vi connected
were centered and aligned using minutiae features to provide sali- with the jth hidden neuron hj .
ent descriptors. These cues are trained via CNN models to improve Each unit in a visible layer V is connected with all the units in
detection results. the hidden layer h. Assume layers, V and h, with binary stochastic
In this paper, we proposed a deep learning method for to recog- variables, i.e., V 2 ½0; 1m and h 2 ½0; 1n . Where m and n are the
nize real fingerprints and detect spoofs. The main contributions of numbers of units of the visible and hidden layers. The energy func-
our method are summarized as follows: tion for connecting configuration between V and h layer from Ber-
noulli distribution with success probability in RBM method [48] is
1) It attempts to distinguish real fingerprint images from fake defined as
ones.
X
m X
n X
m X
n
2) It analyzes the image consistency based on scaled and EðV; hÞ ¼  ai vi  b j hj  vi hj wij ð1Þ
rotated ROIs. Based on these ROIs features, we propose a i¼1 j¼1 i¼1 j¼1
deep discriminative model for training detection.
where ai and bj are the biases of ith visible layer and jth hidden
The rest of the paper is presented as follows. In Section 3, we layer, respectively. The model parameters ai is composed of vector
introduce two main techniques: DRBM and DBM used in the pro- a ¼ ½a1 ; a2 ;    :; aV  and bj is composed of vector b ¼ ½b1 ; b2 ;    :; bh .
posed method identify representation of real and forged finger- The marginal probability of a joint distribution function over visible
prints. Experimental results are presented in Section 4. Finally, layer is defined as
Section 5 gives a conclusion and future works. 1X
PðvÞ ¼ expEðv;hÞ ð2Þ
Z h
3. Proposed method
where Z is the partition function defined as follows:
XX
We propose a novel method to determine fake fingerprints Z¼ expEðv;hÞ ð3Þ
regarding spoof forgeries were used in authentication based sys- v h
tems, we adapt multiple features called deep features which
Z is the sum of possible pairs of (v, h). Let V be a m dimensional vec-
extracted from images such as, Deep Boltzmann Machine (DBM)
tor and let h be a n dimensional binary vector. while visible units
[45]. The main benefit of DMB is that its layered architecture helps
are binary, we have total 2mþn pairs of ðv; hÞ.
to investigate a complex relationships between features and
Since the RBM is a probabilistic binary structure may includes
enables deep learning of high detailed features of data. Extracting
different layers, each higher layer takes the correlation between
features from the input image based on Deep Neural Networks aids
actions of upper hidden features from lower hidden layer. The
to understand data in depth. DBMs are probabilistic deep learning
lower layer generates visible units connected with the higher layer
that copes with complex patterns impressively by extracting
based on joint distribution with mutual independent conditional
highly detailed features from the image. It is suitable for tasks in
distribution probabilities PðhjVÞ and PðVjhÞ are defined as follows
which the patterns might not be easily detected or forged. Results
on the Cross Match [7] dataset show that our technique achieves X
n

good results with comparison to handcrafted based methods, the Pðvi ¼ 1jhÞ ¼ /ð wij hj þ ai Þ ð4Þ
j¼1
results show high detection rate regarding spoofing attack. The
motivation of using deep learning features includes: and
X
m
1) Restricted Boltzmann Machines (RBMs) adapted in many Pðhj ¼ 1jvÞ ¼ /ð wij vi þ bj Þ ð5Þ
applications such as image classification and medical image i¼1
investigation and pattern analysis to solve various learning
problems. where / () stands for the sigmoid function, /ðxÞ ¼ 1=ð1 þ expðxÞÞ.
2) Restricted Boltzmann Machines typically developed to For an nhidden-layer RBM, its model parameters consist of
extract features and build a self-contained framework for value
generating competitive non-linear classifiers. T ¼ fa1 ; b1 ; w1 ; a1 ; b1 ; w1 ;       :; am1 ; bn1 ; wmn1 ; am ; bn ; wmn g
3) We introduce RBM algorithm that introduces a discriminant
ð6Þ
factor to RBM training.
Given the training input data T of the visible neuron, it is not
Thus, the notion of two main deep learning features: Restricted easy to approximate the parameters of vector T by using maximum
Boltzmann Machines (RBM) and Deep Boltzmann Machines (DBM) likelihood criterion directly because of various hidden layers found
are introduced below regarding spoofing forgeries. in the RBM model. To solve this issue, we have implanted a greedy
based learning algorithm to train the stacked RBM model [49] in
the input image. A greedy learning algorithm tries to find out the
3.1. Restricted Boltzmann Machines (RBM)  
parameters of the first layer RBM a1 ; b1 ; w1 to model the visible
The (RBM) as introduced in [46] is a kind probabilistic graphical training data. Then, it saved the parameters of the first layer into
 
model based stochastic neural networks which are used to vector v ¼ a1 ; w1 and produce a Gibbs sampling from the first
describe the dependency among a set of random data variables layer in RBM as Pðhj ¼ 1jvÞ to train the next layer of RBM
 
using a two layer architecture. The architecture of an RBM as a2 ; b2 ; w2 , Let h = (a,b,W) be the set of parameters of RBM. The
 D.M. Uliyan et al. / Engineering Science and Technology, an International Journal 23 (2020) 264–273 267

To improve the recognition performance against noise and missing

data. 3) It has the ability to extract deep features obtained from a
large supply of unlabeled sensory inputs 4) It deals with vague
inputs robustly due to the the approximate inference step works
in a bottom up initialization and also can incorporate with top-
down view.
A deep bottom up Boltzmann machine as shown in Fig. 3, left
panel, in which each layer captures complicated, higher-order cor-
relations between the activities of hidden features in the layer
below.
After a bottom up initialization of the DBM to train layer by
layer manner. The learning procedure implemented using called
Mean Field (MF) [50] method to enhance its performance. First, a
Fig. 2. The graphical model for two layers: v and h in a) first layer of RBM and b) a stack of RBMs have been employed in and estimates the parame-
three hidden layer of RBM as introduced in [47].  
ters a1 ; b1 ; w1 of the first layer in RBM to model the visible train-
ing data. Then, it saved the parameters of the first layer and
produce Gibbs sampling from the first layesto train the second
goal of training step is to maximize the probabilities of occurrence layer of RBM. Mean Field inference method minimizes the total
of all the available training samples from input vector T which is energy of the RBM based on the parameters estimated via partial
defined as follows inferences which converges much faster than with random initial-
Y ization. Then, it computes an approximation of Q MF ðhjv; lÞ that
l ¼ argmax PðvÞ ð7Þ represents a fully factorized distribution to approximate the true
v2l
distribution of the hidden layers PðhjVÞ. This approximation is
When the fingerprint is scanned by reader, then it is converted computed as follows
to grayscale image that has real value pixels. A discriminative RBM  
YL YF  
 l k 
[48] is employed, which generate Gibbs samples from multinomial Q MF
ðhjv; lÞ ¼  q hl  ð10Þ
distribution with probability to make vector T has elements of 
l¼1 k¼1

Gaussian units. Therefore, Equation (1) can be redefined as follows
where L represents the number of hidden layers, F stands for the
 
1X m
ðv i  ai Þ2 X n Xm X n
vi number of nodes in the hidden layer. q hl ¼ l1 , where l ¼ 1. As
k
EðV; hÞ ¼   b j hj  hj wij ð8Þ
2 i¼1 ri
2
j¼1 i¼1 j¼1
ri
a result of DBM, the parameters of the mean field inference are con-
The conditional probability of visible layer based on Eq. (3) is rede- structed as follows
fined as follows l ¼ fl1 ; l2 ; :::; lL g ð11Þ
!
X
n
ai The choice of mean-field was measured due to two main rea-
Pðv i ¼ 1jhÞ ¼ N v ij wij hj þ ð9Þ
j¼1
r2i sons: First, it is a common type of approximate inference algorithm
that calculates the convergence criteria which considerably assists
where r is the variance of the normalGaussian distribution of N.
2
the structure of deep learning machine. Second, when biometric
systems have the tasks of image analysis, we assume that subse-
3.2. Deep boltzmann machines (DBM’s) quent layers over hidden layers of a given image that has a single
extracted feature. Then, mean-field should be suitable.
Deep boltzmann machines are adopted for several reasons: 1) Hence, the proposed method aimed to employ presentation
DBM’s is a talented method which can learn a complex internal attack detection (PAD) and the determination of a presentation
features from shapes and objects in image such as fingerprint. 2) attack starts by recording of a latent fingerprint from LiveDet

Fig. 3. a) three-layer Deep Boltzmann Machine. b): Pre training consists of learning a stack of modified RBM’s, which are then composed to create c) a deep Boltzmann
machine.
268 D.M. Uliyan et al. / Engineering Science and Technology, an International Journal 23 (2020) 264–273
Fig. 4. Input image and selected ROIs with their 10 constructed patches for each
ROI.
Dataset and apply post processing to remove the noise from the
image due to scanning errors and Close of ridge lines by scaling
operation to do the following steps:
1. Inputs and preprocessing: The original input fingerprint images
are entered as training set of grayscale images into the RBM
model to extract features from regions of interest (ROI). Fig. 4
presents, two modalities of the images were used: Grayscale
of the input images and patches of theirs normalized ROIs
includes 10 patches of scaled and rotated ROIs. This step might
help to increase the size of the training dataset and it’s manda-
tory to prevent missing of data when approximating the DBM’s
parameters. The first step is mandatory to observe the reflection
of fingerprint by examining three features: Shape, consistency
of the fingerprint from different rotation angles. It can be
noticed, that Attack presentation detection rate based on these
features differs from artefacts to bona fide fingers. It is imple-
mented as shown in Fig. 4.
2. Deep features extraction based on Discriminative RBM: the next
step is to pre-training patches of grayscale images through
DRBM. The process is applied in a greedy bottom-up way in
order to approximate the real-valued patches to posterior prob-
abilities of activation.
3. DBM training: Mean-Field algorithm was applied in this step
to update the weights and the biases of the RBMs stacked in
the DBM in a more accurate way. Finally, Multilayer Percep-
tron network was constructed using the same architecture
as the DBM to perform backpropagation for training. The
weights of each RBM stack were computed through local
parameters: a, b and W of each individually trained RBM as
shown in Fig. 5.
Fig. 5. The framework of the proposed method.
4. Feature vector generation: for each fingerprint image, we have
employed three stacks of RBMs for three Selected regions of
input image, which are: original scanned fingerprint image
40  40 pixel size, five patches detected under various scaling
factors Sx ¼ 0:3; 0:6; 0:9; 1:2; 1:5 and five rotated cropped
images under rotation angles h ¼ 30; 60; 90; 120; 18. While the
input layer of these RBMs includes the size of the 40  40  3
visible neurons, the hidden layer has 1000 neurons. These fea-
tures are saved in the feature vector of size 40  40  3 for
every patch image.
5. Linear Discriminant Analysis (LDA): we have considered the 11
(1 ROI + 5 scaled + 5 rotated) patch versions of the original fin-
gerprint image of size 725  800 implies a feature space of
40  40  11 = 17600 dimensions. It is computationally chal-
lenging to handle big matrices. The direct LDA algorithm in
[51] for high-dimensional data is employed. It optimizes Fish-
er’s criterion, which maximizes between class scatter and min-
imizes within class scatter. It is suitable for two class
classification tasks like biometric authentication systems to
make a decision if the fingerprint is valid or spoofed.
6. K -Nearest Neighbor (KNN): given a fingerprint image, its 10
patches have been constructed and classified using K-Nearest
Neighbor algorithm [52]. KNN is a nearest-neighbor classifica-
tion model in which classifies new cases based on a similarity
measure (e.g., distance functions). Because KNN classifier stores
training data, the model helps to compute substitution predic-
tions. In particular, KNN classifier aimed to apply binary classi-
fication in the context of spoofed fingerprint detection. Only
live fingerprint images are used for training the proposed
method as shown in Fig. 6.
Fig. 6. A spoof fingerprint detection is trained by KNN classifier to distinguish
samples of bona fide fingerprints from samples of known spoof ones.
 D.M. Uliyan et al. / Engineering Science and Technology, an International Journal 23 (2020) 264–273
4. Experiment results
4.1. Datasets
In this section, we used the state of art LivDet 2datasets: LivDet
2013 [53], and LivDet 2015 for fingerprints spoofing benchmark
[20,25] which are considered in the proposed method. The LivDet
2013 Contains two type of images: genuine images and spoofs with
total of 16.000 images and equally distributed as 50% of the images
were used as a training set and the remaining 50% as the test set for
classification.
Hence, the real images captured by four fingerprint scanners:
Biometrika, Crossmatch, ItalData and Swipe, Spoofs are created
using a gelatin materials like latex, play-doh and wood glue. LivDet
2013 utilized for use of the non-cooperative method without user
interference for creating spoof images. We used only Biometrika
and ItalData due to high resolution of their images. The second
dataset is livDet 2015 which contains a set of images were cap-
tured by four optical scanners; these scanners are Digital Persona,
Green Bit, Biometrika and Crossmatch. It noticed that the testing
set has spoof images which forged using unknown materials not
saved in the training set. Only gelatin material in the Crossmatch
were considered in our experiments as shown in Table 1.
Artificial fingerprints in Biometrika and Italdata have been cre-
ated without user assistance, while fake ones in CrossMatch were
created by user cooperation. Several materials for creating the syn-
thetic fingerprints were applied, such as: gelatin, wood glue, sili-
cone and latex. More concisely, experimental evaluation of the
proposed method is applied on a collection of a large and challeng-
ing LivDet 2013 and 2015 dataset. In LivDet 2013, over 1000 live
attempts were collected as well as, 1000 spoof fingers from various
materials such as Play-Doh, Gelatin and Ecoflex. In LiveDet 2015, it
is about 2011 training attempts have conducted on 1010 live fin-
gers with a resolution of 1000 dpi from 51 subjects depending on
the sensor, 2 fingerprints each of all 10 fingers and 1001 spoofed
fingerprints across 5 spoof materials to produce nearly 200 fin-
gerprint images per spoof material. 500 spoofed fingers made from
each of 5 fingerprints of 20 subjects for each of the five spoof mate-
rials. Only two attempts applied on each spoof.
4.2. Experimental setup
To achieve a perception of these three fingerprint benchmarks,
various parameters in our method have been adjusted to give a
good results as shown in Table 2. The implementation of the pro-
posed method was applied on Intel (R) Xeon (R) CPU E5-2690 v2
(3.00 GHz processor) with 20 GB RAM and NVIDIA GPU uses Mat-
lab 2016a.
4.3. Parameters evaluation
To do training and testing samples of fake fingerprints in the
dataset, we have tried various values of specific parameters: patch
numbers, learning rate and weight decay respectively on our
experiments. Appropriate values for these parameters are chosen.
As shown in Fig. 7, Using the 10 ROIs from the original input finger-
print in the training DBM model, significantly makes the proposed
method robust against missed or damaged ROIs of the input
images. Furthermore, it gives a high accuracy detection rate when
the size of the trained data is suitable in the dataset. The 10
patches of input image show that, the size of trained data is
increased to achieve about 0.95 accuracy rate. Figs. 8 and 9 show
2
LivDet datasets are available to researchers at http://livdet.org/registration.php.
269
the influence of modifying learning rate and weight decay param-
eters in the proposed method.
4.4. Self comparison
DRBM + DBM method is trained on three data sets for finger-
print detection. We use two groups of patches generated from
the input image, Scaled and rotated ROIs. To evaluate progress of
our method, three fingerprint scanners for spoof images were
regarded in our experiments. We have used the following metrics
as shown in Table 3: Accuracy rate (ACC), Half total error rate
(HTER), Fake Fingerprint Accuracy (FFA) and True Fingerprint
Accuracy (TFA), respectively and defined as follows
NT
ACC ¼  100% ð12Þ
NT þ NF
where NT represents the number of the correct fingerprint recog-
nized in the testing dataset and NF is the number of fake finger-
prints in testing dataset.
FAR þ FRR
HTER ¼ ð13Þ
2
where FAR is a False acceptance rate and FRR is a false rejection
rate. In order to assess the performance of the proposed method
based on various materials for spoofing forgeries, fake fingerprint
accuracy (FFA) and true fingerprint accuracy (TFA) are defined as
follows:
N1
FFA ¼  100% ð14Þ
NF
N2
TFA ¼  100% ð15Þ
NT
where N1 represents the number of fake fingerprint images which
are recognized falsely in NF. N2 represents the number of true fin-
gerprint images that, are identified as true ones in NT.
As pointed in Table 3, the computation cost of the proposed
method for a single tested image is 45 s when we test images on
Italdata dataset. Each step is given individually. Fingerprint prepro-
cessing step for one image required 0.2 s and fingerprint deep fea-
tures extraction required 10 s. The DBM training consumed 30 s
and KNN classifier required 4.8 s. Similarly, the total computation
time required to test one image on CrossMatch is 85 s, which is
slower than Italdata and Biometrica due the large blank areas.
We have tested three types of spoof materials: Wood Glue, gelatin,
Play-Doh and 2D-printed ones on the proposed method.
To examine the time complexity of the proposed method, two
experiments are conducted. The experiments considered the num-
ber of training fingerprints as well as number of input features
used in the tested fingerprint image. The computation times of
the proposed DRBM + DBM model are gained for different numbers
of training images. The LivDet dataset is employed in the first
experiment. The number of training images is ranged from 50 to
1000. The average processing time of the proposed method is cal-
culated against the number of training fingerprint images as shown
in Fig. 10. It shows that the time complexity of the proposed
method in regard to the number of training samples is O (n) based
on big O (.) notation.
In the second experiment for examining the time complexity of
the proposed method, different number of image features is con-
sidered to calculate the computation time of the proposed as
shown in Fig. 11. The proposed method used a feature vector with
size 128 features due to dimensionality reduction. The running
time based on 128 features is 45 s. The size of input image is
750  800. We selected ROI with size 40  40 to produce 1600 fea-
tures. For the ROI, 5 rotated patches and 5 scaled patches are
270 D.M. Uliyan et al. / Engineering Science and Technology, an International Journal 23 (2020) 264–273

Table 1
Information of LivDet datasets 2013 and 2015 used in the proposed method.

Dataset LivDet 2013 LivDet 2015

Fingerprint scanner Biometrika ItalData CrossMatch
Scanner model FX2000 ET10 L Scan Guardian
Image size 315  372 640  480 640  480
Resolution in dpi 569 500 500
Training/testing for live image 1000/1000 1000/1000 1510/1500
Training/testing for spoof images 1000/1000 1000/1000 1473/1473
Cooperative Subject No No Yes
Materials used for spoofing Ecoflex, Gelatine, Latex, Modasil, Wood Glue Body Double, Ecoflex, Play-Doh, OOMOO, Gelatin

Table 2 extracted to produce 1600  10 = 16000 features. The running time

Experimental setup. for 16,000 fatures is increased to 50 s which is slightly high which
Parameter Value
is still the future direction to reduce the time computation.
As shown in Table 4, non-cooperative test was used to enhance
Size of input image 750  800
Patch size 40  40
a latent fingerprint left on a surface and printing the negative
Patch numbers 10 patches impression on a 2D sheet. This 2D printed image can then be made
GB- RBM 864 visible, 1000 hidden neurons into a mold. Our method detects about TFA = 89% and FFA = 88%.
Maximum epochs 500 The main difference between TFA and FFA is that TFA represents
Meanfieldalgorithm 30 rounds
the number of fake fingerprint images which are recognized fal-
Learningrate 0.01
Momentum 0.5–0.92 sely. FFA represents the number of true fingerprint images that,
Weight decay 10–4 are identified as true ones.
Pooling size 13 Further, the performance of the our method gives a value of
Size of feature vector 128 posterior probability of the bona fide fingerprints normalized in

Fig. 7. Accuracy rate of the proposed method on Cross Match dataset.

Fig. 8. Accuracy rate versus Learning rate.

 D.M. Uliyan et al. / Engineering Science and Technology, an International Journal 23 (2020) 264–273 271

Fig. 9. Accuracy rate versus Weight decay.

Table 3
Performance evaluation of the proposed method on LivDet, 2013 (Biometrica and
Italdata) and LivDet, 2015 (CrossMatch).

Method
DRBM + DBM model ACC HTER Processing time per Dataset
image in seconds (s)
96.00% 3.50% 66 s Biometrica
95.00% 6.44% 85 s CrossMatch
94.50% 2.80% 45 s Italdata

the range 0–1 (0 is the degree of artifact fingerprints and 1 is bona

fide ones). The threshold parameter to make decision is set to 0.3.
To estimate the classification error rate in the proposed method,
two type of classification error rate are used. Attack Presentation
Classification Error Rate (APCER) and Bonafide Presentation Classi- Fig. 11. Computation time of the proposed method based on different number of
fication Error Rate (BPCER). APCER defines the rate of spoof finger- features per image.
prints called live). BPCER defines the rate of live fingerprints called
spoofs. 4.5. State of art comparisons
The APCER and BPCER rates of our algorithms are presented in
Table 5. The results from the three datasets: Biometrica, Cross- Some experimental results are mentioned in Table 6 to show
Match and ItalData, DRBM + DBM model performed the best in the robustness of our method compared with others: [35,54–56].
CrossMatch with APCER = 9.03% and BPCER = 6.4% due to high res- The Average Classification Error (ACE) is used as a performance
olution of images produced by L Scan Guardian reader. evaluation metric for robustness. It is defined as follows:

Fig. 10. Computation time of the proposed method when using different number of training fingerprints.
272 D.M. Uliyan et al. / Engineering Science and Technology, an International Journal 23 (2020) 264–273

Table 4
The performance of the proposed method using LivDet 2013 with regarding four types of fabricate fingerprint materials with memory space 61.5 MB.

Method Detection of artefact presentation with Spoof Materials

DRBM + DBM model Real Wood glue Gelatin Play-Doh 2D printed
TFA FFAfor spoof materials
0.95 0.88 0.90 0.90 0.88

Table 5
Classification error rates in percentage (%) over three datasets with known spoofs.

Biometrica CrossMatch ItalData

APCER BPCER APCER BPCER APCER BPCER
5.80% 9.7% 3.8% 6.4% 11.6% 9.1%

Table 6
Comparison table shows performance evaluation of existing methods compared with
our method on LivDet 2013.
Reference Techniques used ACE
Nogueira et al. [56] Convolutional neural networks 3.9%
Jiang and Xin [35] Co-occurrence matrix 11.00%
Gottschlich et al. [55] Histograms of gradients. 6.7%
Zhang et al. [54] Wavelet features and local binary pattern 2.1%
The proposed method DRBM + DBM 3.6%
FerrLive þ FerrFake
ACE ¼
2 local patches, Biometrics (IJCB), 2017 IEEE International Joint Conference on,
where Ferrlive is the percentage of misclassified real fingerprints
and Ferrfake is the percentage of misclassified fake fingerprints.
5. Conclusion
We proposed a novel deep learning model for examining finger-
prints based on DRBM and Deep Boltzmann Machine, which deals
with complex texture patterns in a robust way due to its proba-
bilistic multilayer architecture. In the proposed method, after
training a DBM, such structure has employed to extract deep fea-
tures of the grayscale fingerprints. KNN classifier is applied with
the feature vectors of the ROIs extracted by the DBM to examine
spoof forgeries. The experiment results demonstrate that the Deep
learning model is robust against different kinds of spoof forgeries
such as wood glue, Gelatin and Play-Doh. Deep features are
extracted from the real image of grayscale and Depth visual struc-
ture such as scaled and rotated patch ROIs. The performance eval-
uation of the DRBM + DBM method achieved state-of-the-art
results in three public fingerprint recognition benchmarks. How-
ever, our method still struggling to recognize fake fingerprints with
unknown materials. For future work, we need to extend the
method behavior to deal with spatio-temporal features of finger-
print images to explore the liveness properties. Furthermore, we
need to explore the capability of the proposed method in reducing
the time complexity of deep learning machine.
Acknowledgments
The authors are grateful to the Middle East University, Amman,
Jordan for the financial support granted to cover the publication
process of this research article.
References
[1] E. Marasco, A. Ross, A survey on antispoofing schemes for fingerprint
recognition systems, ACM Comput. Surv. (CSUR) 47 (2) (2015) 28.
[2] G. Fumera, Multimodal anti-spoofing in biometric recognition systems, in: S.
Marcel, M.S. Nixon, S.Z. Li (Eds.), Handbook of Biometric Anti-Spoofing:
Trusted Biometrics under Spoofing Attacks, Springer, London, 2014, pp. 165–
184.
[3] C. Roberts, Biometric attack vectors and defences, Comput. Sec. 26 (1) (2007)
14–25.
[4] K.W. Bowyer, Face recognition technology: security versus privacy, IEEE
Technol. Soc. Mag. 23 (1) (2004) 9–19.
[5] D. Menotti et al., Deep representations for iris, face, and fingerprint spoofing
detection, IEEE Trans. Inform. Forensics Sec. 10 (4) (2015) 864–879.
[6] M. Sajjad et al., CNN-based anti-spoofing two-tier multi-factor authentication
system, Pattern Recogn. Lett. (2018).
[7] L. Ghiani et al., Livdet 2013 fingerprint liveness detection competition 2013,
Biometrics (ICB), 2013 International Conference on, IEEE, 2013.
[8] Q. Huang et al., An evaluation of fake fingerprint databases utilizing SVM
classification, Pattern Recogn. Lett. 60 (2015) 1–7.
ð16Þ [9] T. Chugh, K. Cao, A.K. Jain, Fingerprint spoof detection using minutiae-based
IEEE, 2017.
[10] R.K. Dubey, J. Goh, V.L. Thing, Fingerprint liveness detection from single image
using low-level features and shape analysis, IEEE Trans. Inf. Forensics Secur. 11
(7) (2016) 1461–1475.
[11] C. Wang et al., A DCNN based fingerprint liveness detection algorithm with
voting strategy, Chinese Conference on Biometric Recognition, Springer, 2015.
[12] Y. Gao et al., Intermediate spoofing strategies and countermeasures, Tsinghua
Sci. Technol. 18 (6) (2013) 599–605.
[13] D. Baldisserra et al., Fake fingerprint detection by odor analysis, International
Conference on Biometrics, Springer, 2006.
[14] Z. Akhtar et al., Evaluation of serial and parallel multibiometric systems under
spoofing attacks, IEEE Fifth International Conference: Biometrics: Theory,
Applications and Systems (BTAS), IEEE, 2012.
[15] J.J. Engelsma, K. Cao, A.K. Jain, Raspireader: open source fingerprint reader,
IEEE Trans. Pattern Anal. Mach. Intell. (2018).
[16] J.-G. Wang et al., Person recognition by fusing palmprint and palm vein images
based on ‘‘Laplacianpalm” representation, Pattern Recogn. 41 (5) (2008) 1514–
1527.
[17] M. Espinoza, C. Champod, P. Margot, Vulnerabilities of fingerprint reader to
fake fingerprints attacks, Forensic Sci. Int. 204 (1–3) (2011) 41–49.
[18] M. Drahanský, W. Funk, R. Nötzel, Liveness detection based on fine movements
of the fingertip surface, 2006, IEEE Information Assurance Workshop, 2006.
[19] C.J. Lennard et al., Fingerprints and Other Ridge Skin Impressions, CRC Press,
2016.
[20] L. Ghiani et al., Review of the fingerprint liveness detection (LivDet)
competition series: 2009 to 2015, Image Vis. Comput. 58 (2017) 110–128.
[21] D. Yambay, Review of fingerprint presentation attack detection competitions,
Springer International Publishing, Cham, 2019, pp. 109–131.
[22] Y. Park, U. Jang, E.C. Lee, Statistical anti-spoofing method for fingerprint
recognition, Soft. Comput. 22 (13) (2018) 4175–4184.
[23] J.B. Kho et al., An incremental learning method for spoof fingerprint detection,
Expert Syst. Appl. 116 (2019) 52–64.
[24] C. Sousedik, C. Busch, Presentation attack detection methods for fingerprint
recognition systems: a survey, IET Biom. 3 (4) (2014) 219–233.
[25] V. Mura et al., LivDet 2017 fingerprint liveness detection competition 2017,
International Conference on Biometrics (ICB), IEEE, 2018.
[26] IEC, I.S., ‘‘ISO/IEC 30107-3:2016, InformationTechnology-Biometric
Presentation Attack Detection-Part 1: Framework”, in ISO/IEC 30107-1:2016
2016.
[27] H.-U. Jang et al., Fingerprint spoof detection using contrast enhancement and
convolutional neural networks, International Conference on Information
Science and Applications, Springer, 2017.
[28] I. Goicoechea-Telleria et al., Presentation attack detection evaluation on
mobile devices: simplest approach for capturing and lifting a latent
fingerprint, International Carnahan Conference on Security Technology
(ICCST), IEEE, 2018.
[29] K. Cao, A.K. Jain, Hacking mobile phones using 2D printed fingerprints.
Technical Report. MSU-CSE-16-2, 2016.
[30] A. Rattani, W.J. Scheirer, A. Ross, Open set fingerprint spoof detection across
novel fabrication materials, IEEE Trans. Inf. Forensics Secur. 10 (11) (2015)
2447–2460.
 D.M. Uliyan et al. / Engineering Science and Technology, an International Journal 23 (2020) 264–273
[31] Z. Xu, S. Li, W. Deng, Learning temporal features using LSTM-CNN architecture
for face anti-spoofing, Proceedings of the 3rd IAPR Asian Conference on Pattern
Recognition (ACPR), IEEE, 2015.
[32] Rattani, Ross, Automatic adaptation of fingerprint liveness detector to new
spoof materials, IEEE International Joint Conference on Biometrics, IEEE, 2014.
[33] Xia, Z., et al., A novel weber local binary descriptor for fingerprint liveness
detection, In IEEE Transactions on Systems, Man, and Cybernetics: Systems,
2018.
[34] P.D. Lapsley, et al., Anti-fraud biometric scanner that accurately detects blood
flow. 1998, Google Patents.
[35] Y. Jiang, X. Liu, Spoof fingerprint detection based on co-occurrence
matrix, Int. J. Signal Process. Image Process. Pattern Recogn. 8 (8) (2015)
373–384.
[36] L. Ghiani, G.L. Marcialis, F. Roli, Fingerprint liveness detection by local phase
quantization, Proceedings of the 21st International Conference on Pattern
Recognition (ICPR2012), IEEE, 2012.
[37] A. Abhyankar, S. Schuckers, Fingerprint liveness detection using local ridge
frequencies and multiresolution texture analysis techniques, International
Conference on Image Processing, IEEE, 2006.
[38] S.B. Nikam, S. Agarwal, Fingerprint liveness detection using curvelet energy
and co-occurrence signatures, Proceedings of the fifth international conference
on computer graphics, imaging and visualisation, IEEE, 2008.
[39] P. Coli, G.L. Marcialis, F. Roli. Power spectrum-based fingerprint vitality
detection, in 2007 IEEE Workshop on Automatic Identification Advanced
Technologies, 2007. IEEE.
[40] T. de Freitas Pereira et al., LBP TOP based countermeasure against face
spoofing attacks, Asian Conference on Computer Vision, Springer, 2012.
[41] Z. Akhtar et al., Evaluation of multimodal biometric score fusion rules under
spoof attacks, Proceedings of the 5th IAPR International Conference on
Biometrics (ICB), IEEE, 2012.
[42] J. Galbally et al., A high performance fingerprint liveness detection method
based on quality related features, Future Gener. Comput. Syst. 28 (1) (2012)
311–321.
273
[43] A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep
convolutional neural networks, in Advances in Neural Information Processing
Systems. 2012.
[44] T. Chugh, K. Cao, A.K. Jain, Fingerprint spoof buster: use of minutiae-centered
patches, IEEE Trans. Inf. Forensics Secur. 13 (9) (2018) 2190–2202.
[45] D.H. Ackley, G.E. Hinton, T.J. Sejnowski, A learning algorithm for Boltzmann
machines, Cognit. Sci. 9 (1) (1985) 147–169.
[46] G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with
neural networks, Science 313 (5786) (2006) 504–507.
[47] P. Smolensky, Information processing in dynamical systems: Foundations of
harmony theory, 1986, Univ at boulder dept of computer science colorado.
[48] V. Nair, G.E. Hinton. Implicit mixtures of restricted Boltzmann machines, in
Advances in Neural Information Processing Systems, 2009.
[49] G.E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief
nets, Neural Comput. 18 (7) (2006) 1527–1554.
[50] M. Welling, G.E. Hinton, A new learning algorithm for mean field Boltzmann
machines, International Conference on Artificial Neural Networks, Springer,
2002.
[51] H. Yu, J. Yang, A direct LDA algorithm for high-dimensional data-with
application to face recognition, Pattern Recogn. 34 (10) (2001) 2067–2070.
[52] M. Muja, D.G. Lowe, Scalable nearest neighbor algorithms for high dimensional
data, IEEE Trans. Pattern Anal. Mach. Intell. 11 (2014) 2227–2240.
[53] L. Ghiani et al., Livdet 2013 fingerprint liveness detection competition 2013,
International Conference on Biometrics (ICB), IEEE, 2013.
[54] Y. Zhang et al., Fake fingerprint detection based on wavelet analysis and local
binary pattern, Chinese Conference on Biometric Recognition, Springer, 2014.
[55] C. Gottschlich et al., Fingerprint liveness detection based on histograms of
invariant gradients, Biometrics (IJCB), 2014 IEEE International Joint
Conference, Citeseer, 2014.
[56] R.F. Nogueira, R. de Alencar Lotufo, R.C. Machado, Evaluating software-based
fingerprint liveness detection using convolutional networks and local binary
patterns, Biometric Measurements and Systems for Security and Medical
Applications (BIOMS) Proceedings, 2014 IEEE Workshop, IEEE, 2014.