IET Computer Vision

Research Article

Animal classification using facial images with ISSN 1751-9632

Received on 31st January 2017
Revised 11th January 2018
score-level fusion Accepted on 1st March 2018
E-First on 21st March 2018
doi: 10.1049/iet-cvi.2017.0079
www.ietdl.org

Shahram Taheri1, Önsen Toygar1

1Computer Engineering Department, Faculty of Engineering, Eastern Mediterranean University, Famagusta, North Cyprus, via Mersin 10,

Turkey
E-mail: onsen.toygar@emu.edu.tr

Abstract: A real-world animal biometric system that detects and describes animal life in image and video data is an emerging
subject in machine vision. These systems develop computer vision approaches for the classification of animals. A novel method
for animal face classification based on score-level fusion of recently popular convolutional neural network (CNN) features and
appearance-based descriptor features is presented. This method utilises a score-level fusion of two different approaches; one
uses CNN which can automatically extract features, learn and classify them; and the other one uses kernel Fisher analysis
(KFA) for its feature extraction phase. The proposed method may also be used in other areas of image classification and object
recognition. The experimental results show that automatic feature extraction in CNN is better than other simple feature
extraction techniques (both local- and appearance-based features), and additionally, appropriate score-level combination of
CNN and simple features can achieve even higher accuracy than applying CNN alone. The authors showed that the score-level
fusion of CNN extracted features and appearance-based KFA method have a positive effect on classification accuracy. The
proposed method achieves 95.31% classification rate on animal faces which is significantly better than the other state-of-the-art
methods.

1 Introduction many research studies. In [2] the authors suggested a score-level

Animal recognition and classification is an important area which
has not been discussed rapidly. Animal classification which relies
on the problem of distinguishing images of different animal species
is an easy task for humans, but evidence suggests [1] that even in
simple cases like cats and dogs, it is difficult to distinguish them
automatically. Animals have flexible structure that could self-mask
and usually they appear in complex scene. Also, as all objects, they
may appear under different illumination conditions, viewpoints and
scales. There are attempts to apply recognition methods on images
of animals but the specific problem of animal categorisation has
recently attracted limited interest.
Many existing methods showing promising results for human
fare recognition cannot properly represent the diversity of animal
classes with complex intra-class variability and inter-class
similarity. There are several kinds of approaches for solving this
problem with each one having its advantages and disadvantages.
The first approach constructs complex features which represents
and discriminates sample images better but creating such a feature
is complicated and it is problem dependent. The second approach
combines the extracted features from different methods and
concatenates them to build a more powerful feature vector.
Increasing the size of feature space causes increased problem
computation cost. Instead of using complex representation, the
information is consolidated from different classifiers and a decision
is made according to it. This method is known as score-level
fusion.
Score-level fusion shows very good performance in multimodal
biometric systems [2–5]. In a biometric recognition system, the
match score is a measure of similarity between the input and
template biometric feature vectors. When match scores’ outputs by
different biometric matchers are consolidated in order to arrive at a
final recognition decision, fusion is said to be done at the match
score level which is also known as score-level fusion. Apart from
the raw data and feature vectors, the match scores contain the
richest information about the input pattern. Additionally, it is
relatively easy to access and combine the scores generated by
different biometric matchers. Score-level fusion has been used in
fusion of fingerprint and face matchers for personal verification
under stress conditions. Elmir et al. [3] proposed a biometric
identification system which used score-level fusion of fingerprint
and voice. Sim et al. [4] presented a method that combines face
and iris biometric traits with the weighted score-level fusion
technique to flexibly fuse the matching scores from these two
modalities based on their weight availability. In [5], the authors
proposed a multi-biometric system that used three traits such as
fingerprint, palmprint and iris are combined by using weighted
fusion technique for person identification.
In this paper, we show that score-level fusion can be applied to
other classification and recognition problems. We believe that
performance rate can be increased when the information of two
different types of classification system are consolidated. For this
purpose, we construct two different classifier systems, one based
on features extracted from CNN and the other based on appearance
and shape-based features. Then we combine their information and
make decision according to this final result.
We tested our method with LHI-Animal-Faces dataset which
consists of 20 classes, 19 classes of different animals and one
human faces class. It is challenging to discern them for their
evolutional relationship and shared parts. Besides, interesting
within-class variation is shown in the face categories, including
rotation, flip transforms, posture variation and sub-types. From this
collection, we select 30 images from each class for training and
rest of images for test. We used this experimental setting in order to
make comparison with the other state-of-the-art methods. Our
results prove that the score-level fusion approach improves the
classification accuracy significantly.
The outline of this paper is as follows. In Section 2, an
overview of related work is given. Section 3 gives an overview on
methods that are used in our experiments. Section 4 is related to
experimental setup and results, and it is followed by conclusion in
Section 5.

IET Comput. Vis., 2018, Vol. 12 Iss. 5, pp. 679-685 679

© The Institution of Engineering and Technology 2018
Fig. 1  AlexNet architecture [15]
2 Related work
When dealing with a complex image database, it is important to
use a method capable of capturing information from all the
different classes. Sparse texture descriptors [6] depend on the
region detector used and how complex the structure and texture of
object and background are. As a result, using such methods may
fail, especially for animal classes with smooth skin texture and
complex background.
Animal recognition and classification can also be used in expert
systems for determining wild animal's migration corridors.
Animals are one of the object categories used in the literature for
object recognition. Success rate of object recognition depends also
on good object representation and characterisation. Object
characterisation can be achieved by visual descriptors, shape
descriptors or texture representation. In this paper, score-level
fusion of different visual descriptors was used to classify the
animal images.
One of the earliest attempts to perform recognition on an animal
database was done by Schmid [7]. They constructed models for
content-based image retrieval using Gabor-like filters. The method
was tested on only four different classes. All animals used in this
work had complex skin texture. Later, Ramanan et al. [8, 9]
introduced methods to detect textured animals using the shape and
texture information in video sequences. In an application for
searching images on the Internet, Berg and Forsyth [10] used four
cues: nearby texts on the web pages, colour, texture and shape to
re-rank the images retrieved by Google image search. They
reported that animals are among the hardest classes of objects for
recognition in computer vision.
Peng et al. [11] proposed a deep neural network for image
classification and tested it on LHI-Animal-Faces dataset. They
proposed a deep boosting framework based on layer-by-layer joint
feature boosting and dictionary learning. In each layer, they
constructed a dictionary of filters by combining the filters from the
lower layer, and iteratively optimised the image representation with
a joint discriminative generative formulation.
A method for visual object categorisation was presented by
Afkham et al. [12]. It is based on encoding the joint textural
information in objects and the surrounding background. The
authors tested this framework for classification among 13 different
classes of animals.
Si and Zhu [13] introduced a framework for learning a
generative image representation which only needs a small number
of training image samples. Each learned template is composed of
different image patches in which their features such as location,
scale and orientation may adapt in a local neighbourhood for
deformation. The appearances of these patches are characterised,
respectively, by their local sketch, texture gradients, flatness
regions, and colours. Then these patches are sorted and selected
according to their information gain. This automated feature
selection procedure allows their algorithm to scale up to a wide
range of image categories, from those with regular shapes to those
with stochastic texture. They evaluated the hybrid image templates
on several public benchmarks including LHI-Animal-Faces dataset.
3 Feature extraction methods
680
Feature extraction is an important step in a biometric system. The
method used for feature extraction significantly affects the final
decision for a biometric recognition system. Several feature
extractors that are used in the literature are implemented in this
study. These feature extraction methods, namely convolutional
neural network (CNN), histograms of oriented gradients (HOGs),
median robust extended local binary pattern (MRELBP), kernel
Fisher analysis (KFA) and biologically inspired features (BIFs) are
explained below briefly.
3.1 Convolutional neural network
CNN is a powerful machine learning technique from the field of
deep learning which has been used in many computer vision tasks
[14]. In this research, we apply a pre-trained CNN as feature
extractor to extract discriminate representations from animal faces.
In order to train a CNN, a huge set of training image samples is
necessaries. From these large training set images, CNNs
automatically learn extract discrimination features which in most
of the cases, outperform hand-crafted features such as HOG, local
binary pattern (LBP), or speeded-up robust features (SURF).
Traditional CNNs consist of stack of convolutional layers
followed by some fully connected layers and use the softmax
multi-category classifier with the cross-entropy loss function.
Firstly, we briefly describe each layer of CNN:
• Convolutional layer: the purpose of this layer is to extract
spatial correlation and invariant features by using various
convolutional filters.
• Pooling layer: in order to remove high-frequency noises, the
extracted feature maps of convolutional layer down-sample by
extracting local max (max-pooling) or average (avg-pooling)
value of each patch in the feature map.
• Fully connected layer: each neuron in this layer is fully
connected to all neurons of its two adjacent layers.
In the following sub-sections, we briefly introduce two of the
mostly used CNN architectures which we used for automatic
feature extraction from animal faces, namely AlexNet and
VGG-16.
3.1.1 AlexNet: AlexNet [15] is a deep CNN for image
classification that won the ImageNet large-scale visual recognition
challenge (ILSVRC)-2012 competition [16] and achieved a
winning top 5 test error rate of 15.3%, compared to 26.2%
achieved by the second-best entry. AlexNet is composed of five
convolutional layers (C1 to C5) followed by two fully connected
(FC6 and FC7) and a final softmax output layer (FC8). The
architecture of AlexNet is demonstrated in Fig. 1.
3.1.2 VGG-16: VGG-16 [17] architecture had been proposed in
ILSVRC 2014. The Oxford Visual Geometry Groups’ model is
deeper and wider than former CNN structure. VGG-16 has five
batches of convolution operations, each batch consist of 3–5
adjacent convolution layers. Adjacent convolution batches are
connected via max-pooling layers. The size of kernels in all
IET Comput. Vis., 2018, Vol. 12 Iss. 5, pp. 679-685
© The Institution of Engineering and Technology 2018

Fig. 2  VGG16 architecture [17]
Fig. 3  System architecture of BIF [21]
convolutional layers is 3 × 3 convolutional layers and the number
of kernels within each batches is the same (increases from 64 in the
first group to 512 in the last one). Fig. 2 illustrates a 16-layer VGG
architecture. VGG-16 network architecture has been used in many
researches and it was the first one that outperformed human-level
performance on ImageNet [18].
3.2 Histograms of oriented gradients
One of the popular feature extractors that has been used in object
classification and face recognition is HOG [19]. This method
constructs the features by counting occurrences of gradient
orientation in local patches or detection window of an image. In
this approach, an image is decomposed into different partitions and
then the histogram of orientations for each partition is computed.
Finally, all of these histograms are concatenated to construct HOG
feature. HOG is used in several recognition systems [20, 21].
3.3 Median robust extended LBP
Local binary patterns (LBPs) method [22] is considered among the
most computationally efficient high-performance texture
descriptors. LBP is used in several recognition problems such as
face, iris, palmprint and plant recognition [20, 21]. However, the
LBP method is very sensitive to image noise and is unable to
capture macrostructure information. In order to best address these
disadvantages, Liu et al. [23] introduced a novel descriptor for
texture classification, namely MRELBP. Instead of using raw
image intensities, MRELBP compares median of image intensities
in a local region. By using a multiscale LBP-type descriptor and a
novel sampling scheme, this method can capture both
microstructure and macrostructure texture information. MRELBP
IET Comput. Vis., 2018, Vol. 12 Iss. 5, pp. 679-685
© The Institution of Engineering and Technology 2018
is shown to be highly robust to different type of image noises, such
as salt-and-pepper noise, Gaussian noise, Gaussian blur and
random pixel corruption. Liu et al. [24] compared a large number
of LBP variants. They designed different experiments to measure
their feature descriptors’ robustness against changes in rotation,
viewpoint, illumination, scale, different types of image
degradation, number of classes, and computational complexity. The
best overall performance is obtained for the MRELBP feature.
3.4 Kernel Fisher discriminant analysis
The KFA method [25] is an extension of Fisher discriminant
analysis (FDA). In this approach as a first step, input space will be
expanded by using a non-linear mapping, and then in the obtained
feature space the multiclass FDA will be applied. By implementing
non-linear mapping the dimensionality of feature space will be
increased and as a result, it improves the discriminative ability of
the KFA method. The main advantage of the KFA method is that it
can be applied for multiclass pattern classification problems and it
solution is unique which is its superiority to generalised
discriminant analysis [25] method which produces multiple
solutions.
3.5 Biologically inspired features
BIFs are a method that tries to model visual processing in the
cortex as a stack of increasingly sophisticated layers. Riesenhuber
and Poggio [26] proposed a new set of features derived from a
feed-forward network of the primate visual object recognition
pathway, called the ‘HMAX’ model. The model consists of two
different types of layers: S units neurons (simple) C units neurons
(complex) (C). A remarkable feature of this model is that it uses
non-linear maximum operation ‘MAX’ in the simple units’ neurons
instead of using the linear summation operation ‘SUM’ in pooling
inputs at the layer, which is called C1, the maximum values within
local patches and across the scales within a band is computed.
Therefore C1 feature includes eight bands and four orientations.
The bio-inspired features (BIFs) have been investigated for object
category recognition [27, 28] and face recognition [29]. An
overview of BIF system is shown in Fig. 3.
4 Proposed method
According to the success rate of using score-level fusion in
multimodal biometric recognition systems, it is believed that
accuracy can be improved when the information of two different
types of classifiers are consolidated. Due to large inter-class
similarity and intra-class variation, we need to perform fusion of
two different kinds of feature descriptors. Appropriate score-level
combination of CNN and other simple features can achieve even
higher accuracy than applying CNN alone. In our proposed
method, we performed score-level fusion of features that are
extracted from two different descriptors.
For descriptor selection, we investigated different feature
descriptors and tried to select the most powerful ones. Inspired by
recent face and object recognition works [26–28], in the computer
vision community, we used features extracted from a fine-tuned
pre-trained CNN, which achieve state-of-the-art performance in
many computer vision tasks, and combined them with the features
that are extracted by KFA which is one of the most successful
appearance-based recognition systems in which the results of
recent research [25] show that it outperforms linear discriminant
analysis (LDA) [30] and principal component analysis [31]
methods.
The overview of proposed method is shown in Fig. 4. For each
input animal head image, in the pre-processing step, in order to
eliminate the negative effect of different factors such as size,
illumination and picture quality; we perform some simple
manipulation such as image resizing, converting from RGB to
greyscale and histogram equalisation. However, we only resize the
input image for CNN features. Then two different set of features
are computed. After that we compare the similarity of these feature
vectors with all of the feature vectors in the training set and select
the minimum one for each method. The distance between the test
681
Fig. 4  Block diagram of the proposed method
Fig. 5  LHI-Animal-Faces dataset. Five images are shown for each category
and training animal images is assumed to be the score of that
sample. In the next step, we normalised these obtained scores and
fused them together. Finally, we make decision by using nearest
neighbour (NN) classifier which uses the normalised fused scores.
5 Experiments and results
Several experiments have been carried out in order to show the
performance of the proposed method and the other state-of-the-art
methods. In the following subsections, the dataset description and
experimental setup and results are presented.
5.1 Dataset
The LHI-Animal-Faces dataset [13] consists of 19 classes of
animal head images plus one class of human head images with
overall 2200 images. Fig. 5 shows five sample images from each of
these categories. In contrast with other general classification
datasets, LHI-Animal-Faces contain only animal or human faces,
which have a large intra-classes similarities (due to evolutional
relationship, some animal face categories are similar to the other
class) and large inter-class variations (rotation, posture variation,
subtypes).
5.2 Experimental setup and results
In all of the following experiments, the LHI-Animal-Faces dataset
is divided into a training and test set in the same way as in the
AND-OR template (AOT) [32] method. We use 30 animal images
of each classes as a training set, and the remaining images of each
class for testing examples.
In order to eliminate the negative effect of different factors such
as size, illumination and picture quality, all the images have been
resized to 60 × 60 pixels and perform pixel intensity normalisation
682
and histogram equalisation in pre-processing step for local and
appearance-based features but for CNN features we only resized
the image to 224 × 224 for VGG16 and 227 × 227 for AlexNet
architectures. We investigated different type of feature descriptors
for both local feature-based and appearance-based classifier. We
used HOG, completed local binary patterns (CLBP), local binary
pattern histogram Fourier (LBP-HF), Haralick features and
MRELBP as local feature descriptors and tested LDA and KFA as
appearance-based feature extractors. In each method, we used
cross-validation approach to find the optimal parameter settings.
After feature extraction, we used a linear support vector machine
(SVM) for classification. The accuracy of each individual method
is demonstrated in Table 1.
Additionally, we used CNN as an automatic feature extractor.
We selected publicly available CNNs, AlexNet and VGG-16
architectures, pre-trained on the ImageNet database [18] for image
classification. Our choice is motivated by the impressive results
achieved using these two models on the ILSVRC [16] and these
two models in pre-trained version are freely accessible. These
models are trained over hundreds of thousands of different images
and can separate an image into 1000 pre-defined classes of
different objects such as car, bike, airplane and many other object
categories. As a result, these models have learned powerful
discriminative feature sets which can be used in the other object
recognition tasks.
The CNN architectures of AlexNet and VGG-16 contain
millions of parameters. Directly learning so many parameters from
only a few hundred training images from LHI-Animal dataset is
problematic. There are several solutions for this problem. Our first
solution to this problem is using a pre-trained CNN on a large
image collection like ImageNet database, and then re-use it on
LHI-Animal dataset. In order to use the representational power of
pre-trained deep networks, 4096-dimensional features are extracted
from the activations of FC7 layer. The reason that we selected
IET Comput. Vis., 2018, Vol. 12 Iss. 5, pp. 679-685
© The Institution of Engineering and Technology 2018
Table 1 Classification accuracy of different methods on LHI-Animal-Faces dataset
Type of methods Method Accuracy, %
local feature descriptor methods HOG 66.54
LBP 61.74
CLBP [33] 63.59
Fourier-LBP [34] 50.29
Haralick feature 49.27
BIF 68.46
median robust CLBP (MRCLBP) 68.46
appearance-based feature descriptor methods LDA 60.33
KFA 69.87
CNN features FC7 AlexNet features 89.91
FC7 VGG-16 features 92.84
Fine-tuned AlexNet 91.06
Fine-tuned VGG-16 94.39
score-level fusion methods LDA + HOG 74.32
LDA + LBP 68.91
LDA + CLBP 70.23
LDA + Fourier-LBP 62.44
LDA + Haralick feature 61.59
LDA + BIF 77.26
LDA + MRCLBP 76.30
LDA + FC7 AlexNet features 90.61
LDA + FC7 VGG-16 features 93.77
KFA + HOG 76.48
KFA + LBP 74.19
KFA + CLBP 74.14
KFA + Fourier-LBP 72.65
KFA + Haralick feature 70.94
KFA + MRCLBP 78.98
KFA + FC7 AlexNet features 91.37
KFA + FC7 VGG-16 features 94.21
KFA + FC7 fine-tuned AlexNet features 92.86
proposed method (KFA + FC7 fine-tuned VGG-16 features) 95.31

activations of FC7 layer for feature extraction is that based on
Donahue's results [35] the activations of the earlier layers can
extract mid-level image features and activations of the late layers
can be used as very powerful features for many classification
applications. In the next step, we rain a linear multiclass SVM on
the extracted features
In order to extract features from an image, the image is resized
and fed to the CNN as a multidimensional array of pixel intensities.
The input image size of VGG-16 net is 224 × 224 × 3 matrix, while
for the input image of AlexNet architecture is a 227 × 227 × 3
matrix. As the experiment results in Table 1 show, the classifier
trained using AlexNet features provides close to 90% accuracy and
the classifier trained using VGG-16 features provide close to 93%
accuracy which are higher than the accuracy achieved using the
hand-crafted features [36] such as LBP and HOG. The reason that
VGG-16 outperforms AlexNet is that VGG-16 architecture is much
deeper than the AlexNet, with 16 layers in total, 13 convolutional
and three fully connected layers. VGG-16 uses small convolutional
filters of 3 × 3 pixels so each filter captures simpler geometrical
structures but in comparison allows more complex reasoning
through its increased depth.
Another solution for the problem is fine-tuning the pre-trained
CNN by using our limited dataset. However, since our dataset is
too small, fine-tuning the pre-trained network on a small dataset
might lead to overfitting, especially since the last few layers of the
VGG and AlexNet network are fully connected layers. In order to
avoid overfitting, we increase the number of training samples with
the common data augmentation strategies such as translation,
rotation, flipping. Training on the augmented dataset will make the
resulting model more robust and less prone to overfitting. We
increase the size of the training set five times by performing
random rotations and translations on each image sample. Also, we
resize the training images to 256 × 256 and then we select four
different patches with size 224 × 224 for VGG-16 and 227 × 227
for AlexNet by using the four image corners, and use them as the
input of corresponding CNN architecture.
As the first step of fine-tuning, we truncate the last layer
(softmax layer) of the VGG-16 and AlexNet pre-trained networks
and replace them with our new softmax layer with 20 outputs or
categories instead of 1000 categories. We freeze the weight for the
first early layers so that they remain intact throughout the fine-
tuning process. Then fine-tune the network by using stochastic
gradient descent algorithm to minimise the loss function with a
small initial learning rate of 0.001.
We believed that appropriate score-level combination of CNN
and other simple features can achieve even higher accuracy than
applying CNN alone. In order to show that score-level fusion can
improve the accuracy of classification system, we tested different
type of feature descriptors, namely local feature-based, appearance-
based and CNN-based features. We used HOG, CLBP, LBP-HF,
Haralick features and MRELBP as local feature descriptors, tested
LDA and KFA as appearance-based feature extractors and used
FC-7 activation features of AlexNet and VGG-16 as CNN-based
feature. We tested score-level fusion of different combination of
these features and some of the best combinations are mentioned in
Table 1. This procedure is detailed below.
The distance between each test sample and its nearest training
samples is assumed to be the score of that test sample in the
corresponding classifier. These scores are normalised by min–max
normalisation [37] method as follows:
x − min(x)
x′ = (1)
max x − min x

IET Comput. Vis., 2018, Vol. 12 Iss. 5, pp. 679-685 683

© The Institution of Engineering and Technology 2018
Fig. 6  Confusion matrix
(a) Fine-tuned VGG-16, (b) Score-level fusion of fine-tuned VGG-16 and KFA
where x is the raw score, max x and min x are the maximum and
minimum values of the raw scores, respectively and x′ is the
normalised score.
The multimodal score vector x1, x2 can be constructed after
score normalisation, with x1 and x2 corresponding to the
normalised scores of two different systems. The next step is fusion
at the matching score level. The score vector is combined by sum
rule-based fusion method [37] to generate a single scalar score
which is then used to make the final decision as follows:
f s = w1 x1 + w2 x2
The notation wi stands for the weight which is assigned to one of
the two systems and reflects the relative importance of the two
systems. In Table 1, w1 is the weight of the first mentioned method
in the (method1 + method2) syntax. We used grid-search algorithm
to find the optimal value for w1 and w2 by giving different values
between 0, 1 and w1 = 1 − w2, for all the experiments with score-
level fusion method. The optimal values for the proposed method
are w1 = 0.3 and w2 = 0.7 which show the importance of CNN
system over KFA method.
The experimental results show that in all the cases, the score-
level fusion causes meaningful improvement in accuracy. Table 1
shows the classification accuracy of different feature descriptor
methods individually and their score-level fusion results on LHI-
Animal-Faces dataset. As shown in the table, the proposed method,
684
that uses score-level fusion with FC7 activation feature of fine-
tuned pre-trained VGG-16 and KFA, achieves 95.31%
classification rate which is better than all the other methods
presented in that table. Therefore, it can be stated that the proposed
method outperforms the other local and appearance-based methods
and all the possible combination pairs of these methods with score-
level fusion.
In order to show the classification accuracy for each class, we
computed confusion matrices for both cases of fine-tuned VGG-16
alone and score-level fusion of VGG-16 and KFA as shown in
(2) Fig. 6. The classification accuracy of seven classes is 100% and for
many other classes, it is acceptable and near to 100%. In the fine-
tuned VGG-16 confusion matrix, the maximum confusions are
caused by deer head and rabbit head versus dog head (12%). These
confusion values are reduced to 0 and 4% in the score-level fusion
of fine-tuned VGG-16 and KFA, respectively. In the proposed
method, the maximum confusions are caused by bear head versus
pigeon head and rabbit head versus mouse head (8%). In the fine-
tuned VGG-16, 11 classes have confusion with dog head class
which reduced to 4 classes in the Score-level fusion of fine-tuned
VGG-16 and KFA.
On the other hand, the proposed method is compared with the
other state-of-the-art methods that presented the results on LHI-
Animal-Faces dataset. Table 2 exhibits the classification accuracy
of our proposed method and state-of-the-art methods on LHI-
Animal-Faces. It is clearly seen that our proposed method
IET Comput. Vis., 2018, Vol. 12 Iss. 5, pp. 679-685
© The Institution of Engineering and Technology 2018
Table 2 Classification accuracy on LHI-Animal-Faces
dataset
Method Accuracy, %
HOG + SVM [13] 70.8
HIT [13] 75.6
LSVM [38] 77.6
AOT [32] 79.1
deep boosting [11] 81.5
proposed method 95.31
outperforms other state-of-the-art algorithms for animal face
classification.
6 Conclusions
A novel animal classification system is proposed in this paper. The
proposed system uses score-level fusion on two powerful feature
extractors: KFA and FC7 activation feature of VGG-16 pre-trained
on ImageNet dataset and fine-tuned on LHI-Animal-Faces dataset.
Local feature extractors, appearance-based methods, two famous
CNN architectures and several score-level fusion pairs of these
methods are used in the experiments to show the superiority of the
proposed method on LHI-Animal-Face database. Additionally, the
proposed method is compared with several state-of-the-art systems
that use the same dataset. It is demonstrated that the proposed
method using score-level fusion on appearance-based KFA method
and learned features of VGG-16 outperform the state-of-the-art
systems for animal face classification.
7 References
[1] Elson, J., Douceur, J., Howell, J., et al.: ‘Asirra: a CAPTCHA that exploits
interest-aligned manual image categorization’. Proc. ACM Conf. Computer
and Communications Security (CCS), Alexandria VA, USA, 2007, pp. 366–
374
[2] Marcialis, G., Roli, F.: ‘Score-level fusion of fingerprint and face matchers for
personal verification under stress conditions’. 14th IEEE Int. Conf. Image
Analysis and Processing ICIAP, DC, USA, 2007, pp. 259–264
[3] Elmir, Y., Elberrichi, Z., Adjoudj, R.: ‘Score-level fusion based multimodal
biometric identification (fingerprint & voice)’. 6th Int. Conf. Sciences of
Electronics, Technologies of Information and Telecommunications, Sousse,
2010, pp. 146–150
[4] Sim, H., Hishammuddin, A., Rohayanti, H., et al.: ‘Multimodal biometrics:
weighted score-level fusion based on non-ideal iris and face images’, Expert
Syst. Appl., 2014, 41, (11), pp. 5390–5404
[5] Patil, A., Bhalke, D.: ‘Fusion of fingerprint, palmprint and iris for person
identification’. Int. Conf. Automatic Control and Dynamic Optimization
Techniques (ICACDOT), Pune, 2016, pp. 960–963
[6] Takimoto, H., Mitsukura, Y., Fukumi, M., et al.: ‘Robust gender and age
estimation under varying facial pose’, Electron. Commun. Jpn., 2008, 91, (7),
pp. 32–40
[7] Schmid, C.: ‘Constructing models for content-based image retrieval’. Proc.
2001 IEEE Computer Society Conf. Computer Vision and Pattern
Recognition, 2001. CVPR 2001, Kauai, USA, December 2001, pp. 11–39
[8] Ramanan, D., Forsyth, D.A., Barnard, K.: ‘Detecting, localizing and
recovering kinematics of textured animals’. 2005 IEEE Computer Society
Conf. Computer Vision and Pattern Recognition, San Diego, USA, June 2005,
pp. 635–642
[9] Ramanan, D., Forsyth, D.A., Barnard, M.-K.: ‘Building models of animals
from video’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (8), pp. 1319–
1334
[10] Berg, T.L., Forsyth, D.A.: ‘Animals on the web’. 2006 IEEE Computer
Society Conf. Computer Vision and Pattern Recognition (CVPR'06), NY,
USA, 2006, pp. 1463–1470
[11] Penga, Z., Lia, Y., Caib, Z., et al.: ‘Deep boosting: joint feature selection and
analysis dictionary learning in hierarchy’, Neurocomputing, 2016, 178, (20),
pp. 36–45
IET Comput. Vis., 2018, Vol. 12 Iss. 5, pp. 679-685
© The Institution of Engineering and Technology 2018
[12] Afkham, H., Tavakoli, A., Eklundh, J., et al.: ‘Joint visual vocabulary for
animal classification’. Int. Conf. Pattern Recognition, 2008. ICPR 2008,
Tampa, FL, USA, 2008, pp. 1–4
[13] Si, Z., Zhu, S.-C.: ‘Learning hybrid image templates (HIT) by information
projection’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (7), pp. 1354–
1367
[14] Druzhkov, P.N., Kustikova, V.D.: ‘A survey of deep learning methods and
software tools for image classification and object detection’, Pattern
Recognit. Image Anal., 2016, 26, (1), pp. 9–15
[15] Krizhevsky, A., Sutskever, I., Hinton, G.: ‘Imagenet classification with deep
convolutional neural networks’. Advances in Neural Information Processing
Systems, Lake Tahoe, USA, 2012, pp. 1097–1105
[16] Russakovsky, O., Deng, J., Su, H., et al.: ‘Imagenet large scale visual
recognition challenge’, Int. J. Comput. Vision, 2015, 115, (3), pp. 211–252
[17] Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-
scale image recognition’, arXiv preprint arXiv:1409.1556
[18] Deng, J., Dong, W., Socher, R.: ‘Imagenet: a large-scale hierarchical image
database’. Proc. IEEE Conf. Computer Vision Pattern Recognition, Miami
FL, USA, 2009, pp. 248–255
[19] Dalal, N., Triggs, B.: ‘Histograms of oriented gradients for human detection’.
Computer Vision and Pattern Recognition, San Diego, CA, USA, June 2005,
pp. 886–893
[20] Farmanbar, M., Toygar, Ö.: ‘Feature selection for the fusion of face and
palmprint biometrics’, Signal Image Video Process., 2016, 10, (5), pp. 951–
958
[21] Eskandari, M., Toygar, Ö.: ‘Fusion of face and iris biometrics using local and
global feature extraction methods’, Signal Image Video Process., 2014, 8, (6),
pp. 995–1006
[22] Ojala, T., Pietikäinen, M., Maenpää, T.: ‘Multiresolution gray-scale and
rotation invariant texture classification with local binary patterns’, IEEE
Trans. Pattern Anal. Mach. Intell., 2002, 24, (7), pp. 971–987
[23] Liu, L., Lao, S., Fieguth, P., et al.: ‘Median robust extended local binary
pattern for texture classification’, IEEE Trans. Image Process, 2016, 25, (3),
pp. 1368–1381
[24] Liu, L., Fieguth, P., Guo, Y., et al.: ‘Local binary features for texture
classification: taxonomy and experimental study’, Pattern Recognit., 2017,
62, pp. 135–160
[25] Liu, C.: ‘Capitalize on dimensionality increasing techniques for improving
face recognition grand challenge performance’, IEEE Trans. Pattern Anal.
Mach. Intell., 2006, 28, (5), pp. 725–737
[26] Riesenhuber, M., Poggio, T.: ‘Hierarchical models of object recognition in
cortex’, Nat. Neurosci., 1999, 2, pp. 1019–1025
[27] Cord, M., Theriault, C., Thome, N.: ‘HMAX-S: deep scale representation for
biologically inspired image categorization’. Proc. IEEE Int. Conf. Image
Processing, Brussels, Belgium, 2011, pp. 1261–1264
[28] Thériault, C., Thome, N., Cord, M.: ‘Extended coding and pooling in the
HMAX model’, IEEE Trans. Image Process., 2013, 22, (2), pp. 764–777
[29] Bingpeng, M., Su, Y., Jurie, F.: ‘Covariance descriptor based on bio-inspired
features for person re-identification and face verification’, Image Vision
Comput., 2014, 32, (6-7), pp. 379–390
[30] Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: ‘Eigenfaces vs. fisherfaces:
recognition using cla ss specific linear projection’, IEEE Trans. Pattern Anal.
Mach. Intell., 1997, 19, pp. 711–720
[31] Eskandari, M., Toygar, Ö.: ‘Selection of optimized features and weights on
face-iris fusion using distance images’, Comput. Vis. Image Underst., 2015,
137, pp. 63–75
[32] Si, Z., Zhu, S.-C.: ‘Learning and-or templates for object recognition and
detection’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (9), pp. 2189–
2205
[33] Guo, Z., Zhang, D., Zhang, D.: ‘A completed modeling of local binary pattern
operator for texture classification’, IEEE Trans. Image Process., 2010, 19, (6),
pp. 1657–1663
[34] Ahonen, T., Matas, J., He, C., et al.: ‘Rotation invariant image description
with local binary pattern histogram Fourier features’, Lect. Notes Comput.
Sci., 2009, 5575, pp. 61–70
[35] Donahue, J., Jia, Y., Vinyals, O., et al.: ‘DeCAF: a deep convolutional
activation feature for generic visual recognition’. Int. Conf. Machine Learning
(ICML), Beijing, China, 2014, pp. 647–655
[36] Nanni, L., Ghidoni, S., Brahnamb, S.: ‘Handcrafted vs. non-handcrafted
features for computer vision classification’, Pattern Recognit., 2017, 71, pp.
158–172
[37] He, M., Horng, S., Fan, F., et al.: ‘Performance evaluation of score level
fusion in multimodal’, Biometric Syst. Pattern Recognit., 2010, 43, pp. 1789–
1800
[38] Felzenszwalb, P., Girshick, R., McAllester, D., et al.: ‘Object detection with
discriminatively trained part-based models’, IEEE Trans. Pattern Anal. Mach.
Intell., 2010, 32, (9), pp. 1627–1645
685