Environmental Science and Pollution Research (2020) 27:39619–39634

https://doi.org/10.1007/s11356-020-09950-3

RESEARCH ARTICLE

Intelligent animal detection system using sparse multi

discriminative-neural network (SMD-NN) to mitigate
animal-vehicle collision
S Divya Meena 1 & Agilandeeswari Loganathan 1

Received: 11 February 2020 / Accepted: 29 June 2020 / Published online: 10 July 2020
# Springer-Verlag GmbH Germany, part of Springer Nature 2020

Abstract
Animal-Vehicle Collision (AVC) is a predominant problem in both urban and rural roads and highways. Detecting animals on the
road is challenging due to factors like the fast movement of both animals and vehicles, highly cluttered environmental settings, noisy
images, and occluded animals. Deep learning has been widely used for animal applications. However, they require large training
data; henceforth, the dimensionality increases, leading to a complex model. In this paper, we present an animal detection system for
mitigating AVC. The proposed system integrates sparse representation and deep features optimized with FixResNeXt. The deep
features extracted from candidate parts of the animals are represented in a sparse form using a feature-efficient learning algorithm
called Sparse Network of Winnows (SNoW). The experimental results prove that the proposed system is invariant to the viewpoint,
partial occlusion, and illumination. On the benchmark datasets, the proposed system has achieved an average accuracy of 98.5%.

Keywords Animal-vehicle collision . Animal detection . Deep features . Sparse representation . Sparse Network of Winnows

Introduction
Animal-vehicle collision (AVC) signifies the mounting issues
in terms of conservation, safety to both animals and humans,
as well as the financial apprehensions across India, specifical-
ly the Southern zone. Both animals and humans are crashing
together on the roads, leading to loss of life. All these crashes
are due to the rapid depletion of the forest, which is being used
for the various human causes like urban sprawl, expanding the
infrastructure for transportation, etc. Most instances of AVCs
are by far reported in the rural settings. However, with the on-
going rapid destruction of the forest, sooner, both the urban
and suburban landscapes will be occupied. According to a
report (Bíl et al. 2019), by 2050, urban areas alone will be
holding approximately 6.3 billion people. Such engorgement
of cities will have a higher traffic rate with unprecedented road
use, and this will have a higher number of accidents involving
Responsible editor: Philippe Garrigues
* Agilandeeswari Loganathan
agila.l@vit.ac.in
1
School of Information and Technology, Vellore Institute of
Technology, Vellore, India
AVC. However, only very few cities are equipped with facil-
ities to mitigate AVC. According to Society for Prevention of
Cruelty to Animals (SPCA) (Sharma and Shah, 2016), ap-
proximately 500 cattle were admitted in the past 3 years, and
90% of the cases were due AVC.
Animal detection systems are mostly developed for real-time
animal monitoring applications like human-animal conflict
(Divya and Agilandeeswari 2019), animal-vehicle
collision (Meena and Agilandeeswari 2020), animal monitoring
systems (Divya Meena and Agilandeeswari 2020a, b), wildlife
population estimation (Divya Meena and Agilandeeswari
2020a, b), and much more. However, a large portion of animal
detection problem is yet to be solved. The biggest challenge with
animal detection systems is the physical characteristics of ani-
mals like their shape, size, color, and the unpredictable behavior
of the animals leading to pose variation, camouflage, and occlu-
sion. Detecting animals on roads is more complex than in an
image or video, as both the animal and the vehicle are in motion.
However, the base of any animal detection system is detecting
the animals from images or videos. In this work, we describe an
approach for detecting animals on roads and highways to miti-
gate the AVC. Detecting animals on the roads or highway is
challenging due to the noise and cluttered environmental set-
tings. Furthermore, animals may be occluded by vehicles or
by signboards. Hence, the right representation of the problem
39620
is essential to find an optimal solution. Based on this belief, we
propose a sparse representation for the deep features to support a
computationally inexpensive animal detection system.
According to Fabre-Thorpe et al. (2001), human eyes get
exhausted easily and require a lot of rest to focus. However,
without a proper system for AVC avoidance, human has to act
swiftly to prevent a collision with animals which appears unex-
pectedly. Few works on AVC (Burghardt and Ćalić 2006) de-
tects animals on the road, but it requires the animals to look
forward towards the camera, which is not feasible in most of
the cases. A similar system was developed by Viola and Jones
(2004), where the authors detect animals with their faces.
However, since the animal can appear from different directions,
detecting animals by their face is hard and impractical. The an-
imal movement has been considered as one of the prime factors
to detect animals (Walther et al. 2004). The idea is to subtract the
background and keep the remaining blobs as foreground and
detect animals from that. However, background subtraction is
difficult in both highways and roadsides. Threshold segmenta-
tion (Nascimento and Marques 2006) has also been used to
subtract background; however, deciding on the appropriate
threshold value is difficult in moving backgrounds, and this
may lead to false positives. Ramanan et al. (2006) detected an-
imals using texture and SIFT features. However, this approach is
applicable only in the case where it contains only a single animal
and a very minimal background clutter.
One of the earliest animal-vehicle collision systems was
developed with thermal images by Zhou et al. (2009). The
authors specifically focused on deer-vehicle collision (DVC)
through deer detection with pattern recognition and matching
techniques. The detailed patterns of deer were stored in an
array format, and this constituted their database. Using
pattern matching techniques, the animals were detected.
Ragab et al. (2011) proposed a real-time camel vehicle colli-
sion avoidance system with GPS support. The system consists
of two sub-systems, namely the animal detection and warning
system. With the programmable GPU device, the movement of
the camel on the roads can be detected, and the driver is alerted.
This is one of the first intelligent transport system (ITS).
Despite the huge cost involved, the system has large false-
negative cases due to the delay in receiving the SMS alert.
Mammeri et al. (2014) proposed a two-stage animal detec-
tion system for avoiding animal-vehicle collisions using LBP
features and AdaBoost classifier. The RoI from the first stage
is sent to the second stage, which uses a histogram of oriented
gradient (HOG) features and SVM classifier for classifying
the RoI as animal or non-animal. Jaskó et al. (2017) also
proposed a novel animal detection system using monocular
color vision images for avoiding animal-vehicle collisions in
traffic scenarios. Based on the intensity, color, and orientation
features, a saliency map is generated to detect the animal from
the heavy traffic scenes. The salient region is considered as the
RoI, and it is fed to the SVM classifier for final classification.
Environ Sci Pollut Res (2020) 27:39619–39634
On a dataset of animals that show their side pose, the model
achieved an accuracy of 90%. Despite the good results, the
model is incapable of detecting animals from their front view.
Sharma and Shah (2016) proposed a real-time animal detec-
tion system for avoiding animal-vehicle collisions using com-
puter vision techniques. The authors used a series of pre-
processing techniques, followed by HOG-based features ex-
traction and finally boosted cascade classifier for the classifica-
tion. The method to calculate the distance of the animal from
the vehicle is also proposed. One of the practical animal-vehicle
collision systems was proposed by Forslund and Bjärkefur
(2014). The authors proposed a vehicular animal detection sys-
tem named Autoliv that is now practically used in cars like
Audi, BMW, and Daimler. The classification system based on
cascade boosting is claimed to be robust to occlusion and pose
variation. The model can detect animals as far as 200 m away
from the car and produces less than one false positive a year.
Biological vision theory attempts to solve object detection by
decomposing the objects into individual parts and establishing a
relation over the parts. Based on this theory, we describe a sparse
image representation for the deep features that are extracted from
the candidate parts and for which the spatial relationships are
established. Sparse representation is gaining attention and has
already been applied to several applications like car detection
(Agarwal and Roth 2002), hyperspectral imaging (Chen et al.
2011). It can identify the essential patterns of images and can
work even on noisy or occluded images. Few image processing
applications like face recognition, image restoration has been
developed using the sparse representation. We have employed
sparse representation for class-specific animal detection based on
candidate feature extraction that learns to detect the animals ac-
curately, even with minimal candidate features.
While traditionally objects were represented using low-level
features such as edges, bars, and other primitive structures, we
believe that the candidate part-based representation produces
information-rich and meaningful features. In our proposed sys-
tem, we first localize the target animals and identify the candidate
parts using YOLO. We then extract small patches around the
candidate parts and construct a very large dictionary. The large
feature space eliminates the viewpoint-variations. We first extract
deep features and characterize them as higher-level sparse repre-
sentation. The spatial relationship among the parts is learned with
the sparse representation, which is otherwise not possible with
other forms like pixel-based representation.
Contribution
The contribution of the proposed system is as follows:
(i) To develop an efficient and computationally inexpensive
animal detection system for mitigating the animal-vehicle
collision.
Environ Sci Pollut Res (2020) 27:39619–39634
(ii) Identifying the candidate parts with CNN and extracting
their features instead of manually extracting them with
hand-crafted feature extraction techniques.
(iii) Representing the images as parts along with the
spatial relationships. This higher-level feature rep-
resentation is more robust and efficient than a
low-level representation.
(iv) Representing the deep feature space in a sparse
representation to reduce the complexity and com-
putation cost.
The paper is structured as follows. Section 2 describes the
proposed methodology. The experimental framework is pre-
sented in Section 3 where we discuss the dataset, performance
metrics, and the experimental setup. Experimental results are
discussed in Section 4. Sections 5 conclude with the summary
and future direction of the work.
Proposed methodology
In this section, we present the proposed system in detail. First,
we pre-process the images using FixResNeXt to resolve the
resolution discrepancy between the training and testing data.
Then we extract the candidate features through target locali-
zation, where we highlight the salient region and identify the
candidate sub-regions. The distinct representative parts are
identified, and a dictionary is constructed with those parts,
and the spatial relationship between the parts is also studied.
Finally, we learn a sparse classifier to detect and identify the
target animals from the given image. The overall flow of the
proposed system is depicted in Fig. 1.
Pre-processing with FixResNeXt
Image resolution discrepancy between the train and test data
affects the detection performance. Recently, Touvron et al.
(2019) proposed FixResNeXt based on ResNet to resolve
the resolution discrepancies. As the name suggests, it is a
typical method that fixes the train-test resolution discrepancy
and thereby improves the detection performance. The FixRes
network takes input images of any resolution and normalizes
them with the mean and standard deviation value that varies
based on the image size. For our application, we have a mean
value of [0.485, 0.456, and 0.406], and the standard deviation
is [0.229, 0.224, and 0.225]. To avoid over-fitting and also to
generalize the model, we perform data augmentation during
the training. Typical augmentations techniques used are
cropping at random sizes, flipping horizontally, and color jit-
ter. By combining multiple data augmentation techniques, we
improve test accuracy. As a fast rule, we adopt the same pa-
rameters of FixResNeXt, except for the learning rate and the
number of epochs. We set the learning rate to be 0.003.
39621
Region selection and activation statistics
Generally, CNN requires input images to be pre-processed
before it is fed to the network. The primary step in pre-
processing is region selection. The required region or the area
of interest is selected and extracted from the whole image,
after which we resize them based on the chosen network. By
and large, CNN is invariant to translation changes but scale-
invariant. FixResNeXt is proposed based on this idea.
Consequently, resizing the images will have an apparent effect
on network activations. Figure 2 illustrates how FixResNeXt
resolves the resolution discrepancy between the train and test
data.
Initially, the region is selected from the input image both
during training and testing. The red color bounding box is
resampled as a cropped image to be fed to the neural network.
When the objects in the image are of the same size, then the
cropped image will be larger at the training time than at the test
time. This will lead to a train-test resolution discrepancy. To
overcome this, we either increase the test resolution (3rd and
4th column) or decrease the train data resolution. This will
make the cropped image to have the same resolution during
both train and test time. Furthermore, this leads to scale in-
variance, which is one of the essential requirements for the
neural network. The above computation is computationally
very cheap, besides being efficient.
The cumulative distribution of the activation function is
measured after the pooling layer. The values are all positive,
as we apply the rectified linear unit (ReLU) activation func-
tion. For different train-test resolutions, we have different ac-
tivation maps. For instance, for the train-test resolution of
224 × 224 pixels, the activation map is 7 × 7, and for the res-
olution of 448 × 448, the activation map is 14 × 14. In Fig. 3,
we plotted the cumulative distribution function (CDF) of the
activation function on the output of the pooling layer.
Target localization and candidate feature extraction
To achieve meaningful representation of the animal, the candi-
date features or parts of the animals has to be captured. Such
features should also capture the inter-class variance with other
animal classes. Automating the extraction of candidate parts is
the best approach for a real-time animal-vehicle collision avoid-
ance system. In one of our previous works (Meena and
Agilandeeswari 2019), we proposed a multi-part CNN–based
system that automatically identifies and extracts the discrimina-
tive features of the animals. In (Divya Meena and
Agilandeeswari 2020a), we extended the multi-part system to
handle the real-time unlabeled data by employing a supervised
clustering technique over the classified data. In this work, we
propose yet another technique for extracting the candidate fea-
tures of the animals. Our method for automatic candidate feature
extraction is based on You Only Look Once (YOLO v1)
39622
Fig. 1 Proposed animal detection framework
network by Redmon et al. (2016). The YOLO is a variant of
single-shot detectors (SSD) that directly outputs a fixed-length
regression output for a given fixed-sized input image without the
need for a separate region proposal network (RPN). The goal of
using YOLO is to perform bounding box regression and animal
classification around all target objects and obtain a result that
includes a collection of image sub-regions that can be cropped
Fig. 2 Illustration of FixResNeXt for resolving train-test resolution discrepancy
Environ Sci Pollut Res (2020) 27:39619–39634
into a candidate feature set. The YOLO takes input images of
size 448 × 448 and outputs bounding boxes for each of the target
objects along with the label and classification score. We essen-
tially treat these localizations as salient target detections and use
their sub-regions as candidate features for further processing. The
motivation for using the target localization is that we want to
process relevant sub-regions of the targets and also to eliminate
Environ Sci Pollut Res (2020) 27:39619–39634
Fig. 3 CDF of the activation function on the output of the pooling layer, on testing for different test resolutions. Left: 128 pixels, center: 224 pixels, and
right: 448 pixels
the distracting pixel information and thereby reduce the compu-
tation complexity of animal identification.
Sample results for target localization and candidate feature
extraction are presented in Fig. 4. The ground truth is
marked in the green color bounding box, and the red color
bounding box is the result of target object localization. The
region within the red bounding box is treated as a salient
region. The target animal in the green bounding box has
positive patches (blue color) and true negative patches (red
color). The boxes in yellow color are not considered as they
represent non-target objects. The blue patches are considered
as the candidate features since they represent the salient re-
gion of the target object.
From the candidate parts, we extract small patches and
construct a large dictionary. By constructing a large dictionary
from all the target objects, we will be able to handle new
unlabeled data. That is, the new object class can be represent-
ed from the subset of this large dictionary. This approach is
very efficient since most of the animals that come on the road
are quadruped and follow the same general structure. Thus,
the subset of the dictionary can represent any new animal
classes. In our experiments, we used 100 samples from each
animal class to build the dictionary. As seen in the above
figure, most of the extracted parts are visually very similar to
one another. So, we cluster similar parts to individual groups.
We follow the bottom-up clustering approach, where each part
from the dictionary is initially put in a distinct cluster. A sim-
ilarity measure has been used to verify the similarity between
two clusters. On this basis, similar clusters are grouped until
Fig. 4 Illustration of a target object localization, b candidate feature extraction
39623
no similar cluster remains alone. The similarity measure is
defined as;
∑p ∈C ∑p ∈C S ðp1 ; p2 Þ
S ðC 1 ; C 2 Þ ¼ 1 1 2 2 ð1Þ
jC 1 j  jC 2 j
The above equation calculates the average similarity be-
tween the distinct parts. A cluster that represents a single part
is considered as one conceptual part. Since more than one
cluster represents a single part, we were able to obtain a
higher-level abstract representation of the parts from the re-
dundant representation. Having constructed the candidate part
dictionary, we now represent the images using the dictionary
parts. Based on the similarity measure, we determine which all
parts in the input image are present in the dictionary. Finally,
each of the images is represented as a feature vector based on
the dictionary parts. To search for dictionary parts in the given
image it is computationally quite expensive. Hence, we detect
the candidate features and highlight the region around the
parts. The patches extracted from the highlighted region are
then compared with the dictionary parts based on the similar-
ity measure used for the clustering. If sufficiently similar dic-
tionary art is found, the patch in the image is represented by
the feature id corresponding to that dictionary part.
The spatial relationships among the candidate parts are de-
fined in terms of distance and direction between each pair of
parts. Instead of using a geometric alignments based tech-
nique, we try to learn a model that best represents the relation-
ship among the candidate parts. Basically, we discretize the
distance and direction between the arts into bins. The distance
39624
is defined relative to the size of the candidate parts and is
discretized into 5 bins. Similarly, the direction is discretized
into 4 different ranges, each covering an angle of 45° and thus
covering an entire animal of 360°. Thus, we get 20 (5 × 4 bins)
possible distance-direction relations between any two candi-
date parts. Since the number of active candidate parts is min-
imal, the computation cost of the distance-direction relation is
also negligible.
Learning a sparse classifier
The training images are converted into a feature vector, and
the number of features for any given image will be very large.
Adding to this is the 20 possible relations between the candi-
date parts. However, we focus only on the candidate features,
and so the number of active features will be very small.
Utilizing this sparseness property, we train our classifier using
the Sparse Network of Winnows (SNoW) learning architec-
ture (Carlson et al. 1999). SNoW is a sparse network with
linear units over a pre-defined learned feature space. The input
nodes are used as the input features, and the linear units are the
target nodes that correspond to the number of animal classes
we have in the dataset. Given a set of candidate input features,
each input image is mapped onto a set of features that are
active in it. This feature representation is fed to the input layer
and propagates to the target nodes that are connected via the
weighted edges. The active set of features is represented
by At = {i1, i2, …, in}, where t is the target node. The active
Fig. 5 Sample images from dataset a AoW, b COCO, and c AwA dataset
Environ Sci Pollut Res (2020) 27:39619–39634
state is then represented by ∑i∈A wti > θt , where wti is the
weighted edge connecting target node t and ith feature and θt
is the threshold value. Each target class is represented by a
sub-network. So, a given input image is treated autonomously
by the corresponding target sub-network. Separate feature
vectors are learned for each of the animal classes, and so a
distinct function is learned from the shared feature
space, for each of the target classes. When a new input
feature vector is given, the so-far learned functions will
output an activation value using the sigmoid activation
function. The class with the highest activation value
will be the appropriate class for that given input vector.
Basically, the model follows the winner-take-all-decision
approach. The activation level–based conclusion pro-
vides a robust confidence score, and we identify the
animal based on this confidence score in the final stage.
The learning policy is mistake-driven, and several up-
date rules can be used in SNoW. In this application, we
used the most successful update rule, the Littlestone’s
Winnow update rule. It is a multiplicative update rule
where the input features are not known prior. That is,
(1) input features are allocated in a data-driven way—an
input node for the feature i is allocated only if the feature
i is active in the input image and (2) a link (Le., a non-
zero weight) exists between a target node t and a feature i
if and only if i has been active in an image labeled t.
Thus, the architecture also supports augmenting the feature
types at later stages or from external sources in a flexible
way, an option we do not use in the current work.
Environ Sci Pollut Res (2020) 27:39619–39634 39625

Table 1 Effects of different proportion of dataset splits

Dataset Dataset proportion Accuracy (%) Specificity (%) Sensitivity (%) F-measure (%)

Train (%) Test (%)

COCO animals 10 90 88.7 95.6 70.9 83.0

20 80 90.0 98.3 72.0 87.0
30 70 93.1 100.1 77.9 88.0
40 60 97.8 96.4 78.7 82.0
60 40 97.0 99.2 81.5 83.0
Animals on the Web 10 90 87.9 99.3 81.9 82.0
20 80 90.3 98.3 82.9 93.0
30 70 92.8 91.1 87.0 93.0
40 60 96.2 85.9 86.5 95.0
60 40 97.4 89.3 87.9 97.2
Animals with Attributes 10 90 90.3 92.2 89.0 89.7
20 80 94.6 96.7 92.8 92.0
30 70 96.1 98.0 95.5 96.1
40 60 98.8 99.4 98.4 98.8
60 40 98.9 99.6 98.5 99.0

Implementing the Littlestone’s Winnow update rule via a
sparse network follow the two steps:
1) The input node is allocated with a feature i, only if it is
active in the given input image.
2) Link is established between a target node t and a feature i
if i is active in an image.
The key property of the Winnow update rule is the number
of examples it requires to learn a linear function that grows
linearly with the number of candidate features. This property
is crucial in applications where the number of features is vast,
but a relatively small number of them is sufficient. Winnow
efficiently learns any linear threshold function and is robust to
various kinds of noises. Once target sub-networks complete
learning, the winner-take-all mechanism selects the dominant
active target node in the SNoW unit to produce the final
prediction.
dataset, the COCO animal, was released by researchers of
Stanford University (Patterson and Hays 2016). It is a subset
of the COCO dataset and has 8 classes of animals, including
bear, bird, cat, dog, giraffe, horse, sheep, and zebra. The dataset
has a total of 800 training images and 200 test images. The test
images are of different resolutions and contain partially occlud-
ed animals in a highly cluttered background. The last dataset is
AwA, a large-scale dataset with 50 different animal classes and
has over 30,000 images. The dataset was released by Lampert
et al. (2009) and has images that are mostly taken on roads.
Sample images for each dataset are depicted in Fig. 5.
Performance metrics
The accuracy of the model is measured with two quantitative
metrics, namely positive detection rate and false detection
rate. The positive detection rate is the number of correct

100
99
Experimental framework 98
97
Accuracy

96
Dataset description
95
94
The proposed approach has been tested on three benchmark 93
datasets for animals, namely, Animals on the web (AoW), 92
COCO animals, and Animals with Attributes (AwA) datasets. 91
Imbalanced Balanced Imbalanced Balanced Imbalanced Balanced
The first dataset, Animals on the web, was released by Berg COCO Animals Animals on the Web Animals with Attributes
and Forsyth (2006). The dataset has a total of 14,051 images of
Accuracy
10 different animal classes like an alligator, giraffe, penguin,
etc. Each image is of at least 120 × 120 pixels. The second Fig. 6 Performance of balanced vs. imbalanced dataset
39626 Environ Sci Pollut Res (2020) 27:39619–39634

Table 2 Performance of the

proposed system on “COCO Class Accuracy Precision Recall Specificity F- FPR FNR
Animal” dataset measure

Bear 0.970 0.883 0.978 0.469 0.981 0.531 0.022

Bird 0.956 0.851 0.971 0.3 0.978 0.7 0.029
Cat 0.971 0.88 0.977 0.455 0.981 0.545 0.023
Dog 0.973 0.927 0.987 0.602 0.986 0.398 0.013
Giraffe 0.974 0.934 0.989 0.618 0.987 0.382 0.011
Horse 0.966 0.898 0.981 0.524 0.983 0.476 0.019
Sheep 0.971 0.917 0.985 0.579 0.985 0.421 0.015
Zebra 0.97 0.915 0.985 0.572 0.984 0.428 0.015

Average accuracy: 0.97

Average precision: 0.90
Average sensitivity: 0.98
Average specificity: 0.51

detection relative to the total positive sample. Similarly, the image that is positive (correctly classified). The recall
false detection rate is the number of incorrect detections with is the total percentage of positive images obtained.
respect to total negative samples. Precision is also called a positive predictive value
ðTrue positive þ True negativeÞ (PPV). The recall is also called sensitivity. Besides,
Accuracy ¼
True positive þ True negative þ False positive þ False negative we also have specificity and sensitivity metrics.
ð2Þ Number of true positives
Precision ¼ ¼ PPV
Number of true positives Number of true positives þ Number of false positives
Positive detection rate ¼ ð3Þ
Total number of positive samples ð5Þ
Number of false positives Number of true positives
False detection rate ¼ ð4Þ Recall ¼ ¼ Positive Detection Rate ð6Þ
Total number of negative samples Total number of positive samples
Number of true positives
The trade-off between the correct and incorrect detec- Sensitivity ¼ ð7Þ
Number of true positives þ Number of false negatives
tions can be appropriately expressed with a ROC curve Number of true negatives
(receiver operating characteristics). It plots the correct Specificity ¼ ð8Þ
Number of true negatives þ Number of false positives
detection vs. incorrect detections. One other important
metric in the detection problem is the precision-recall F1 measure is a weighted average value of precision
curve. Precision corresponds to the percentage of the and recall. It is a balance between the two. This is more

Table 3 Performance of animal

detection system on “Animals on Class Accuracy Precision Recall Specificity F- FPR FNR
the Web” dataset measure

Alligator 0.975 0.939 0.99 0.626 0.987 0.374 0.01

Ant 0.912 0.923 0.986 0.593 0.985 0.407 0.014
Bear 0.978 0.95 0.992 0.648 0.988 0.352 0.008
Beaver 0.977 0.948 0.992 0.644 0.988 0.356 0.008
Dolphin 0.986 0.923 0.972 0.469 0.981 0.428 0.019
Frog 0.981 0.898 0.966 0.3 0.986 0.374 0.015
Giraffe 0.985 0.917 0.971 0.455 0.987 0.407 0.022
Leopard 0.981 0.95 0.981 0.644 0.981 0.428 0.023
Monkey 0.986 0.948 0.985 0.572 0.986 0.374 0.013
Penguin 0.987 0.915 0.978 0.626 0.987 0.407 0.011

Average accuracy: 0.98

Average precision: 0.93
Average sensitivity: 0.98
Average specificity: 0.56
Environ Sci Pollut Res (2020) 27:39619–39634 39627

Has both cluster and relation No cluster but has relations 1 Layer CNN 2 Layer CNN 3 Layer CNN

No relations but has clusering No clustering and relations 4 Layer CNN 5 Layer CNN
1
1.2
0.9
1 0.8
0.7
0.8
Recall

Accuracy
0.6
0.6 0.5
0.4
0.4 0.3
0.2 0.2
0.1
0 0
0 0.1 0.2 0.3 0.4 0.5 0 10 20 30 40 50 60 70 80 90 100

Precision Number of iterations

Fig. 7 Effects of clustering and spatial relationship on the overall Fig. 9 Effects of CNN depths on COCO animals dataset
performance
376.82 GPU workstation with Intel(R) Xeon® W-2102 CPU
important than accuracy when the classes are uneven, @ 2.90GHz processor and 128GB RAM. For storage pur-
and this is the case most of the time. It is given by; poses, we used 1 TB Seagate Hard disk drive. The target
  localizer is trained using a Python wrapper around the original
precision*recall
F1 measure ¼ 2* ð9Þ open-source implementation. Jupyter Notebook IDE was used
precision þ recall
to build and visualize the systems. Pycharm was used to pre-
Apart from these, we have type I and type II error. FPR and process the data. Keras-GPU was used to build a neural net-
FNR are given by; work and tune the hyper-parameters.

FP
α¼ ð10Þ
ðFP þ TN Þ
Experiments and results
FN
β¼ ð11Þ
ðTP þ FN Þ In this section, we discuss the results of various experiments.
The experiments have covered the broad aspects of dataset
Type I error defines the false positive rate and is sometimes
slitting and data balancing, sparse representation, depth in
called Alpha. It is the inverse of specificity. The false-negative
CNN, and its features. We also present an ablation study of
rate belongs to the type II error, and it is the inverse of
the proposed system. Finally, a comparative study is
sensitivity.
presented.

Experimental setup Dataset splitting and data balancing

The experiments were carried out in Windows 10 (64 bit) The proportion of train and test data plays an important role in
environment in a single NVIDIA GeForce 940 M Version computational time and complexity, besides influencing the
performance. Similarly, data balancing is an essential step in
1

0.9 100
0.8

0.7 95
0.6
90
Accuracy

0.5
Accuracy

0.4 85
1 Layer CNN
0.3
2 Layer CNN
0.2
80
3 Layer CNN

0.1 4 Layer CNN

5 Layer CNN
75
0 10 20 30 40 50 60 70 80 90 100
0 10 20 30 40 50 60 70 80 90 100

Number of iterations Number of deep features for each layer

Fig. 8 Effects of CNN depths on Animals on the web dataset Fig. 10 Effects of number of features for each layer of the CNN
39628 Environ Sci Pollut Res (2020) 27:39619–39634

100
combining multiple data augmentation techniques, we improved
99 the quality of augmented data, and therefore, the model did not
98 overfit. There exists a trade-off in the performance metrics for a
97 balanced and imbalanced dataset. For a balanced dataset, accu-
96 racy is the best metric. On the other hand, for an imbalanced
Accuracy

95 AWA+Deep dataset, precision and recall are the appropriate metrics.

94 AWA+Sparse
93 COCO+Deep
COCO+Sparse
92
91
0 100 200 300 400 500
Dictionary size
Fig. 11 Analyses of deep vs. sparse representation on the two datasets
training. The effects of different proportions of training data
on the accuracy of the proposed system are evaluated, and the
result is presented in Table 1.
The proposed model is validated against a different propor-
tion of training data, like 10%, 20%, 30%, 40%, and 60%. The
system has achieved good accuracy, even with 10% training
data. Nevertheless, the accuracy increases with an increase in
the proportion of training data. However, the accuracy starts
saturating at around 40% of training data, and further on, it
does not have much impact on the performance. Nevertheless,
increasing the size of training data increases the computational
cost and complexity. Hence, the proposed system is trained
with 40% training data. Thus, the following experiments are
trialed with 40:40:20 proportions, where 40% is for training
and testing, and the remaining 20% is for validation. The
effect of a balanced vs. imbalanced dataset is studied, and
the result is presented in Fig. 6. Following the previous exper-
iment, the dataset proportion is maintained at 40:40:20 on all
the benchmark datasets.
From the figure, it is clear that an imbalanced dataset hinders
the performance. Generally, it is good to have a balanced dataset,
as an imbalanced dataset will have a bias favoring the majority
class. Usually, the problem of an imbalanced dataset is predom-
inant in multi-class problem, where more than one class may
have minimal data. Data augmentation was done to balance the
dataset with typical augmentations techniques like cropping at
random sizes, flipping horizontally, and color jitter. By
Performance analysis on the benchmark datasets
600 We analyzed the performance of the proposed system on two
standard datasets with the performance metrics discussed in
Section 3.2. The results are presented in Table 2 and Table 3.
From the table, it is clear that giraffe has the highest accuracy
of 97.4%, and birds have the least accuracy of 95.6%. The model
has a good recall rate of 98% and a precision of 90%.
Animals on the Web dataset have marginally higher per-
formance than the COCO dataset. Among the 10 classes, pen-
guin has the highest accuracy of 98.7%, and ants have the least
accuracy of 91.2%. The test images in this dataset were diffi-
cult than the COCO animal dataset. The images were mostly
submerged in the background, which was already cluttered.
Nevertheless, the model has achieved a recall of 98% and a
precision of 93%, which is higher than the previous dataset.
Effects of clustering and relations
Candidate parts are identified, and the patches around the
candidate parts are extracted, which was then used to construct
a dictionary. The patches are clustered based on the similarity.
Besides, the spatial relationships among the parts are also
considered. In this experiment, we analyze the effects of clus-
tering and spatial relationship.
In this experiment, we considered 4 different combinations
of clusters and relationships. The first set considered both the
clustering as well as spatial relationship. The second and third
sets considered only the spatial relationship and clustering,
respectively. To highlight the importance of both clustering
and spatial relationship, we considered the fourth set that has
neither clustering nor relations. It is evident from the figure
that both the clustering and spatial relationship has significant
importance on the overall performance of the system. When

Table 4 Ablation study on the

proposed system Dataset Representation Accuracy (%) Throughput Training Testing time (sec)
time (sec)

Animals on the web Deep 85.42 431 167 138

Sparse 98.67 486 115 97
COCO animals Deep 83.84 409 143 121
Sparse 97.14 476 90 78
Animals with attributes Deep 84.84 429 173 134
Sparse 98.50 462 102 87
Environ Sci Pollut Res (2020) 27:39619–39634
Fig. 12 Sample results for correct
detection
compared with pure clustering models (Agarwal and Roth
2002), spatial relations among the parts have better effects
on detecting the animals. Moreover, it is evident from Fig. 7
that “no cluster but has relations” is better in performance than
“no relations but has clustering.”
Analysis of CNN depth and features
The proposed system extracts deep features and identifies the
candidate arts Depth is one of the important factors in CNN.
More the depth of the network, the deeper the features extract-
ed. We conducted a few experiments on both the datasets to
analyze the effects of the depth of the network. We tested for
five different depths of CNN, and the number of features was
set to 100. For a different depth, the overall accuracy was
estimated. Figure 8 and Fig. 9 illustrates the CNN depth effect
on the two datasets. The features from deeper layers are more
robust and have higher representational power.
From the figures, it could be inferred that the accuracy in-
creases with the increase in depth of the network. The initial
layers can capture only the low-level features, and the level of
abstraction increases with the depth. The first dataset (AOW)
overfitted the model, and this has resulted in lower accuracy
when compared with the COCO animal dataset. The dimension
of the spatial features is governed by the number of features on
CNN. We conducted experiments on both of the benchmark
datasets to analyze the accuracy of the model for a varying num-
ber of features. For the experiment, we varied the number of
features from 10 to 100, and CNN was set to five-layer architec-
ture throughout.
Fig. 13 Sample results for incorrect detection
39629
From Fig. 10, it could be inferred that the accuracy
of the COCO animal dataset increased with the number
of features, although they were not steady. However,
the accuracy of Animals on the Web dataset is not
having any increasing trend. Beyond 80, there is no
improvement in accuracy. This shows that accuracy is
unstable beyond a certain number of features. Thus, the
number of features was set to 80.
Analysis of sparse representation
Sparse representation has been adopted to reduce the overall
complexity of the system. Deep features increase the dimen-
sion of the network, and the pooling layer tends to lose infor-
mation. In contrast, sparse representation has better learning
power and performs better with fewer features. To analyze the
performance of sparse representation, we compared it with
deep feature representation on both the datasets.
Figure 11 illustrates the relationship between the dictionary
size and the performance of the system in terms of accuracy.
The figure evaluates both the deep and sparse representation
forms on the two datasets. The accuracy is plotted as a func-
tion of training dictionary size. On both the datasets, sparse
representation has a significant improvement in accuracy
when compared with deep representation. In terms of deep
feature representation, it could be inferred that the accuracy
increases with an increase in training dictionary size. In both
the representations, the COCO animal dataset has better accu-
racy than Animals on the Web dataset.
39630
Fig. 14 Sample results for detecting animals in the roads
Ablation study on the proposed system
The ablation study is conducted for assessing the two different
representations of the proposed system. In addition to accura-
cy, the throughput measure is also considered for measuring
the number of images processed per second. While usually
FLOPS (FLoating point Operations Per Second) is used to
measure the number of images processed per second, here
throughput is considered rather since the inference speed of
FLOPS is not proportional to GPU. The result of the ablation
study is presented in Table 4. The ablation study is conducted
on a random 500 images from each dataset.
The sparse representation has significantly lower processing
time both at training and testing phases. The high dimensional
features affect the processing time, and this is evident from the
Fig. 15 Sample results for a partial occlusion, b viewpoint invariance
Environ Sci Pollut Res (2020) 27:39619–39634
time complexity of deep representation. Of all the datasets, COCO
animals have the least training and testing time. In contrast, the
AoW dataset has the highest accuracy followed by AwA.
However, there is a trade-off between throughput and accuracy.
The proposed system balanced the trade-off and has achieved
significantly higher accuracy without affecting the throughput.
Qualitative results for animal detection and
recognition
In this part, we represent the sample results for both correct
and incorrect detections. The animals are detected with a
bounding box, and the detected animals are recognized by
its class, and a confidence score is given along with the class.
Environ Sci Pollut Res (2020) 27:39619–39634
Fig. 18 Sample results for a
blurred image, b shaken image
Figure 12 is a representation of correct detection. The samples
are tested on both the dataset.
Figure 13 represents the sample results for incorrect detec-
tion. In the case of the alligator, a reflection of the animal is
also detected as a true positive. In the case of the zebra, two
instances of zebra standing very close to each other are detect-
ed as one single zebra. False positives due to overlapping or
closely-standing animals are not severe, as the application is
about detecting animals and not population estimation.
Nevertheless, at least one instance of the animal is detected;
indicating the presence of animals on the road, and thus the
driver can be alerted. Figure 14 presents sample results for
animal detection on the roads to simulate a real-time AVC
scenario.
Animal detection in challenging image conditions
In this section, we present the challenges in detecting
animals in roads and highways. Some of the prominent
challenges in detecting animals on roads include partial
occlusion, pose variation or viewpoint variation, and
cluttered background. Besides these, there are other
challenges like poor illumination and overexposure that
might arise due to lighting effects. Finally, images
might get blurred or shaken due to the fast movement
of the vehicle, and this makes it difficult to detect an-
imals from the images. The performance of the pro-
posed system on these challenging image conditions is
analyzed, and the sample results are presented in
Fig. 16 Sample results for animals in a cluttered environmental setting
39631
Fig. 15–Fig. 18. Since the dictionary is populated with
the patches around the candidate parts, the model can
detect animals even when it is partially occluded (see
Fig. 15a). Besides, with the large dictionary of candi-
date parts, the proposed system is invariant to viewpoint
variance (see Fig. 15b).
Despite the cluttered environmental setting and other
challenges, the model was able to detect each instance
of the animals correctly with a good confidence score.
Figure 16 represents a sample for animal detection in a
cluttered environment. Despite the clutter and being far
away from the camera, the model was able to detect the
animals.
The training images are usually of good quality, and the
training accuracy will be quite good when the parameters are
fine-tuned. On the other hand, the real-time test images cap-
tured on the road may not always be clear. The camera limi-
tations may cause unnecessary noise in the images like a blur,
poor illumination, and overexposure. To tackle these chal-
lenges, the proposed system pre-processes the images with
FixResNeXt, which adjusts the resolution discrepancy among
the train and test data.
As far as animal recognition is concerned, we set a thresh-
old of 55%. If the detection score is less than 55%, the model
detects the animal without labeling them. However, a
bounding box is shown to indicate the presence of an animal.
In Fig. 17, the animals are detected as deer despite the illumi-
nation problem. Figure 18 illustrates some of the challenges
due to hardware limitations.
39632 Environ Sci Pollut Res (2020) 27:39619–39634

Fig. 17 Sample results for a poor

illumination, b overexposure

Comparative study feature learning and the way of representations. In comparison

The proposed system is compared with other related systems
on benchmark Animals with Attributes (AwA) dataset
(Lampert et al. 2009). AwA, an attribute-based dataset, has
been benchmarked by several systems like learning-based sys-
tems, semantic attribute-based system, and category-based at-
tributes. Among all, Bo Zhao et al. (2019), Guo et al. (2015),
Kovashka et al. (2011), and Tian et al. (2017) were based on
learning systems. On the other hand, Al-Halah and
Stiefelhagen (2015, 2017), and Lampert et al. (2009) were
based on the semantic attributes. Finally, Yu et al. (2013),
Batra and Parikh (2017), and Al-Halah et al. (2016) were
based on category-level attributes. Steve Branson et al.
(2010) proposed an interactive human-computer system
intending to identify the true class while minimizing the num-
ber of questions asked to humans, using the visual content of
the image. Table 5 compares the existing system with the
proposed system on the benchmark AwA dataset.
In comparison with existing systems, the proposed system
has faster convergence and is significantly better than others.
The better performance is mainly attributed to the choice of
with deep representation, sparse representation has better con-
vergence. Similarly, with a higher number of features, the
system took more time to perform detection.
Conclusion and future scope
In this paper, we presented an animal detection system that
integrates sparse representation with deep features that are
automatically extracted from the candidate discriminative
parts of the animals. An efficient feature-efficient learning
algorithm called Sparse Network of Winnows is learned over
the feature space. The results demonstrate the effectiveness of
the model on detecting animals on roads. It works on chal-
lenging image conditions like partial occlusion, poor illumi-
nation, overexposure, pose variation, and cluttered back-
ground. The proposed model has been tested on several
benchmark animal datasets with varied experiments on both
deep characteristics of CNN and the effectiveness of sparse
representation.

Table 5 Comparative analysis of the proposed system with the related existing systems on benchmark AwA dataset

S.no Systems Technique Accuracy (%)

1 Bo Zhao et al. (2019) Zero-shot learning 87.29

2 Yuchen Guo et al. (2015) Latent attribute learning 65.25
3 Adriana Kovashka et al. (2011) Active learning with objects and attributes 75.01
4 Tian et al. (2017) Learning attributes from the crowdsourced relative labels 76.34
5 Al-Halah and Stiefelhagen (2017) Semantic attribute discovery 89.73
6 Al-Halah and Stiefelhagen (2015) Hierarchical transfer of semantic attributes 83.75
7 Lampert et al. (2009) Information transfer by attribute sharing 65.97
8 Yu et al. (2013) Category-level attributes 84.38
9 Batra and Parikh (2017) Attribute-category model 83.51
10 Al-Halah et al. (2016) Class-attribute associations 79.74
11 Steve Branson et al. (2010) Visual recognition with humans in the loop 87.22
12 Proposed system Candidate deep features with sparse classifier 98.50
Environ Sci Pollut Res (2020) 27:39619–39634
This work can be extended to detect animals from real-time
videos. Besides, the system can be extended to work with
thermal images, since they are the best choice to detect ani-
mals at night time. In contrast,, visible images are the best
choice to detect animals during day time. Thus, a fusion of
visible and thermal images can be used for detecting animals
on the roads.
Acknowledgments The authors thank VIT for providing "VIT SEED
GRANT” for carrying out this research work.
References
Agarwal S, Roth D (2002) Learning a sparse representation for object
detection. In: European conference on computer vision. Springer,
Berlin, pp 113–127
Al-Halah Z, Stiefelhagen R (2015) How to transfer? Zero-shot object
recognition via hierarchical transfer of semantic attributes. In 2015
IEEE Winter Conference on Applications of Computer Vision (pp.
837-843). IEEE
Al-Halah Z, Stiefelhagen R (2017) Automatic discovery, association es-
timation and learning of semantic attributes for a thousand catego-
ries. In Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition (pp. 614-623)
Al-Halah Z, Tapaswi M, Stiefelhagen R (2016) Recovering the missing
link: predicting class-attribute associations for unsupervised zero-
shot learning. In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition (pp. 5975-5984)
Batra T, Parikh D (2017) Cooperative learning with visual attributes.
arXiv preprint arXiv:1705.05512
Berg TL, Forsyth DA (2006) Animals on the web. In 2006 IEEE
Computer Society Conference on Computer Vision and Pattern
Recognition (CVPR’06) (Vol. 2, pp. 1463-1470). IEEE
Bíl M, Andrášik R, Duľa M, Sedoník J (2019) On reliable identification
of factors influencing wildlife-vehicle collisions along roads. J
Environ Manag 237:297–304
Branson S, Wah C, Schroff F, Babenko B, Welinder P, Perona P,
Belongie S (2010) Visual recognition with humans in the loop. In:
European Conference on Computer Vision. Springer, Berlin, pp
438–451
Burghardt T, Ćalić J (2006) Analysing animal behaviour in wildlife
videos using face detection and tracking. IEE Proceedings-Vision,
Image and Signal Processing, 153(3), 305–312
Carlson A, Cumby C, Rosen J, Roth D (1999) The SNoW learning
architecture (p. 24). Technical report UIUCDCS
Chen Y, Nasrabadi NM, Tran TD (2011) Hyperspectral image classifica-
tion using dictionary-based sparse representation. IEEE Trans
Geosci Remote Sens 49(10):3973–3985
Divya Meena S, Agilandeeswari L (2019) Adaboost cascade classifier for
classification and identification of wild animals using movidius neu-
ral compute stick. Int J Eng Adv Technol 9(1S3):495–499
Divya Meena S, Agilandeeswari L (2020a) A new supervised clustering
framework using multi discriminative parts and expectation–
maximization approach for a fine-grained animal breed classifica-
tion (SC-MPEM). Neural Process Lett. https://doi.org/10.1007/
s11063-020-10246-3
Divya Meena S, Agilandeeswari L (2020b) FSSCaps-DetCountNet:
fuzzy soft sets and CapsNet-based detection and counting network
for monitoring animals from aerial images. J Appl Remote Sens
14(2):026521. https://doi.org/10.1117/1.JRS.14.026521
39633
Fabre-Thorpe M, Delorme A, Marlot C, Thorpe S (2001) A limit to the
speed of processing in ultra-rapid visual categorization of novel
natural scenes. J Cogn Neurosci 13(2):171–180
Forslund D, Bjärkefur J (2014) Night vision animal detection. In 2014
IEEE Intelligent Vehicles Symposium Proceedings (pp. 737-742).
IEEE
Guo Y, Ding G, Jin X, Wang J (2015) Learning predictable and discrim-
inative attributes for visual recognition. In Twenty-Ninth AAAI
Conference on Artificial Intelligence
Jaskó G, Giosan I, Nedevschi S (2017) Animal detection from traffic
scenarios based on monocular color vision. In 2017 13th IEEE
International Conference on Intelligent Computer Communication
and Processing (ICCP) (pp. 363-368). IEEE
Kovashka A, Vijayanarasimhan S, Grauman K (2011) Actively selecting
annotations among objects and attributes. In 2011 International
Conference on Computer Vision (pp. 1403-1410). IEEE
Lampert CH, Nickisch H, Harmeling S (2009) Learning to detect unseen
object classes by between-class attribute transfer. In 2009 IEEE
Conference on Computer Vision and Pattern Recognition (pp.
951-958). IEEE
Mammeri A, Zhou D, Boukerche A, Almulla M (2014) An efficient
animal detection system for smart cars using cascaded classifiers.
In 2014 IEEE International Conference on Communications (ICC)
(pp. 1854-1859). IEEE
Meena SD, Agilandeeswari L (2019) An efficient framework for animal
breeds classification using semi-supervised learning and multi-part
convolutional neural network (MP-CNN). IEEE Access 7:151783–
151802
Meena D, Agilandeeswari L (2020) Invariant Features-Based Fuzzy
Inference System for Animal Detection and Recognition Using
Thermal Images. Int J Fuzzy Syst. https://doi.org/10.1007/s40815-
020-00907-9
Nascimento JC, Marques JS (2006) Performance evaluation of object
detection algorithms for video surveillance. IEEE Trans
Multimedia 8(4):761–774
Patterson G, Hays J (2016) Coco attributes: attributes for people, animals,
and objects. In European Conference on Computer Vision (pp. 85-
100). Springer, Cham
Ragab K, Zahrani M, Haque AU (2011) Camel-vehicle accidents mitiga-
tion system: design and survey. In: Future Information Technology.
Springer, Berlin, pp 148–158
Ramanan D, Forsyth DA, Barnard K (2006) Building models of animals
from video. IEEE Trans Pattern Anal Mach Intell 28(8):1319–1334
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once:
Unified, real-time object detection. In Proceedings of the IEEE con-
ference on computer vision and pattern recognition (pp. 779-788).
Redmon J, Farhadi A (2018) Yolov3: An incremental improvement.
arXiv preprint arXiv:1804.02767
Sharma SU, Shah DJ (2016) A practical animal detection and collision
avoidance system using computer vision technique. IEEE Access 5:
347–358
Tian T, Chen N, Zhu J (2017) Learning attributes from the crowdsourced
relative labels. In Thirty-First AAAI Conference on Artificial
Intelligence
Touvron H, Vedaldi A, Douze M, Jégou H (2019) Fixing the train-test
resolution discrepancy. In Advances in Neural Information
Processing Systems (pp. 8252-8262)
Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput
Vis 57(2):137–154
Walther D, Edgington DR, Koch C (2004) Detection and tracking of
objects in underwater video. In Proceedings of the 2004 IEEE
Computer Society Conference on Computer Vision and Pattern
Recognition, 2004. CVPR 2004. (Vol. 1, pp. I-I). IEEE
Yu FX, Cao L, Feris RS, Smith JR, Chang SF (2013) Designing category-
level attributes for discriminative visual recognition. In Proceedings
39634 Environ Sci Pollut Res (2020) 27:39619–39634

of the IEEE Conference on Computer Vision and Pattern Conference on Robotics and Biomimetics (ROBIO) (pp. 688-693).
Recognition (pp. 771-778) IEEE
Zhao B, Fu Y, Liang R, Wu J, Wang Y, Wang Y (2019) A large-scale
attribute dataset for zero-shot learning. In Proceedings of the IEEE Publisher’s note Springer Nature remains neutral with regard to jurisdic-
Conference on Computer Vision and Pattern Recognition tional claims in published maps and institutional affiliations.
Workshops (pp. 0-0)
Zhou D, Dillon M, Kwon E (2009) Tracking-based deer vehicle collision
detection using thermal imaging. In 2009 IEEE International