FAST ANIMAL DETECTION IN UAV IMAGES USING CONVOLUTIONAL NEURAL
NETWORKS
Benjamin Kellenberger, Michele Volpi, Devis Tuia
MultiModal Remote Sensing, University of Zurich (Switzerland)
{benjamin.kellenberger, michele.volpi, devis.tuia}@geo.uzh.ch
ABSTRACT
Illegal wildlife poaching poses one severe threat to the en-
vironment. Measures to stem poaching have only been with
limited success, mainly due to efforts required to keep track
of wildlife stock and animal tracking. Recent developments
in remote sensing have led to low-cost Unmanned Aerial Ve-
hicles (UAVs), facilitating quick and repeated image acqui-
sitions over vast areas. In parallel, progress in object detec-
tion in computer vision yielded unprecedented performance
improvements, partially attributable to algorithms like Con-
volutional Neural Networks (CNNs). We present an object
detection method tailored to detect large animals in UAV im-
ages. We achieve a substantial increase in precision over a
robust state-of-the-art model on a dataset acquired over the
Kuzikus wildlife reserve park in Namibia. Furthermore, our
model processes data at over 72 images per second, as op-
posed 3 for the baseline, allowing for real-time applications.
1. INTRODUCTION
In this paper we address the task of animal detection from
sub-decimeter resolution images acquired by low-cost Un-
manned Aerial Vehicles. This task is of particular interest to
live stock conservation, where accurate and cost-effective so-
lutions to animal monitoring would lead to targeted counter-
actions to poaching [1, 2]. Such actions are of paramount im-
portance, as can be seen by the increasing numbers of killed
individuals, that raised from 13 to 668 in the case of Rhinos
in South Africa in the period 2007-2012 [3] and amounted to
tens of thousands for African Elephants in 2011 alone [2].
The answer to the needs of automatic counting might be
found in UAV-based monitoring systems. They allow for fre-
quent acquisitions over large areas at sub-decimeter resolu-
tion. The task of animal detection has been traditionally car-
ried out by manual annotation, which requires trained experts
and large amounts of time. To offer a more efficient system,
we propose a pipeline performing animal (object) detection
based on Convolutional Neural Networks (CNNs) [4]. An ex-
ample of the results achieved by our system is shown in Fig. 1.
This work has been supported by the SNSF grant PP00P2 150593. The
authors would like to acknowledge the SAVMAP project and Micromappers
for providing the data and ground truth used in this work.
‹,(((
Authorized licensed use limited to: Universiti Teknikal Malaysia Melaka-UTEM. Downloaded on June 02,2023 at 04:52:10 UTC from IEEE Xplore. Restrictions apply.
’ Ground truth
’ Predictions
Fig. 1. Example result on animal detection using the proposed
model.
We demonstrate the performance of our model on an set of
UAV images of the Kuzikus wildlife reserve in Namibia1 .
2. RELATED WORK
Object detection is the task of drawing bounding boxes (lo-
calization) and identifying classes of objects (recognition) in
an image. It is one of the most investigated fields in com-
puter vision [5]. The common principle of object detection
algorithms has hardly changed and still consists of two to
three steps: (i.) identifying candidate locations for objects
(object proposals), (ii.) extracting expressive features for
each candidate region, and (iii.) classifying each location
according to these features. Traditional models typically
rely on hand-crafted features like HOG [6] or SIFT [7], and
use classifiers like Boosting [8] for recognition. Recently,
CNNs have become state-of-the-art in many computer vision
tasks by performing end-to-end, task specific joint learning of
features and classification. Typically, standard object detec-
tion pipelines relying on CNNs perform tasks (ii.) and (iii.)
jointly. For step (i.) there are mainly two paradigms:
Sliding window-based: These models perform recogni-
tion at every possible location and scale in the image [9], in a
sliding window fashion.
Region proposal-based: These models suggest a large
number of bounding boxes (object proposals) that are likely
to contain an instance of any object. A discriminator then
has the task of classifying each proposal into different object
classes or background. Examples of such models are R-CNN
1 http://www.kuzikus-namibia.de/
 ,*$566
 Fig. 2. Architecture of the proposed model. It is based on a pre-trained AlexNet and learns specific features for recognition and
localization in two separate branches.
[10] and its extensions, e.g. Fast R-CNN [11]; a proposals
generator typically employed is Selective Search [12].
Due to the large computational complexity of sliding
windows detectors, object proposal-based models have tra-
ditionally been preferred. However, recent studies such as
YOLO [13] exploit the intrinsic multi-scale properties of
CNNs and show large speed-ups compared to methods rely-
ing on object proposals. These models also reduce problems
related to inaccurate object-proposal generation.
Remote sensing tasks have a number of peculiarities that
set them apart from traditional computer vision problems:
objects are typically seen only from above with an absolute
scale, orientation is not discriminative and there is no abso-
lute location prior. Some successful applications of object
detectors on over-head imagery are the detection of seals on
Greenland ice shelves [14], and airplanes [15].
Models not relying on hand-crafted features and region
proposals have only been hesitantly applied to remote sens-
ing datasets. We argue that the aforementioned properties of
remote sensing scenarios could actually be beneficial to ob-
ject detectors using pure CNNs pipelines: objects of simi-
lar classes are of comparable absolute size and the candidate
search space encompasses the entire image. We propose a
CNN architecture, which is optimized for fast and efficient
detection of small, similarly sized objects at arbitrary loca-
tions, and does not require object proposals.
3. PROPOSED ARCHITECTURE
Figure 2 presents the architecture of our model. We base it on
a pre-trained instance of AlexNet [4], which has given good
results in natural image classification [4] and has already been
used successfully in object detection [11]. Under the assump-
tion that low-level features are similar between different im-
age analysis tasks (e.g. color gradients or edge detectors), we
fine-tune the first layers from AlexNet and add new learnable
layers on top.
We adopt a two-branch strategy in our CNN. Our network
performs animal recognition (i.e., assigning a local likelihood
score of the presence of an animal) and localization (i.e., lo-

Authorized licensed use limited to: Universiti Teknikal Malaysia Melaka-UTEM. Downloaded on June 02,2023 at 04:52:10 UTC from IEEE Xplore. Restrictions apply.
calizing plausible animals) in a hybrid-parallel fashion. By
doing so, we let the two branches of the network learn com-
plementary aspects: the former learns the local appearance
of animals while the second learns the size of animals based
on the local likelihood provided by the first branch. Both
branches learn directly from AlexNet features.
In this implementation, we predict locally over 24 × 24
cells (from 224 × 224 pixels inputs, as explained in the next
section): during the forward pass, each cell receives a con-
fidence score on the presence of an animal (from the first
branch) and an estimation of the height and width of the most
likely bounding box (from the second branch). The former is
learned using two convolutional blocks (blocks 1a and 2a in
Fig. 2). The latter is learned by using both features learned
directly form the image (the 128 filters coming from block 1b
in Fig. 2), stacked with the confidence score used to perform
recognition in the first branch, therefore letting the localiza-
tion branch know the spatial extent of detections.
The confidence in the recognition branch is normalized
using a sigmoid. We observed that using a sigmoid activa-
tion function reduced chances of exploding gradients. In turn,
BatchNorm layers [16] reduce vanishing gradient effects. The
constrained output range of [0, 1] facilitates setting thresholds
on the final confidence map. Further studies are needed to
evaluate alternative scoring functions, such as Softmax. Note
that during backpropagation the two branches communicate
on the update of the AlexNet features, therefore performing
multi-task learning.
4. EXPERIMENTS
4.1. Dataset
We evaluate our pipeline on a dataset of UAV images of the
Kuzikus Wildlife Reserve park in central Namibia, acquired
by the SAVMAP project2 in May 2014 [17]. The campaign
resulted in 654 RGB orthorectified images3 . Ground truth
was established via a crowdsourcing campaign organized by
2 http://lasig.epfl.ch/savmap
3 An example can be found at http://dx.doi.org/10.5281/zenodo.16445
 MicroMappers4 to retrieve positions of large animals. Some
missing animals were manually added and bounding boxes
refined to exclude animal shadows. In the end, a total of 1196
animals could be identified.
4.2. Model Training
We divide all images (of size 4000 × 3000 pixels) into 224 ×
224 sub-frames to match the predefined AlexNet input size,
yielding a total of 1379 frames. Out of those, 690 images
(50%) with 1004 animal bounding boxes are used for train-
ing, 276 (20%; 372 bounding boxes) for validation and 413
(30%; 509 animals) for testing. The number of training ex-
amples is relatively small and could potentially lead to over-
fitting. However, note that only a small portion of the weights
is learned from scratch, as we employ the pre-trained AlexNet
in the common branch of our model and only fine-tune it. We
use extensive data augmentation, including mirroring (hori-
zontal and vertical), rotations, shifting (horizontal and verti-
cal) as well as adding small Gaussian noise to the images.
We train our model with stochastic gradient descent with
momentum of 0.9 for 300 epochs, gradually reducing the
learning rate from 10−4 to 10−7 . We employ weight decay
of 0.001 and use for testing the average model of the last ten
epochs. For training, we backpropagate over all the grid cells
scoring with high confidence: outside ground truth bounding
boxes as negative examples and all the grid cells correspond-
ing to ground truth bounding boxes as positive (even when
there are multiple detections on a single bounding box). We
expect this procedure to make the model aware of all the
appearance variations of animals. We filter out overlapping
bounding box predictions using Non-Maximum Suppression
(NMS) as a post-processing step.
As a baseline, we train a Fast R-CNN model using the
same pre-trained AlexNet base network with object propos-
als from Selective Search [12]. This model corresponds to
the CaffeNet model described in [11]. For both models, NMS
threshold and detection cutoff (the threshold above which a
region is detected as “animal”) were selected to yield maxi-
mum F1 scores on the validation set.
5. RESULTS AND DISCUSSION
Table 1 presents performance measures obtained on the held
out test set; visual examples for detections are displayed in
Fig. 3. The PASCAL VOC challenge requires a predicted
bounding box to have an Intersection-over-Union (IoU) score
above 50%, with the closest ground truth bounding box to
be counted as a true positive [18]. However, note that this
threshold is optimized for problems where objects occupy a
large fraction of the image. Assuming fixed image size, de-
viations of a prediction from its closest ground truth have a
much more severe impact (in IoU terms) in small bounding
boxes than in large ones. In our case, the coverage of animal
4 https://micromappers.wordpress.com

Authorized licensed use limited to: Universiti Teknikal Malaysia Melaka-UTEM. Downloaded on June 02,2023 at 04:52:10 UTC from IEEE Xplore. Restrictions apply.
Table 1. Detection results. A bounding box is counted as
correct if its IoU with the closest ground truth exceeds 25%.
Fast R-CNN (baseline) Proposed Model
Ground Truth Objects 509 509
True Positives 429 379
False Positives 843 254
False Negatives 80 130
Precision (UA) 0.34 0.60
Recall (PA) 0.84 0.74
F1 Score 0.48 0.66
Avg. Speed [Hz] 2.96 73.62
bounding boxes hardly exceeds a mere one percent of the im-
age area. We therefore counted a prediction as a true positive
if its IoU exceeded 25%. However, we did retain the rule that
multiple predictions of the same object count as one positive
hit only, crucial when automatically counting instances.
In comparison to Fast R-CNN, we observe a significant
increase in precision (User’s Accuracy; UA) of about 0.25
points. This improvement is attributable to a lower number of
false positives (254 vs. 843). On the one hand, our model has
to evaluate far less potential candidates (242 = 576 against up
to 5000 proposals) and thus the overall false alarm rate upper
bound is lower. On the other hand, the object probability map
from our model allows tuning the localization in one back-
ward pass; a property that does not hold for proposal-based
methods. Looking at the examples in Fig. 3, the lower false
alarm rate can indeed be confirmed.
Our model struggles in detecting certain animals, leading
to a recall rate (Producer’s Accuracy; PA) 0.1 points lower. To
some extent, our model is more prone to predicting multiple,
smaller hits per animal, as can be seen in the bottom left two
images in Fig. 3. This issue could be partially mitigated by
employing stronger NMS, although this does not correct for
too small bounding box sizes. Instead, predicting on a slightly
coarser confidence grid (i.e., smaller than 24 × 24) could lead
to improvements, as it decreases chances for single animals
to be located on multiple prediction cells.
Finally, while Fast R-CNN is able to evaluate an aver-
age of 2.96 images per second5 , our model processes 72.65
images in the same amount of time, both evaluated on a
GTX980TI graphics card. Our model could be used for real-
time applications reducing latency in wildlife monitoring.
6. CONCLUSION
Illegal poaching of wildlife animals remains a global threat
and calls for measures for near real time monitoring of live-
stock. In this paper, we proposed an animal detection system
able to efficiently operate on sub-decimeter images acquired
by UAVs. Experiments have shown our proposed method to
be far more precise in predicting the location of animals in im-
ages compared to the state-of-the-art Fast R-CNN model. Our
5 This estimate does not include the time required to calculate the object
proposals in Fast R-CNN.
 Correct predictions by our model
Multiple detections of a single animal
Fig. 3. Detection examples on the test set (blue; with IoU scores for predictions) and Fast R-CNN baseline (cyan). Ground
truth is in green. Top row shows correct detections for our model, while bottom row show failure cases.
model is able to predict sufficiently accurate bounding boxes,
while producing significantly less false positives. Moreover,
our system is able to operate in near-real time (73Hz), which
is a promising characteristic for more ubiquitous deployment
in real-life, automated, animal monitoring systems.
7. REFERENCES
[1] M. Mulero-Pázmány, R. Stolper, L. D. Van Essen, J. J. Ne-
gro, and T. Sassen, “Remotely piloted aircraft systems as a
rhinoceros anti-poaching tool in Africa,” PLoS One, vol. 9,
no. 1, pp. 1–10, 2014.
[2] G. Wittemyer, J. M. Northrup, J. Blanc, I. Douglas-Hamilton,
P. Omondi, and K. P. Burnham, “Illegal killing for ivory drives
global decline in African elephants,” Proc. Natl. Acad. Sci.,
vol. 111, no. 36, pp. 13 117–13 121, 2014.
[3] D. Biggs, F. Courchamp, R. Martin, and H. P. Possingham,
“Legal Trade of Africa‘s Rhino Horns,” Science, vol. 339, no.
6123, pp. 1038–1039, 2013.
[4] A. Krizhevsky, I. Sulskever, and G. E. Hinton, “ImageNet Clas-
sification with Deep Convolutional Neural Networks,” Adv.
Neural Inf. Process. Syst., pp. 1–9, 2012.
[5] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma,
Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg,
and L. Fei-Fei, “ImageNet Large Scale Visual Recognition
Challenge,” Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252,
2015.
[6] N. Dalal and B. Triggs, “Histograms of Oriented Gradients for
Human Detection,” CVPR, pp. 886–893, 2010.
[7] D. G. Lowe, “Distinctive Image Features from Scale-Invariant
Keypoints,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110,
2004.

Authorized licensed use limited to: Universiti Teknikal Malaysia Melaka-UTEM. Downloaded on June 02,2023 at 04:52:10 UTC from IEEE Xplore. Restrictions apply.
False positives Missed animals
[8] A. Torralba, K. P. Murphy, and W. T. Freeman, “Sharing fea-
tures: efficient boosting procedures for multiclass object detec-
tion,” CVPR, vol. 3, pp. 762–769, 2004.
[9] C. Park, Dennis and Ramanan, Deva and Fowlkes, “Multires-
olution models for object detection,” ECCV, vol. 6314, pp.
1–14, 2010.
[10] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich
Feature Hierarchies for Accurate Object Detection and
Semantic Segmentation,” CVPR, pp. 580–587, 2014.
[11] R. Girshick, “Fast r-cnn,” in ICCV, December 2015.
[12] J. R. R. Uijlings, K. E. A. Van De Sande, T. Gevers, and
A. W. M. Smeulders, “Selective search for object recognition,”
Int. J. Comput. Vis., vol. 104, no. 2, pp. 154–171, 2013.
[13] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only
Look Once: Unified, Real-Time Object Detection,” arXiv,
2016.
[14] A.-B. Salberg, “Detection of seals in remote sensing images
using features extracted from deep convolutional neural net-
works,” IGARSS, no. 0373, pp. 1893–1896, 2015.
[15] F. Zhang, B. Du, S. Member, L. Zhang, S. Member, and M. Xu,
“Weakly Supervised Learning Based on Coupled Convolu-
tional Neural Networks for Aircraft Detection,” IEEE Trans.
Geosci. Remote Sens., vol. 54, no. 9, pp. 1–11, 2016.
[16] S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating
Deep Network Training by Reducing Internal Covariate Shift,”
Arxiv, pp. 1–11, 2015.
[17] F. Ofli, P. Meier, M. Imran, C. Castillo, D. Tuia, N. Rey,
J. Briant, P. Millet, F. Reinhard, M. Parkan, and S. Joost,
“Combining Human Computing and Machine Learning to
Make Sense of Big (Aerial) Data for Disaster Response,” Big
Data, vol. 4, no. 1, pp. 47–59, 2016.
[18] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and
A. Zisserman, “The pascal visual object classes (VOC) chal-
lenge,” Int. J. Comput. Vis., vol. 88, no. 2, pp. 303–338, 2010.