See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/275409650

Quadcopter with camera visual tracking

Article · March 2015

CITATIONS READS
0 1,421

3 authors, including:

Gregor Tonic
University of Ljubljana
1 PUBLICATION   0 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Gregor Tonic on 25 April 2015.

The user has requested enhancement of the downloaded file.

 Quadcopter with camera visual tracking
Gregor Tonic Marko Sandar
University of Ljubljana University of Ljubljana
Ljubljana, Slovenia Ljubljana, Slovenia
Email: gregor.tonic@globalook.com Email: marko.sandar@globalook.com
Abstract—With the rising popularity of quadcopters and
similar flying devices more and more possibilities are open for
various computer vision applications. Quadcopters often feature
advanced high-resolution cameras and can provide image on-
the-fly over wireless networks to the stationary computer system.
In this paper we present a method to autonomous tracking and
following an object with camera-featured quadcopter. Tracking is
done using popular object tracker CamShift and is fast enough to
follow the quadcopter in real time. Advanced quadcopter Walkera
Scout X4 was used to evaluate and confirm that the algorithm
works well on various light and atmosphere conditions.
Keywords—Quadcopter with camera, tracking, drone
I. I NTRODUCTION
Nowadays, on the field of mobile robotics, quadcopters
have become very popular. Quadcopters are flying devices
similar to helicopters, but with four propellers instead of just
one. Such devices can feature multiple sensors like GPS, height
sensor, and accelerator, but also more advanced devices like
video camera or wifi hotspot. They are used for a range of
applications and can be found in military, sport events and
entertainment industry. In this paper we present application of
tracking objects space.
A. Related work
Object tracking belongs in the field of computer vision,
specifically video analysis. Lots of research has been con-
ducted on this field. Video analysis is also used in number
of applications on field of video surveillance, video editing,
object tracking in sports, medicine, games and virtual reality.
Algorithms for object tracking in video give the infor-
mation about location of object in image for every image
in sequence. Usually this information is given in pixels and
region size which covers area of object in image. In connection
with computer vision it is especially interesting autonomous
navigation in space with quadcopters. The navigation is also
supported with GPS and other sensors present on the device.
Using laser and localization algorithms map can be build that
can support quadcopter in its autonomous movements.
II. O BJECT TRACKING
For our tracking purposes we used simple, yet fairly
robust tracker CamShift. Algorithm CamShift (Continuously
Adaptive Mean Shift Algorithm) was derived from similar
algorithm MeanShift. Initially it was intended for face tracking
but can also be used for any other object.
Philip K. Kataay
University of Ljubljana
Ljubljana, Slovenia
Email: philip.kataay@globalook.com
The CamShift algorithm first locates center of the region
using MeanShift and then calculates rotation and size of the
object. MeanShift is iterative algorithm which finds region
in the image that matches our reference image the best. The
search is based on similarity of color histogram of regions in
the image. The HSV color space is used, specifically H (hue)
component. For every image element probability is calculated
that element belongs to tracked region. Probability distribution
map is created. Algorithm Mean Shift locates maximum on
the map and thus center of the region. To prevent algorithm
from using too bright and too dark pixels, border values
are determined for H component of the image. Algorithm
then dynamically adjust color distribution of the target region
among sequential images. After CamShift locates the region
in the image that fits reference the best it remembers color
distribution of that region and uses it in searching in next
image. Similarity measure d(K1 , K2 ) between histograms,
Bhattacharyy distance is used
!1/2
N q
X
d(K1 , K2 ) = 1 − K1(n) × K2(n) .
n=1
Inner square root represents scalar product of histograms
K1 and K2 , represented as vectors. Variable N represents
number of bins in histograms. The whole equation is to cosine
of angle between vectors K1 and K2 .
III. O BJECT FOLLOWING
In this section we describe the procedure our quadcopter
use to autonomously regulate its flight according to the location
of the object in the image. As the 3D object from the space
translates to 2D object in the camera we have to use the
received information to estimate object’s position in space.
A. Camera calibration
First part of translations requires to understand the way
image is formed from 3D space. From intrinsic parameters of
the camera K, rotation matrix R and translation vector t we
obtain the model matrix that is used to perform the projection
P = K · [R|t],
where calibration matrix consist of elements fx , fy , cx and cy .
Process of calibration is intended to obtain intrinsic param-
eters of camera. These include focus length, position of center
of camera, margin of elements and distortion of image. We
see that focus length and center of image is needed to produce
calibration matrix.
B. Real-time estimation of object position
Once we have calibrated system, the easies part is adjust-
ment of jaw and pitch to keep object centered and to maintain
reasonable distance from the object obtained from estimation
of object’s size on image.
On horizontal axis we keep the object center by adjusting
tilt (rotation jaw) y 0 . This can be simply passed as motion
parameter to quadcopters motion processor and is simply
calculated using
p x − w/2
1 − y0 = ( ),
w/2
where x represents current x-coordinate of object in the image
and w represents full width of an image.
To achieve greater stability of the quadcopter during the
flight we use hysteresis which means we only change the
position of object in image if x is somewhere between chosen
xmin and xmax .
Our assumption here was that object height never change
and that quadcopter is always on the same vertical plane, which
is not always the case since the speeding up also produces
some pitch. Principle of vertical adjustments is very similar
as with jaw adjustment. We use hysteresis here as well with
the difference it is set on different axis and positioned slightly
below the center of the image. We have to use such positioning
because the changes of object in space are not visible on the
image if the center of camera lays on the same height as the
center of the object.
IV. E XPERIMENT
We evaluated our method on modern and full-feature
quadcopter Walkera Scout X4, displayed on Image 1.
Fig. 1. Walkera Scout X4 quadcopter
We presented it to various trackable objects and observed
speed and accuracy of its response. It turned out it was able
to follow most of the object with no regard to various light
conditions and speed of the movement.
R EFERENCES
[1] E. Altug, J. Ostrowski, R. Mahony, Control of a quadrotor helicopter
using visual feedback, na: Proceedings of the IEEE International Con-
ference on Robotics and automation, IEEE, 2002, pp. 72-77.
View publication stats
[2] LaFleur, Karl, et al. ”Quadcopter control in three-dimensional space us-
ing a noninvasive motor imagery-based braincomputer interface.” Journal
of neural engineering 10.4 (2013): 046003.
[3] A. Bachrach, A. de Winter, R. He, G. Hemann, S. Prentice, N. Roy,
RANGE - Robust Autonomous Navigation in GPS-denied Environments,
IEEE International Conference on Robotics and Automation, May 3-8,
2010, Anchorage, Alaska, USA, pp.1096-1097.
[4] L. Derafa, A. Ouldali, T. Madani, A. Benallegue, Four Rotors Helicopter
Yaw and Altitude Stabilization, Proceedings of the World Congress on
Engineering 2007, vol I, WCE 2007, July 2 - 4, 2007, London, U.K.
[5] Achtelik, Markus, et al. ”Visual tracking and control of a quadcopter
using a stereo camera system and inertial sensors.” Mechatronics and
Automation, 2009. ICMA 2009. International Conference on. IEEE,
2009.
[6] Luukkonen, Teppo. ”Modelling and control of quadcopter.” Independent
research project in applied mathematics, Espoo (2011)
[7] Quadcopter with camera, Walkera Scout X4, accessed April 2015
[8] Chen, Michael Y., et al. ”Designing a spatially aware and autonomous
quadcopter.” Systems and Information Engineering Design Symposium
(SIEDS), 2013 IEEE. IEEE, 2013.
[9] Olivares-Mendez, Miguel A., et al. ”Quadcopter see and avoid using a
fuzzy controller.” Proceedings of the 10th International FLINS Confer-
ence on Uncertainty Modeling in Knowledge Engineering and Decision
Making (FLINS 2012). World Scientific, 2012.
[10] G. R. Bradski, Computer Vision Face Tracking For Use in a Perceptual
User Interface, Intel Technology Journal Q2 98, Microcomputer Research
Lab, Santa Clara, CA, Intel Corporation.
[11] Allen, John G., Richard YD Xu, and Jesse S. Jin. ”Object tracking using
camshift algorithm and multiple quantized feature spaces.” Proceedings
of the Pan-Sydney area workshop on Visual information processing.
Australian Computer Society, Inc., 2004.
[12] Chu, Hongxia, et al. ”Object tracking algorithm based on camshift algo-
rithm combinating with difference in frame.” Automation and Logistics,
2007 IEEE International Conference on. IEEE, 2007.