Parallelization of the PY3D monocular visual odometry algorithm
Dionata da Silva Nunes
Signal and Image Processing Laboratory and
Electrical Engineering
Federal University of Rio Grande do Sul
Porto Alegre, Brasil 3308-3136
Email: dionata.nunes@gmail.com
Gustavo Ilha
Signal and Image Processing Laboratory and
Electrical Engineering
Federal University of Rio Grande do Sul
Porto Alegre, Brasil 3308-3136
Email: cucailha@yahoo.com.br
Abstract—This paper presents an improvement to the PY3D
Visual Odometry Algorithm aiming for real time embeded
solutions. The availability of Multicore Systems on Chip de-
mand new software aproaches in order better benefit from
the computacional power. In new complex digital systems
designs, power consumption is a severe retriction and clock
frequency limitation pushes to the parallelism expoitation in
order to cope with computational intensive applications such as
vision. We propose parallel branches by hardware replication
when there is no data dependency in a two stages pipeline
to complete the algotithm. The proposal minimizes downtime
during processing thus seeking better performances. The tests
performed, applying the proposed method, were extracted with
good results, reaching twice the processing speed performed
by the original algorithm. The conclusion that the method
is promising can yield even more guaranteed gains, expand-
ing image subdivisions and applying parallel processing. The
number of cores that a processor can provide can also help to
achieve even better performance gains.
1. Introduction
Currently, one of the relevant topics is Visual Simultane-
ous Localization and Mapping (VSLAM) due to numerous
advantages over other existing methods, such as: cost; acces-
sibility; precision; dependence on external signals, among
others. Its application can also be highlighted in: robotics,
augmented reality, Virtual applications.
One of the problems with this type of application is the
high response time. Applications that use this technology
face many challenges, and one of the most pertinent is
optimization and processing speed. This article proposes a
method to minimize this problem through parallel process-
ing, making it possible so you can perform more than one
Altamiro Amadeu Susin
Signal and Image Processing Laboratory and
Electrical Engineering
Federal University of Rio Grande do Sul
Porto Alegre, Brasil 3308-3136
Email: altamiro.susin@ufrgs.br
Edison Pignaton De Freitas
Electrical Engineering
Federal University of Rio Grande do Sul
Porto Alegre, Brasil 3308-3136
Email: edison.pignaton@ufrgs.br
task at a time. The pipeline technique is also introduced to
optimize processor downtime. Using the parallelism model
becomes very useful for processing large amounts of in-
formation quickly and efficiently. This method has other
inherent advantages due to the division of processing tasks
and, one of the most important is the ability to distribute pro-
cessing on existing CPU cores. This distribution allows for
maximum utilization of the resources available to perform
the processing, achieving considerable performance gains,
in the case of this article the around 50 percent compared
to serialized version.
It can be highlighted some works that used this technic
for performance improvements. Zhang cite c1 ’s work
stands out parallel feature used on NVIDIA GPUs, the
implementation was centralized on the consumption step,
calculating particle weight, and improving response time
[1]. Another recent method proposed for better response
time for visual SLAM was proposed by Pereira [2], the
proposal was to modify SLAR Direct Sparse Odometer
(DSO) and implement code parallelization techniques us-
ing OpenMP, an API which support multi-platform shared
memory multiprocessing programming, several policy-based
code modifications have been made to make the SLAM
algorithm considerably faster.
This article is structured as follows, starting with related
works session where an explanation of the work is made
and a brief comparison with the proposed method. In the
following, we have a problem definition session of that
contextualizing and defining the problem to be dealt with,
right after we have overview of the proposal explaining the
method didactically, we also have a design and implemen-
tation section technically explaining the work done. And a
session of experiments and results, where the presented and
discussed and, finally, a conclusion session.
2. Related Works
The proposed method in this paper has as reference the
works that also use parallelism to obtain performance gains,
as already mentioned, the work by Pereira [2] which use the
OpenMP library to perform this task. The proposal of this
paper doesn’t use library ready to execution parallel, but
proposes an algorithm that parallelizes parts of the code to
obtain better response times.
In the work by Zhang [1] takes parallel processing to
Graphics Processing Uni (GPU). Using NVIDIA graph-
ics card and using Compute Unified Device Architecture
(CUDA) to achieve performance gains in Find and Simulta-
neous Mapping (SLAM) algorithms. Approach that guaran-
tees considerable performance gains by taking into account
specific hardware for calculations. At the proposed work
or parallelism is being performed on the CPU, not using
auxiliary hardware to perform the calculations required to
perform parallelism.
In the ENGEL [5] article, a method of visual odometry
that uses a straightforward and sparse monocular configu-
ration is presented, a combination indicated by him as the
best for real-time applications. But for more robust appli-
cations, the indirect method such as the one displayed for
FORSTER [6] article that proposed the SVO method: Semi-
Direct Visual Odometry for Monocular and Multicamera, a
probabilistic depth estimation algorithm, is indicated that
allows us track corners and weak edges in environments
with little or high-frequency texture.
Another relevant proposal is the work of Mur-Artal
[3], a robust system that implements the visual odometry
algorithm. In this one of the relevant points that can be
highlighted are the good results presented regarding the
automatic corrections in difficult situations of movement
disturbances.
One of the works that was researched and used as a
reference and from material supply for the experiments
was KITTI(Karlsruhe Institute of Technology) dataset. The
project collected images using a car equipped with two color
cameras and two PointGrey Flea2 grayscale video cameras
(10 Hz, resolution: 1392 x 512 pixels, aperture: 90 degrees
x 35 degrees), a scanner Velodyne HDL-64E 3D laser (10
Hz, 64-ray laser, range: 100 m), a GPS / IMU tracking unit
with RTK correction signals (open sky tracking errors ¡ 5
cm) and a computer activated in the Real-time database.
Collecting a total of 3 Tera Bytes of data approximately.
[10]
3. Problem definition
The algorithms that implement the visual odometry
method aim to allow an agent to estimate its spatial location
using only visual information captured by cameras. There
are variations in the technique that can be used, one of which
would be how to catch the images, that can follow three
different lines: monocular, stereo and omnidirectional. The
project in question will be developed using the monocular
method as it is a more attractive method in terms of cost,
simplicity and more compact. Another classification related
to visual odometry refers to the way information is pro-
cessed, which can follow two main lines: direct method and
indirect method. [5] [6]
In the method used, direct information coming directly
from the camera, and not indirect, there is a pre-processing
step of the images received by the camera, generating inter-
mediate values that will be used in a second step referring to
the probabilistic model. In the project, the indirect method
was followed, generating geometric measures, such as points
and vectors, performing a geometric optimization, which
in the case of the direct method is defined as photometric
optimization. [4] [5]
The last definition made for this work concerns the
amount of sampled points that would be used. There are
basically three levels of sampling: sparse, semi-dense and
dense methods. The sparse methods use a limited amount
of strategic image points, such as corners and edges in
Fig ??, we have an example of the sparse method. There
is no correlation between these points. In dense methods,
the objective is to capture as many points as possible in the
image, making a correlation between them. The project was
proposed followed by algorithms that implement the sparse
method. [5]
Figure 1. Sparse sampling levels
For obtain performance gains in the reconstruction of
three-dimensional maps using the monocular visual odome-
try sparse method. It is proposed to divide the input images
into four parts to be processed in parallel, and each part of
the image to be processed in two steps: extraction of points
of interest and calculation of descriptors. The purpose of
separating the method into two distinct steps is to be able to
perform in the pipeline. When an image’s processing stage
is already in the second step, a next image can be called
to run the first stage of calculations. Therefore minimizing
downtime in the execution stream.
4. Proposal Overview
To process the images in the proposed method, initially
the images are subdivided, so that each part of the image is
processed in a new thread, unlike the traditional method
in which the images are processed entirely in a single
master thread. In Fig. 2 represents the subdivisions made
to the images in this job. For the tests, 4 subdivisions were
produced that can be scaled to new subdivisions in the
future, depending on the processing cores existing in the
hardware used. The images used to carry out the experiments
were taken from the KITTI data set, a comparative refer-
ence set for visual odometry, which consists of 22 stereo
sequences with 11 sequences of real training trajectories and
11 unconfirmed soil sequences for evaluation. [10]
Figure 2. image subdivisions for parallel processing
To obtain even better performance gains, a pipeline
for the algorithm was also implemented. For this part, the
main tasks of the algorithm were subdivided into two large
groups called: stage 1 and stage 2. These steps contain
the tasks with the longest response time for processing the
algorithm, which should in its final step generate a 3D map
of the environment. Stage 1 contains the features extraction
calculation, in which the Shi-Tomasi algorithm is used to
detect the detection of corners in the image. In stage 2,
the task of tracking the features extracted in stage 1 is
assigned, in the next image, Lucas Kanade’s algorithm was
used for this task. The implementation of the pipeline allows
parallel processing from phase 1 to phase 2, by the time the
features are being tracked in phase 2, it is already possible
to be extracting a new features in the next image. In Fig
ref fig: pipeline is to apply a diagram of the operation
flow of the proposed method. Can be seen: image capture,
image subdivision, point extraction block (step 1), descriptor
calculation block (step 2). Each stage is promoted to a new
chain so that each stage can be processed in parallel.
Figure 3. Pipeline method application diagram
5. Design and Implementation
For implementation, modifications were made to the
PY3D visual odometry software developed by the Signal
and Image Processing Laboratory (LaPSI) which is part of
the Department of Electrical Engineering (DELET) and the
Graduate Program in Electrical Engineering (PPGEE) of the
Federal University. from Rio Grande do Sul (UFRGS). In
this algorithm 3 main modifications were made:
• Implementation of image subdivisions;
• New offset calculations and image size;
• Code separation in two stages for parallelization.
Generate the 3D map of the environment, which are:
frame and camera position, these two inputs are necessary to
be able to perform the calculations inherent to the process, in
Fig. 4 the step is illustrated and then it proceeds to the image
subdivision stage. For the tests, a loop was implemented
that loads the images from the kitty data set and also the
positions of the camera. For this implementation, was used
of the python language and the opencv library were used to
handle the images.
Figure 4. Image Capture
In the original algortimo flow everything is done in a
single master thread as seen in Fig. 5.
Figure 5. Pipeline method application diagram
This proposal aims to break this flow in parallel pro-
cesses taking advantage of the various processing colors of
the chip. To achieve efficient parallelization, it is necessary
to divide the image into blocks distributed by the processors,
in order to o process several blocks of an same image. It is
also possible to break the calculations in two stages and
execute them in a pipeline Fig. 6. But so that the data
resulting from the first stage be used in the second stage
were saved in a shared memory being possible recovering
and making available for the second stageto use to complete
the calculations.
To ensure that thread are completed safely the first stage
to the beginning of the second stage a call to join () is made
to stop the execution of the current topic until the topic it

Figure 6. Pipeline method application diagram
joins is finished or the topic you are trying to join is not
active, the current thread does not need to go back. The
Fig. 7 shows the complete flowchart of the proposed system.
Figure 7. Flowchart
In the implementation of the image subdivisions, a
function was developed to perform this task splitImage
(filename, numberCores), takes only two parameteres, the
input image and number of cores on the chip, and returns
images subdivided according to the number of cores.
With python’s multiprocessing.cpuCount() function it
is possible to find out the number of hardware cores
and the input image is read through the function
cv2.imread(filename[,flags]) requested by the multiplatform
library OpenCV (Open Source Computer Vision Library)
for the development of applications in the area of computer
vision, which contains modules for Image Processing, which
will be useful in this implementation.
The tests carried out applied 4 subdivisions in the image.
On the Four new images, the developed function imgAt-
tribute(file name) is called, which calculates the displace-
ment of each one. The offsets in the original version of the
code were defined as the center of the image, to be used
as references in the calculation, since now the input image
has been divided into 4 parts, the new offsets are defined as
the lower right corner of the image 1, upper right corner of
image 2, lower left corner of image 3 and upper left corner
of image 4 as defined in Fig 8.
After loading the input image, it is called a developed
function splitImage (file name) in which the width and
length of the image are extracted, using the format attribute,
after the proportional clipping of the image in four parts.
Figure 8. New offset points
The new images are processed in threads (way for a
process to divide itself into two or more tasks that can
be performed concurrently) going through two processing
stages, as shown in Fig 9. At stage 1 are point of inter-
est extraction calculations. In the OpenCV library, some
algorithms are available to perform this task. The algorithm
chosen to play this role, was the Shi-Tomasi [11] corner
detector algorithm available in the cv2.goodFeaturesToTrack
(...) function. The Shi-Tomasi detector has several input and
output parameters. They can be seen in Figure 5.
The image parameter receives the input image to be
processed. The parameter corners receive the output vector
with the corners detected by the algorithm. The maxCorners
parameter indicates the maximum number of corners the
algorithm returns. The qualityLevel parameter indicates the
minimum value that a corner can have as weight. This

Figure 9. OpenCV function cv2.goodFeaturesToTrack
parameter is multiplied by the value of the best corner de-
tected by the algorithm, ie, the smallest eigenvalue Lambda.
For example, if the best quality found is 3,000 and the
qualityLevel is 0.01, corners with a quality value less than
30 will be rejected. The minDistance parameter indicates
the minimum distance that must exist between two corners.
The mask parameter is used to shrink the region in which
to search for corners, or to hide regions in which there is
no interest in finding objects. The blockSize parameter is
used to indicate the size of the neighborhood centered on a
pixel to look for corners. The useHarrisDetector parameter
replaces the Shi Tomasi detector with the Harris detector.
In stage 2 descriptive calculations and optical flow track-
ing are reserved, for this task the Lucas-Kanade algorithm
[12] was used, the principle of this algorithm is to search,
in the following features, for a pixel combination similar to
current features using optical flow equations in a predefined
neighborhood the algorithm then returns a vector with all
the new coordinates of the positions of detected features.
The OpenCV library has a function that implements the
Lucas-Kanade algorithm to cv2.CalcOpticalFlowPyrLK (...).
Lucas-Kanade optical flow tracking has several input param-
eters. They can be seen in Fig 10.
The prevImg parameter gets the previous grayscale
frame of the video processed. The nextImg parameter re-
ceives the current frame of the video, also in grayscale. The
prevPts parameter receives the coordinates of the selected
features. The nextPts parameter predicts the location of
features in the current frame for easier tracking. The winSize
parameter denotes the size of the window around the feature
in which to match the pixel combination. The criteria pa-
rameter defines the stopping criterion when searching for the
depixels combination. Two options are available: stop after a
certain amount of retries using the cv2.TERM-CRITERIA-
COUNT flag; and stop after the search window moves less
than a given value, with the cv2.TERM-CRITERIA-EPS
flag.
Figure 10. OpenCV function cv.CalcOpticalFlowPyrLK
6. Experiments and Results
The algorithm performance tests were performed on
Windows 7 professional 64-bit operating system. The hard-
ware configurations used were as follows: HP Z620 Work-
station computer model; Intel (R) Xeon (R) processor E5-
2643 v2 CPU @ 3.50GHz, 3501 MHz, 6 Core (s), 12 Logic
processor (s) and 32 GB physical memory.
The Fig 11 presents a comparison of the execution times
(in seconds) obtained with the serialized and parallelized
versions of the PY3D algorithm as a function of the first
processing stage, extraction of points of interest, which was
configured for 30000 point tracking. Each point shown in
figure 4 represents the time taken to process one image in the
case of the original serial and the time taken to process four
images in the parallel version. The results show a significant
performance of the parallelized version being on the order of
twice as fast on average. To perform this time measurement,
a subclass of the Thread class defined in python called timer
() are used, it is started by calling the start () function, thus
capturing the time before and after processing applying the
formula 1 gets the result of time spent.
∆t = t − t0 (1)
To perform this measurement, 15 images taken from
the KITTI dataset were used, using the same sequence of
images, both for the original algorithm and for the parallel
version.
The Fig 12 presents a comparison of time in the second
processing stage, which is the calculation of the descriptors,
that observed also significant performance gain from the
parallelized version over the serialized version of the algo-
rithm being twice as fast that the serial version. Analogue
than that performed for time measure performed in the first
stage, here also each points in Fig 12 represent one image
for serial processing and four images for parallel processing.
In Fig 11 and Fig 12 we see the potential of using
the parallel process as opposed to the serialized version
of the PY3D algorithm. Through the use of threads, was
parallelizes the subdivisions made in the input images. But

Figure 11. Stage 1 processing comparison
Figure 12. stage 2 processing comparison
this is only possible because threads are competitors lines
of execution, they are concurrent in the sense that they
execute simultaneously, but each with its own line, behaving
as if they were different programs. Different programs use
different memory area which makes communication more
complex to perform, because it needs other techniques to
exchange information, such as socket, pipes, shared mem-
ories and others. Already for the threads, the memory area
is shared making information exchange more optimized and
simple.
To guarantee the order of data access and prevent one
thread from writing while the other thread reads, some
precautions were taken in this work. One of them was the
use of the join () w function to ensure the finalization of
the thread set of first stage before going to the second stage
processing set, which has a degree of dependence on the
data processed in the first stage and thus avoiding possible
occurrences of deadlocks.
The last comparative graph Fig 13 shows the total pro-
cessing time of the images, and it can also be noted that
performance gains of about two times faster are achieved.
In Fig 13 is notorious the performance gains, for a
better understanding of why these gains, it is necessary to
understand how parallel processing behaves in the CPU.
First of all, it should be noted that the performance gains
are linked directly to the number of processor cores that
a computer has, that is, if we have a computer with only
one CPU, the threads do not run at the same time and
the operating system (OS) will ”scaling” the threads to
allow them to run a little at a time, seeming to be running
concurrently. On the other hand, on a computer with more
than one CPU, which in our case we have 6 core (s) and12
Logical Processor (s), as already mentioned earlier, threads
can run simultaneously: one on each CPU available and
free. There may still be some concurrency issues, but the
operating system can easily stagger and work around these
potential issues.
Figure 13. Total Processing Time Comparison
One of the works that can be mentioned and which also
achieved good results was the CUDA Accelerated Robot
Location and Mapping article [1], which used CUDA to
parallelize the SLAM algorithm, enabling the use of call
functions running on the NVIDIA GPU, dedicated hardware
for video processing. In the case of the test performed in
this article, it was implemented and tested on a GeForce
GTX 660 and a CPU, the Intel Core i5-3570K. [1].
The proposed implementation using CUDA texture
memory increased performance by 11 times. Excellent re-
sults when it comes to running dedicated video processing
hardware. Correlating with the purpose of this paper, the pa-
per discussed does not subdivide images in their parallelism
implementation, so a combination of methods can be made
for even more significant performance gains.
Analyzing the results of the article Pragma-oriented Par-
allelization of the Direct Sparse Odometry SLAM Algorithm
[2], another work that obtained very satisfactory gains much
because of by being able to identify the bottleneck of the
visual odometry algorithm used, which in this case was the
addActiveFrame () function, which represented about 82
For C. Pereira’s work [2], this analysis of the execution
time of the functions was extremely important in order to
define the best work approach, which in this case was the
choice of CPU parallelization of the single loops in the code,
using OpenMP directives. Already this article has a slightly [10] GEIGER, A.; LENZ, P.; URTASUN, R. kitti web site.
different approach aims to use the subdivisions of images 2012. [Online; accessed 2016-07-11]. Available from Internet:
¡http://www.cvlibs.net/datasets/kitti/eval odometry¿.
and pipeline, but achieving gains as good as C.Pereira [2].
In C.pereira’s work [2] performance gains were calcu- [11] SHI, Jianbo et al. Tomasi. Good features to track. In: Computer Vision
and Pattern Recognition. 1994. p. 593-600.
lated using average execution times, ie percentage reduction
in execution time, for example, 43.5 [12] LUCAS, Bruce D. et al. An iterative image registration technique
with an application to stereo vision. 1981.
One point that works can share in the same analysis is
about the gains that can be gained from increasing image [13] MATLOFF, Norman; HSU, Francis. Tutorial on threads programming
with Python. University of California, 2007.
resolution, this is easily explained. Higher resolution images
mean more data per image, and therefore more calculations [14] YANG, Zhiyi; ZHU, Yating; PU, Yong. Parallel image processing
to be made and, in this situation, more advantages are taken based on CUDA. In: 2008 International Conference on Computer
Science and Software Engineering. IEEE, 2008. p. 198-201.
from the image parallelization.
[15] NEIRA, Jos; DAVISON, Andrew J.; LEONARD, John J. Guest edi-
torial special issue on visual SLAM. IEEE Transactions on Robotics,
7. Conclusion v. 24, n. 5, p. 929-931, 2008.
[16] MOURAGNON, Etienne et al. Real time localization and 3d recon-
It is concluded that the proposed parallel processing struction. In: 2006 IEEE Computer Society Conference on Computer
method is really promising obtaining significant perfor- Vision and Pattern Recognition (CVPR’06). IEEE, 2006. p. 363-370.
mance gains in the order of twice as fast purchased with the [17] FUENTES-PACHECO, Jorge; RUIZ-ASCENCIO, José; RENDÓN-
serial method of the original version of the PY3D algorithm, MANCHA, Juan Manuel. Visual simultaneous localization and map-
improving considerably its response times. ping: a survey. Artificial Intelligence Review, v. 43, n. 1, p. 55-81,
2015.
Improvements that can be even greater if applied to a
larger number of processing cores. Also, this is where future [18] Marques, R., Paulino, H., Alexandre, F., Medeiros, P.D.: Algorithmic
skeleton framework for the orchestration of GPU computations. In:
work can go deeper, that is, by further subdividing the Euro-Par 2013. Number 8097 in Lecture Notes in Computer Science,
images and performing parallel processing. Another point Springer-Verlag (2013) 874–885
that can also be explored in the future is the realization of [19] DAVISON, Andrew J. Real-time simultaneous localisation and map-
this method on dedicated hardware, such as GPU (Graphics ping with a single camera. In: Iccv. 2003. p. 1403-1410.
Processing Unit) processing. [20] Veloso, A., Meira, W., Ferreira, R., et al.,“ Asynchronous and Antic-
ipatory Filter-Stream Based Parallel Algorithm for Frequent Itemset
References Mining”, European Conference on Principles of Data Mining and
Knowledge Discovery, 2004.
[21] LI, Jincheng et al. Realization of CUDA-based real-time multi-camera
[1] Zhang, H. and Martin, F., 2013, April. CUDA accelerated robot
visual SLAM in embedded systems. Journal of Real-Time Image
localization and mapping. In 2013 IEEE Conference on Technologies
Processing, p. 1-15, 2019.
for Practical Robot Applications (TePRA) (pp. 1-6). IEEE.
[2] PEREIRA, Cesar; FALCAO, Gabriel; ALEXANDRE, Luı́s A.
Pragma-oriented parallelization of the direct sparse odometry SLAM
algorithm. In: 2019 27th Euromicro International Conference on
Parallel, Distributed and Network-based Processing (PDP). IEEE,
2019. p. 252-259.
[3] Mur-Artal, R., Montiel, J.M.M. and Tardos, J.D., 2015. ORB-SLAM:
a versatile and accurate monocular SLAM system. IEEE transactions
on robotics, 31(5), pp.1147-1163.
[4] Engel, J., Schöps, T. and Cremers, D., 2014, September. LSD-SLAM:
Large-scale direct monocular SLAM. In European conference on
computer vision (pp. 834-849). Springer, Cham.
[5] ENGEL, Jakob; KOLTUN, Vladlen; CREMERS, Daniel. Direct
sparse odometry. IEEE transactions on pattern analysis and machine
intelligence, v. 40, n. 3, p. 611-625, 2017.
[6] FORSTER, Christian et al. SVO: Semidirect visual odometry for
monocular and multicamera systems. IEEE Transactions on Robotics,
v. 33, n. 2, p. 249-265, 2016.
[7] SULEIMAN, Amr et al. Navion: A 2-mW Fully Integrated Real-Time
Visual-Inertial Odometry Accelerator for Autonomous Navigation of
Nano Drones. IEEE Journal of Solid-State Circuits, v. 54, n. 4, p.
1106-1119, 2019.
[8] KIM, Hyungjin et al. RGB-D and Magnetic Sequence-based Graph
SLAM with Kidnap Recovery. In: 2018 18th International Conference
on Control, Automation and Systems (ICCAS). IEEE, 2018. p. 1440-
1443.
[9] PEREIRA, Fabio Irigon. High precision monocular visual odometry.
2018.