Available online at www.sciencedirect.

com

ScienceDirect
Procedia CIRP 79 (2019) 433–438
www.elsevier.com/locate/procedia

12th CIRP Conference on Intelligent Computation in Manufacturing Engineering, 18-20 July 2018,
12 Gulf of Napleins,M
It anyufacturing Engineering, '

Control strategies for small-scaled conveyor modules enabling highly

flexible material flow systems
Claudio Uriartea,*, Abish Asphandiarb, Hendrik Thamera, Ariandy Benggoloc, Michael Freitaga,c
a
BIBA – Bremer Institut für Produktion und Logistik GmbH at the University of Bremen, Bremen, Germany
b
Faculty of Physics / Electrical Engineering, University of Bremen, Bremen, Germany
c
Faculty of Production Engineering, University of Bremen, Bremen, Germany

* Corresponding author. Tel.: +49 421 218 50163; fax: +49-421-218-9050163. E-mail address: uri@biba.uni-bremen.de

Abstract

Nowadays, material flow operations in factories and warehouses are characterized by dynamic changes due to volatile markets. A new
generation of small-scaled conveyor systems, based on identical and intelligent modules can cope with these challenges. Through
omnidirectional capabilities, the systems can be used in complex conveying applications. For accurate movements, intelligent control strategies
are required. This paper describes different control strategies, in particular a vision-based feedback for controlling the correct material flow on
the conveyor. The approaches were evaluated using an exemplary small-scaled conveyor system. The results show that camera-based feedback
reduces considerably the positioning error in small-scaled conveying systems.
© 2019 The Authors. Published by Elsevier B.V.
Keeeyrw-roervdise: w
P Muatnedriearl rfleosw
posnyssitbem
ilisty of the ;scFileenxtiibile
f c pcroom
dum
ctiitotenesyosftethmes 12th CIRP Conference on Intelligent Computation in Manufacturing Engineering.

Keywords: Material flow systems; Control; Flexible production systems; Omnidirectional conveyor system
1. Introduction such
tasks. This makes the conveyor technology rigid, inflexible,
1. ITnhtreoddeuscigtinonof new material flow systems for factories and have tombaeinintetengarnactedfriinenodrldyer toanredalizecossutchintbeanssiicvaell. y simple
forces increasing tasks. This makes the conveyor technology rigfliedx,iibniflilteyxibale,

The desipgrnodoufcntew
cusmtoam
teirzialtifolno,wmsyosretem
dysnfaom
r ifcacctoursiteosmaenrd aledsasptabm
iliatiyntenanfucetu-frreiepnrdoldyuctioanndand lcoogsits-tiinctsenssyisvtem
. s inThthues

w
suaprpelhioerusesreilsatidornivshen ipsby oerxterinnaclrefaosricnegs such as invcorleuam siensg. conventionaslmsayrtstfeamctsorairees n ofottenderleiqvueirrieng Btehseidfelsextihbeilietayseanodf
desire for product customization, more dynamic customer- adaptability that future production and logistics systems in the
supplier relationships or increasing parcel volumes. cao lsnoterxept of smart factories often require. Besides the ease of
C
adoamptpaqnuieicskltyhatto wcahnatngtoes re[1m]ain competitive in -this highly smyasitnem tensanAcdediatniodnarollbyunstness provided by these features, they
d
poinatmoifcvaienw
yn d ,vsoulcahtilaen madaarkpetattieonnvibrroinnm
gse num
neeerdoutso cbhealalebnlegetso. falloswo respysretesm enst m
a duesctisive purchasing criterion for material flow
adapt quickly toofctheannges [1]. From a coinvtehyeolra-tyeocuhtnoflotghye csy
osnttermacst. pAedridoidtisonaanldlyshnoew rt ltyeramcqcuhiraendgaeustionmtahteioonrdaenrd sm ituataetriioanl
point, ow f vhiiecw
h , csuch an adaptation brings numerous challenges. flow syspteermsanmeustadajm usotrm
tiezne tswtiothtihne afusnhcotritontiam
litey, osirnecveen shtohret
Pr
meters of cnon
o c e s c ha gesveyoofrtetnecreqhnuoilroegyan A addadpitionailnlyt,htehleayinotuetgoraftitohne contract periods and short-term changes in the order situation
plant, which can influence several hundred to a thousand require permanent adjustments to the functionality or even the
meters of conveyor technology. Additionally, the integration la
ofyo cruittiocfalthiemmpoartetarniaclef,loaw
s isnyvsetesm
tm.ent must payoff within the
o
gfoondesw components to a material flow system may cause a This is especially for third party logistic service perovtoidebres
complete redesign of the conveyolosgyystiesmchfoaratchtertirzaendspboyrtthoef of critical .importance, as investment must payoff within the

goods. i durA
attiotnhe ocfonth
veyosrertveicchenoclongtyralcetv,elf,orthitshemepaunrschtahsaet htoighblye

taskCso,nnvaem
nteiolynam
l coovninvgeyoobrjetecctshnoonloga ysitsracighhatralcitneer.izFedorbm
y othre performed. - - -

fact that, in psri(nsucicphlea,s itroctantinogn,lsyinfguulflialtli sim

, spolertiinngtr,aeltocg.)i,sttihces
tasks, namely moving objects on a tshtraadigdhittiolninael.mFeocrham niocrael
complex tasks (such as ro.tating, singulaetvinegn, sinodrtuinstgr,iaeltcr.o),btohtes
conveyor system must be extended with additional mechanical
components or modules. Sometimes even industrial robots
2212 8271 © 201
areAntecthesesacroyn. vTehyeorre-teischthneorleofgoyreleavneel,edthfios rmroebaunst tahnadt highly
flexible, easy-to-maintain and cost-efficienltectoonvreelyiaobr lsystems
asre nmeacnesysary. There is tfhuenrcetfiornealaitineesed afsor probssuisbtlea,ndahnidghliys
flexible conveyor technology, which is able to reliably fulfill
as many conveying functionalities as possible, and is

- .
22221122-8-8227711©©20210719ThTehA e uAthuotrhso.rPsu.bPliushbelidshbeydEblsyevEielsreBv.iV er. B.V.
PPeeere-rr-erveiveiw
ewunudnedrerrersepsopnosnibsiib
liitlyitoyf othfethsecisecniteinfitcificcom
com
mim
tteitt
e oeef tohfet2h8eth12Cth
IRC
PID
RePsiC
gnonCfeornefnecreenocne 2In0t1e8ll.igent Computation in Manufacturing Engineering.
10.1016/j.procir.2019.02.117
434 Claudio
Claudio Uriarte
Uriarte et
et al.
al. // Procedia
Procedia CIRP
CIRP 79
79 (2019)
(2019) 433–438
433–438 434

additionally designed according to a modular approach, so that

new system layouts can be configured without much effort. In
addition, they system shall be able to handle as many different
types of goods as possible. Furthermore, the failure of
individual components of the system should not cause a
complete system failure and as far as possible be repaired by the
company's own personnel. Further criteria to be met, lie in the
simple integration of the conveyor technology into existing
Fig. 1. The omnidirectional motion of an object on celluveyor [5].
systems through the use of standardized interfaces and the
expandability of the conveyor system.
and size, as long as they are larger than one conveying
A suitable approach for increasing flexibility is the use of
module and have a flat contact surface.
small-scaled conveyor systems. These systems consists of
Due to its high flexibility, the system meets all the
single modules, which can be combined to arbitrary layouts.
requirements of modern intralogistics systems. With some
Their ability to move objects in different directions is controlled
minor modifications, the system's capabilities can be
only via software. A change in the control software can fully
enhanced to perform operations like sorting, palletizing,
change the material flow task performed by the system [3].
depalletizing, etc. The modular design of the celluveyor also
Therefore, these kind of systems must have a suitable control
offers the advantage of easy adaptation to changes in the flow
strategy. There is a wide range of different control strategies
of conveyed material. So far, if a conventional conveyor
ranging from common traditional centralized control up to
system reaches its limits, it needs to be replaced or
decentralized control strategies. For applications where the
supplemented by a secondary system. Contrary, the
actual position of the conveyed objects is critical, such as
celluveyor can be adapted to the new flow rate simply by
positioning in automatic palletizers, a feedback system,
adding or removing modules. Furthermore, if a delivery
which analyze the pose of the objects of the conveyor and
module fails during operation, it could be deactivated and the
correct the movement is necessary. As the conveyed object
packages would be bypassed, preventing the complete failure
is larger than the single module, it is in contact with several
of the system. Thus, the celluveyor also offers a high degree
modules at any given time, the feedback becomes relevant for
of robustness due to its modular design. Moreover, as only the
robust operation
motors which are in contact with the object are driven, and the
There are two approaches for realizing such a system. The
low friction of the mechanical components, a high energy
first one is to equip each module with a sensor, which
efficiency is achieved. The celluveyor is used as exemplary
increases complexity and costs. The second approach is the
small-scaled conveyor systems within the evaluation of the
use of vision systems for analyzing the situation on the whole
feedback system.
conveying surface. This paper describes the later system and
the usage of analyzed camera images for performing the
correction of the object movements. In the following chapter,
3. Feedback system
the relevant state of the art is presented. Afterwards, the
concept and implementation of the feedback system is described
This section describes the concept of the feedback control
in detail. Finally, the approach is evaluated using an exemplary
system. First, the structure of the concept is explained.
small-scaled conveyor system called celluveyor. The paper
Afterwards, the used hardware and software components are
concludes with a short summary and an outlook on further
listed. The main part of this section explains the implementation
research activities in this field.
of the concept.
2. State of the art
3.1. System design
Modular conveyor system are currently a topic in many
The feedback system consists of different subsystems. A
research development activities. Small-scaled systems
RGB-D camera is used as a sensor. The camera delivers a 2D
represent a category of modular conveyor systems where the
image and a 3D point cloud. The low cost RGB-D cameras open
single module is smaller than the object to be transported.
a way for countless computer vision applications. These
Examplary systems are the magic carpet [2], the cognitive
cameras were commercially introduced for human interaction
conveyor [3] as well as sliding conveyor [4]. The small-scaled
in computer game environments. But now, they are attracting
conveyor system used in this paper is the cellular conveyor
many researchers in developing various computer vision
(celluveyor). The celluveyor is a unique modular conveyor
applications. One of the RGBD or 3D cameras is Asus Xtion
and positioning system. As described by Uriarte et al. [5], it is
Pro which is used in the development of the feedback system.
made of several small hexagonal modules, each consists of
The general perspective of the system is illustrated in
three omnidirectional wheels independently driven by electric
Figure 2. The process starts with object tracking. Objects are
motor as shown in Figure 1. Due to the special arrangement of
detected from the depth image of the camera. These detected
the wheels and the independent drives, several objects can be
objects are tracked and their parameters such as pose and
moved and positioned simultaneously and independently of
dimensions are determined.
each other. The objects can be transported regardless of shape
435 Claudio
Claudio Uriarte
Uriarte et
et al.
al. // Procedia
Procedia CIRP
Fig. 2. General structure of the feedback control system.
The next process is path planning or path creation. This is
done by specifying the path leading to the goal. Once that is
done, the difference between the desired and current pose,
determined by object tracking, will be given as an input to the
controller, which will output the velocity to move the object.
Furthermore, this velocity will not be applied to all the cells
of the Celluveyor but only to the cells underneath the object.
This could be done by selecting only those cells from the list
of available cells that are in the vicinity of the object. Thus,
the cells become free once the object leaves the area, allowing
them to be utilized for transporting another object. In that
way, it would be possible to transport multiple objects to
different paths at once, utilizing the full potential of the
Celluveyor. Finally, the speed of each individual motor in a
cell would be determined to move the object in a specified
direction. This cycle continues until the object reaches its
goal.
3.2. Object tracking
In this section, the task is to specify features extracted from
the target objects, which are then used to control the object's
motion to reach the desired pose. This is known as Visual
Servoing. There are two main types of visual servoing
methods, namely, Position Based Visual Servoing (PBVS)
and Image Based Visual Servoing (IBVS). However, in recent
past, a number of other hybrid methods have also emerged
[7,8]. These methods are different combinations of PBVS and
IBVS. In PBVS, the information from the image is extracted
to reconstruct the 3D pose of an object. It is then compared
with the desired 3D pose to generate an error signal that
drives the object to the goal [9]. Contrary, in IBVS, the error
is computed directly on the features extracted in the 2D image
plane. Hence the system should actuate the object in order to
achieve the desired values in an image plane.
In the feedback system, the PBVS approach is considered.
This is because the other modules of the project, such as path
planner, has no information about the camera. The desired
Fig. 3. Block diagram of the object detection method.
CIRP 79
79 (2019)
(2019) 433–438
433–438 435
goal generated by the path planner is in the world coordinate
system. Thus, in order to integrate with these modules, PBVS
is used. Figure 3 illustrates the block diagram of the proposed
method for object tracking. This includes passing the depth
image through the height filter to get the blobs. The pose and
dimensions of these blobs are determined and tracked. The
tracking of objects is done on depth image at 30 frames per
second (fps) which is passed through a height filter for
segmentation. Unlike standard gray scale image whose pixel
value is an integer ranging from 0 to 255, the data type of this
image is float, which means that each pixel has been given a
value based on height from the camera. The depth image is
passed through a height filter which allows only a certain
range of pixel values and generates a binary image. Any value
in this range will be assigned a pixel value of one and pixel
value outside the range will get zero. Once the filtered depth
image is obtained, it is passed through a couple of algorithms
to detect the object parameters. After passing the depth image
through a height filter, the next step is to detect the blobs in a
binary image. This is done by applying the contour and ellipse
detection on to the binary image. These techniques are robust
against noise and distortion, even when the blobs are not in
the perfect shape.
The first detection algorithm applied to the filtered image
is contour detection. Contours are the list of continuous points
along the boundary having similar intensities. The contours
are detected from the binary image according to the algorithm
explained in [10]. Once the contours are detected, they are
filtered based on an area which is computed using the Green's
theorem [11]. The contours of a small area would be
discarded as a possible distortion. The center points of the
remaining contours are determined using spatial moments based
also on Green's theorem. The contours are then passed for
ellipse detection to get the object's orientation and dimensions.
As contour detection only determines the center points, the
orientation and the dimensions of the object are still
unknown. For this reason, ellipse detection is used. Using the
algorithm described by Fitzgibbon and Fisher [12], the object's
size and angle are obtained. It is to be noted that this technique
also gives the center point of the ellipse. As the ellipse is formed
inside the blob and the blob could be distorted, the ellipse does
not reach the boundary of the blob. This causes a slight shift in
the center compared to the center calculated using the Green
theorem. Being less robust to distortions, the center calculated
by Fitzgibbon and Fisher algorithm is discarded. Finally, the
object's parameters are saved in a container are passed for
tracking. A disadvantage of this algorithm lays in the
orientation detection. The angle range after ellipse detection
from OpenCV functions is from
90° to -90° because of the symmetry of the object. This causes
a break when the object goes beyond 90°, resulting in unusual
behavior. In addition to that, angle extension is required if the
box is needed to be rotated more/less than 90°. Here, the
Hungarian Algorithm is used to assign IDs to keep track of
the objects. The algorithm is based on the objects’ distance
difference from one frame to another [13]. A dynamic matrix
is formed containing the cost distance between the previous
positions of the tracked objects with IDs (rows) and the
positions of the currently detected objects (columns). The
436
current object with the least cost distance in a particular row
of the matrix would be assigned an ID of the previous tracked
object. The matrix would be updated after every frame.
Whenever a new object comes into the frame (columns >
rows), it would contain a higher cost distance to previous
tracked objects' positions as compared to the other objects.
Hence it would not be assigned to any existing ID but a new
one will be generated for that object. Similarly, when the
object is not detected in a certain amount of consecutive
frames (rows > columns), the unassigned ID would be taken
out of the matrix. After object assignment using Hungarian
Filter, the list of objects is passed through a Kalman Filter.
3.3. Path planning
Today, there are many path planning techniques which will
find the shortest and most reliable path, like Time Elastic
Band (TEB) planner [14]. However, in the context of material
flow systems the shortest path is not the only requirement.
The speed and orientation of the object needs to be controlled
for better handling, which unfortunately cannot be done by
any existing path planning techniques. Also, some objects'
center of mass is higher, causing them to be unstable. So if there
is a sharp turn at high speed, the object might topple and cause
damage. The object path is made by Multi Segmented Cubic
Bézier curves. The Bézier curve is a special type of curve
whose curvature is influenced by one or more intermediate
control points. Bézier curve is basically a form of a binomial
function that follows Bernstein polynomials equation. To
create a multi-segmented cubic Bézier curve, it is essential to
understand the single segment cubic Bézier curve. The cubic
Bézier curve (n = 3) of a single segment (k =
1), having two knots, is expressed in equation (1). The control
points of a single segmented Bezier curve (line in this case)
would be initialized on the line.
𝐵𝐵 ͵ ǡ 𝑡 ൌ ͳ − 𝑡 ͵ 𝑃𝑜𝑜 ൌ ͵ ͳ − 𝑡 ʹ 𝐶��ͳ ൌ ͵ ͳ − 𝑡 𝑡� ʹ 𝐶𝐶 ʹ ൌ 𝑡� ͵ 𝑃ͳ
Ͳ≤ 𝑡 ≤ ͳ
A single segmented cubic Bézier curve might seem enough
but it lacks the flexibility in order to create a complex path.
To realize the spline, it is important to first determine the
control points of every segment. So that when the Bézier
curve is initialized, it would create a smooth path passing
through all the knots. In case of more than two knot points (k
> 1), the Bézier control points must be calculated in such a
way that the spline curve has have two continuous derivatives
at knot points [15]. For the cubic Bézier curve with k + 1
knots and k segments, the equation at the ith segment is given
in equation (2).
𝐵𝐵 ͵ ǡ 𝑡 ൌ ͳ − 𝑡 ͵ 𝑃��𝑖 −ͳ ൌ ͵ ͳ − 𝑡 ʹ 𝐶��ͳ ǡ𝑖𝑖 ൌ ͵ ͳ − 𝑡 𝑡�ʹ 𝐶��ʹ ǡ𝑖𝑖
𝑖 ൌ ͳǡ … ǡ 𝑘𝑘
Once it is understood how the control points of
multisegmented cubic Bézier curve are realized, it can be used
to draw the paths as well as the velocity and angle profiles of
the paths. Similar to path creation, the velocity and angle
profiles are created using Bezier curve but using only a single
segment (with two knots).The generation of angle profile is
quite trivial, as it is the value that is being denoted on the
vertical axis. Contrary, the velocity profile, which is created
Claudio
Claudio Uriarte
Uriarte et
et al.
al. // Procedia
Procedia CIRP
CIRP 79
79 (2019)
(2019) 433–438
433–438 436
without the knowledge about the path, must be linked with the
path. In other words, as the time interval for both path and
velocity are constant, so changing the velocity would affect
the distance reached in each time in the interval. Hence,
besides recording the velocities for every cycle during the
interval, it should also impact the path. This would result in path
points scattered when the velocity is high (more distance
covered) or clustered when the velocity is low (less distance
covered) or remain equidistant when the velocity is constant
(same distance covered every time).
One quick way for realization is simply running through
the path again and record a discrete set of time for distance
values at a higher resolution. The lookup table (LUT) created
can then be used to interpolate the time, given a distance
along the curve. With an appropriate time and the path's curve
equation, the exact point on the path can be marked [15]. The
distance that must be used to get the time from the LUT
should be from the velocity time graph. The distance is the
product of normalized integral of velocity with the total
distance achieved by the path. Finally the created path is
transformed to the world coordinate system.
3.4. Control system
The controller discussed in this section provides the
velocities (x˙c, y˙c, θ˙c) to the wheels of the Celluveyor with
which the object needs to be moved. It takes the current
position and orientation of the object as a feedback to decide
where it needs to go with a certain velocity and angle. The
controller is divided into two parts. The first part is the
supervisory controller which makes the object follow the path
provided by the path planner. The second part of the
controller controls the direction and orientation with which
the object needs to be moved. Both parts of the controller will
be explained in this section. Each trajectory provided by the
path planner consists of a list of local set points containing the
(1) desired position, speed and angle .The list of these
set points is passed to the supervisory controller, whose job is
to check which set points the object had already achieved and
selects the next target set point. These selected set points
would be passed to the position and angle controller. This is
done by keeping the reference of the set point that needs to be
reached. When the object reaches the selected set point, the
supervisory controller would update the reference to the next
one and so on until it reaches the goal. This makes sure that if
there are two identical set points along a path, the object
would still make the whole course. This can be seen in Figure
4. The values given by the supervisory controller acts as the
reference value for the main controller. It is subdivided into two
ൌ 𝑡��͵ 𝑃ͳ parts for controlling the direction and angle of the object as
(2) both are independent of one another.
Fig. 4. Supervisory controller.
437 Claudio
Claudio Uriarte
Uriarte et
et al.
al. // Procedia
Procedia CIRP
CIRP 79
79 (2019)
(2019) 433–438
433–438 437

simple translation which contains a constant speed and no

rotation of the object. The objective is to determine the
discrepancy between the desired and actual speed. Thirty
experiments are performed using the trajectory by two
different speeds. The results are presented in Figure 7. As it
can be seen the speed of the tracked object varies between
±0.02m/s with a time delay of approximately 1.5s. This is the
minimum noise fluctuation in the depth image. This noise can
be even reduced by tuning the covariance matrices of the
Kalman Filter but that causes an increase in the response time,
causing a lag between the actual and the tracked position of
the object. Hence, it’s a compromise between the noise and
Fig. 5. Control loop. the speed. Also, the delay of 1.5s is the cumulative effect of
controller initialization as well as passing the signal through
As shown in Figure 5, the angle controller is designed to the computer to microcontrollers of the motors via PLC,
control the orientation of the object with a simple PID which generates the overhead time.
controller. The desired angle provided by the
supervisory controller is given as a reference to the PID
controller. The current angle of the object is provided as
a feedback. The parameters of a PID controller are tuned
manually. (3)
𝜃��𝑒
𝜃��𝑒 ൌ 𝐾��𝑃𝑃 𝜃��𝑒𝑒 ൌ 𝐾��𝐼 𝜃��𝑒 𝑑𝑑𝑡𝑡 ൌ 𝐾��𝐷𝐷
𝑑𝑑𝑡𝑡
𝜃��𝑡 ൌ 𝑡�𝑎��𝑟��𝑔��𝑒��𝑡� 𝑎��𝑛��𝑔��𝑙�𝑒
𝜃��𝑤𝑤 ൌ 𝑐 𝑢��𝑟��𝑟��𝑒��𝑛��𝑡� 𝑎��𝑛��𝑔��𝑙�𝑒��𝑜��𝑓�� 𝑡 ℎ𝑒��𝑜��𝑏𝑏𝑗𝑗𝑒��𝑐𝑐𝑡� 𝑖�𝑛�� 𝑤𝑤𝑜��𝑟��𝑙�𝑑�� 𝑐 𝑜��𝑜��𝑟��𝑑𝑑𝑖�𝑛��𝑎��𝑡��𝑒��𝑠𝑠
𝜃��𝑒 ൌ 𝑒��𝑟��𝑟��𝑜��𝑟�� 𝑎��𝑛𝑛𝑔��𝑙�𝑒
𝜃��𝑡 ൌ 𝑐 𝑜��𝑛��𝑡�𝑟��𝑜��𝑙�𝑙�𝑒��𝑟�� 𝑎��𝑛��𝑔��𝑢��𝑙�𝑎��𝑟�� 𝑣��𝑒��𝑙�𝑜��𝑐𝑐𝑖�𝑡��𝑦𝑦
𝐾��𝑃𝑃 ǡ 𝐾��𝐼 ǡ 𝐾��𝐷𝐷 ൌ 𝐶��𝑜��𝑛��𝑡�𝑟��𝑜��𝑙� 𝑝𝑝𝑎��𝑟��𝑎��𝑚𝑚𝑒��𝑡�𝑒��𝑟��𝑠�� 𝑓𝑓𝑜��𝑟�� 𝑃��ǡ 𝐼� ǡ 𝐷𝐷
The second part of the controller is used to control the
object's direction of motion. This is done by finding the error
vector based upon the current position of the object with Fig. 6. Experiment setup.
respect to the reference provided by the supervisory
controller. The unit vector of this error vector is then Figures 8, 9, 10 and 11 show the results of the second step
determined. Once the unit vector is calculated, it is coupled with of the experiments: the graphs between actual and tracked
the target speed to get the velocity x˙c and y˙c. Here vt is the position with the error along the x-axis and y-axis. As it can
target velocity provided by the path planner. After the be seen from the figures, the error is bigger at the edges (start
controlled velocities have been determined, they are used to and end) and minimizes as the object comes in the center.
drive the wheels of the conveyor system.

4. Evaluation

This chapter discusses the setup to perform the

experiments and displays the result of them. Figure 6 shows
the experiment setup. It consists of the omnidirectional
conveyor system celluveyor and an Asus Xtion Pro camera
mounted on top of the celluveyor. The camera has a resolution
640 x 480 pixels and a depth range from 0.8 to 3.5 m by a frame
rate of 30 fps. The celluveyor conveyor system has a surface of Fig. 7. Measured speeds.
1.60 m x 1.45 m. The distance from camera to conveyor is
approximately 2.0 m.
Before passing the depth image through object detection
and tracking algorithms described in chapter 3.1, intrinsic
calibration needs to be done to rectify the images. It is done
separately for both depth and RGB outputs with a chessboard
pattern in various positions and orientations. Afterwards the
calibration parameters were saved. Once the path is created, it
will forwarded to the controller to do further processing.
Fig. 8. Object moved along x-axis.

In the first step, the experiments focus on controlling the

speed of the objects. For this, a trajectory is defined as a
438 Claudio
Claudio Uriarte
Uriarte et
et al.
al. // Procedia
Procedia CIRP
Fig. 9. Error in the x-axis.
Fig. 10. Object moved along y-axis.
Fig. 11. Error in the y-axis.
There are two reasons for this. First, the camera's depth
sensor has temporal and spatial distortion at the edges which
creates a curved depth image for the parallel plane. This fault
can be minimized by intrinsic calibration but cannot be
removed completely. Second, the increase in error on the
sides is due to the perspective error of the image. If an object
is perpendicular to the camera, the sides of the object will not
be seen and the top layer of the object being directly over the
actual position. However, on the edges, the top layer of the
object will be shifted than the actual position.
Conclusion
This paper presented a closed loop control system using
the position and orientation of the objects estimated by
analyzing camera depth images. The object detection and
tracking are used to get the position, orientation and dimensions
while keeping track of the objects. The object tracking has
an error range of ±5cm on the sides of the conveyor system due
to distortion in depth image and perception. Besides, the objects
must be kept with a minimum separation distance of 3cm from
each other. The controller designed will determine the linear
and angular velocities with which the object will follow the
trajectory provided by the user. The error between desired
velocity and the actual velocity of the object is ±0.02m/s.
CIRP 79
79 (2019)
(2019) 433–438
433–438 438
The first results are very promising and confirm the great
potential of camera based feedback system for highly flexible
and modular conveyor systems. At the moment, the camera was
mounted parallel to the table. In future, one or more cameras
could be used, not necessarily right above the table, if the table
is bigger that it will not fit in a single image. For that, the
multiple depth images need to be stitched according to the
common features detected in the images. The transformation
matrix could be expanded to include the other degrees of
freedom (pitch and roll). Finally, the object detection
algorithm still offers potential for improvement.
Acknowledgement
We would like to thank the German Ministry for Economic
Affairs and Energy for their support of the “Celluveyor-
03EFEHB024” project.
References
[1] Günthner W, Wilke M. Anforderungen an automatisierte
Materialflusssysteme für wandelbare Logistikstrukturen. Dokumentation /
Wissenschaftssymposium Logistik der BVL 2002.
[2] Oyobe H, Hori Y. Object conveyance system “Magic Carpet” consisting
of 64 linear actuators-object position feedback control with object
position estimation. IEEE/ASME International Conference on Advanced
Intelligent Mechatronics 2001: p. 1307 - 1312.
[3] Krühn T, Falkenberg S, Overmeyer L. Decentralized control for small-
scaled conveyor modules with cellular automata. IEEE International
Conference on Automation and Logistics 2010: p. 237-242.
[4] Gerthsen M, Bruns R. Flexible und adaptive Intralogistik Gleitförderung –
eine neue Materialflusstechnologie.Hebezeuge Fördermittel 2009;49(1-2).
[5] Uriarte C, Thamer H, Freitag M, Thoben K-D. Flexible Automatisierung
logistischer Prozesse durch modulare Roboter- und Materialflusssysteme.
Tagungsband 11. Fachkolloquium der WGTL 2015: p. 206-211.
[6] Celluveyor. Cellular conveyor. celluveyor.com/system-technology/
cellular-conveyor, 2018. [Online; accessed 18.02.2018].
[7] De Luca A, Ferri M, Oriolo G, Giordan P. Visual servoing with exploitation
of redundancy: An experimental study. IEEE International Conference on
Robotics and Automation 2008: p. 3231 - 3237.
[8] Chaumette F, Hutchinson S. Visual servo control, part i: Basic
approaches. IEEE Robotics and Automation Magazine, Institute of
Electrical and Electronics Engineers 2006: p. 82-90.
[9] Siciliano B, Sciavicco L, Villani L, Oriolo G. Robotics: Modelling, Planning
and Control. London: Springer: 2009.
[10] Suzuki S, Abe K. Topological structural analysis of digitized binary
images by border following. CVGIP 1985; 30 1: p. 32-46.
[11] Chen Q. A basic introduction to opencv for image processing.
DiscoverLab School of Information Technology and Engineering,
University of Ottawa: 2007.
[12] Fitzgibbon A, Fisher R. A buyer's guide to conic fitting. Proc.5th British
Machine Vision Conference 1995: p. 513-522.
[13] The hungarian algorithm explained step by step and an online tool to solve
your own assignment problems. hungarianalgorithm.com, 2013. [Online;
accessed 06.02.2018].
[14] Rösmann C, Feiten W, Wösch T, Hoffmann F, Bertram T. Efficient
trajectory optimization using a sparse model. Proc. IEEE European
Conference on Mobile Robots 2013: p. 138-143.
[15] Polikarpotchkin O, Lee P. Draw a smooth curve through a set of 2d
points with bezier primitives. codeproject.com/Articles/31859/ Draw-a-
Smooth-Curve-through-a-Set-of-D-Points-wit, 2009. [Online;accessed
06.02.2018]