You are on page 1of 7

Procedia CIRP 97 (2020) 217–223

Contents lists available at ScienceDirect

Procedia CIRP
journal homepage: www.elsevier.com/locate/procir

8th CIRP Conference of Assembly Technology and Systems

Operator - mobile robot collaboration for synchronized part


movement
Aswin K Ramasubramanian, Nikolaos Papakostas∗
Laboratory for Advanced Manufacturing Simulation and Robotics, School of Mechanical and Materials Engineering, University College Dublin, Ireland

a r t i c l e i n f o a b s t r a c t

Keywords: Mobile robotic platforms have become increasingly popular. Commercially available versions of mobile
Assembly robots are designed to support human operators in typical production environments. They may be used
Adaptive control
for transferring parts from one place to another, as well as for assisting the operator in a series of tasks,
Customization
by utilizing the dexterity of their arm and end effector. This paper focuses on the development of a novel
Control
approach that allows the handling and transportation of parts through the simultaneous operation of hu-
man operators and mobile robots. In particular, a straightforward, easy to implement control strategy is
used to adapt the operation of the mobile robot to the tasks carried out by the operator. This paper dis-
cusses also the advantages of introducing mobile robots in typical industrial environments and compares
their potential against fully automated robotic solutions.
© 2020 The Author(s). Published by Elsevier B.V.
This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/)

1. Introduction tially leading to an increase of the production flexibility and output


(Barattini et al., 2019).
Starting with the deployment of industrial robots in work cells In HRC, operators and robots are expected to work together,
to perform complex and repetitive tasks, manufacturing compa- carrying out a task in a way that utilizes the capabilities of both.
nies have now come a long way forward towards commissioning These tasks may include pick and place, manual assembly tasks as
collaborative robots to facilitate safe Human-Robot Collaborative well as drilling or riveting operations (Barattini et al., 2019). Addi-
(HRC) operations in the shop floor (Lasi et al., 2014). In partic- tionally, these mobile manipulators can potentially be used to as-
ular, the popularity of mobile collaborative robotic platforms has sist the human operators to transfer parts between different loca-
grown significantly due to their ability to support various opera- tions.
tions in typical shop-floor assembly environments. Therefore, they Some commercially available collaborative manipulators have
have a direct impact on shaping the configuration and evolution inbuilt sensors to measure the internal and the external force, as
of the factories of the future (Ajoudani et al., 2018). In principle, well as the torque values at each joint axis for HRC (Chawda and
mobile robotic platforms are capable of operating autonomously, Niemeyer, 2017). Likewise, the mobile platforms are typically
carrying out complex tasks, thus supporting flexible and reconfig- equipped with a floor scanner device to avoid obstacles as well as
urable manufacturing systems towards achieving higher production to prevent contact with human operators, thus making them suit-
output (Shneier, 2015, Makris et al., 2012), since they are capa- able for HRC operations (Barattini et al., 2019).
ble of moving around the work environment. Mobile collaborative Even though collaborative mobile manipulators add flexibility
robot manipulators comprise a mobile robotic platform, which is to production environments, a number of limitations and con-
usually similar to that of an automated guided vehicle (AGV), and straints need to be addressed, such as dynamic path planning with
a collaborative robotic arm mounted on top of it. This configura- obstacle avoidance, localization, and the positioning accuracy of
tion provides an additional degree of support to the human opera- the mobile platform (Michalos et al., 2016). Moreover, manufactur-
tors to perform assembly or materials’ transport operations, poten- ing companies, mainly Small and Medium Enterprises (SMEs), have
technical and monetary constraints when deploying complete, in-
dustrial robotics solutions within their operations. A hybrid envi-
ronment, where humans and robots can cooperate and work to-

Corresponding author. gether as a team, can potentially overcome some of the current
E-mail address: nikolaos.papakostas@ucd.ie (N. Papakostas). challenges faced by SMEs (Antonelli et al., 2016). This paper ex-

https://doi.org/10.1016/j.procir.2020.05.228
2212-8271/© 2020 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
A.K. Ramasubramanian and N. Papakostas Procedia CIRP 97 (2020) 217–223

plores one such idea, where humans and robots can work together Additionally, process simulation tools, such as Gazebo, may be
as a team. used to evaluate the performance of the robot virtually before de-
The aim of this research is to develop a method that would en- ploying it into complex industrial environment. A reinforcement
able both human operators and mobile manipulators to work to- learning algorithm to deal with the trajectory planning plan trajec-
gether in order to perform simple part transportation tasks in the tory for complex pick and place tasks is discussed in (Iriondo et al.,
shop-floor. The proposed approach uses compliance control and 2019).
external force feedback data from the collaborative robotic arm to On the other hand, collaborative manipulators offer various in-
allow the human operator to perform a shared part transportation built functionalities, such as force / torque sensors, impedance con-
task with the mobile manipulator. In essence, the mobile manipu- trol mode to assist hand guided operations and collision detec-
lator is guided by the human operator towards moving a part to a tion for safe HRC (Chawda and Niemeyer, 2017, Popov et al., 2017,
different location in a collaborative fashion. Lee et al., 2016).
In this paper, a new method is presented that allows direct
2. Related work physical Human-Robot Interaction for tasks carried out by the op-
erator and the collaborative mobile manipulator working in syn-
Collaborative mobile manipulators have been used for assist- chronization for transferring parts in a manufacturing environ-
ing human operators in the shop-floor in a series of processes, ment.
including assembly of parts, pick and place as well as warehouse In the following Section 3, the task, the experiment and the
management tasks. A two-phase approach, which enables the hu- synchronization approach are presented. Section 4 presents the re-
man operator to configure the mobile manipulator for executing sults of the experiments, while the last part of the paper presents
autonomous operations in order for complex tasks to be carried the conclusions as well as a few ideas for future work.
out in the shop-floor, considering also uncertainty, is discussed in
(D Ö Mel et al., 2017). However, human operators teaching au-
tonomous mobile manipulators about how to deal with complex 3. Experiments
tasks is a quite challenging process that does not necessarily lead
to an improvement of cycle times in a production environment 3.1. Task description
(Madsen et al., 2015).
A dual-arm mobile collaborative manipulator, which is capable The primary goal of the proposed work is to facilitate the devel-
of autonomous navigation, is demonstrated in reconfigurable man- opment of safe HRC that will in turn enable the coordinated trans-
ufacturing systems, assisting human operators within an assembly fer of parts within a typical manufacturing environment utilizing a
line environment (Kousi et al., 2018). mobile manipulator.
One of the main challenges of using mobile manipulators in The task in the investigated scenario involves a long aluminum
a production environment is the complexity of programming the profile being picked up from a rack where one end of the profile is
robot. A skill-based approach that enables the operators to recon- held by the robot and the other end is held by the human opera-
figure the program of the manipulator by teaching new skills to tor. The operator will guide the mobile manipulator to the desired
the robot is discussed in Ref. (Schou et al., 2018). Such approaches location. The guidance system of the mobile manipulator is based
simplify the process of programming mobile manipulators and en- on cartesian impedance control coupled with the feedback from
able the operator to perform collaborative tasks more efficiently. the force/torque sensors located at the joints of the collaborative
There is a wide range of research studies carried out, mainly fo- manipulator.
cusing on the development of mobile manipulators to carry out as-
sembly line tasks, optimization, and scheduling (Yang et al., 2019).
Furthermore, a lot of research works in the field of HRC focus 3.2. Experimental setup
either on the development of state-of-the-art mobile platforms
(AGV) or the enhancement of collaborative manipulators. The mobile collaborative platform, KUKA KMR iiwa, is shown
However, there is still a lack of clarity on tasks that can al- in Fig. 1 (KUKA, 2017). This platform is used to validate the pro-
low human and mobile manipulators work together as a team. A posed approach in a case where a part has to be co-manipulated
Multi-Human mobile-robot control framework model proposed in in a shop-floor. The KMR iiwa platform consists of a KUKA KMR
Ref. (Kim et al., 2019), discusses an application involving two hu- iiwa R820 robot mounted on top of a KUKA KMP 200 mo-
man operators performing various actions, such as handing over bile platform. The mobile manipulator is programmed using the
an object to a mobile manipulator, drilling of the object, while it KUKA Sunrise workbench, which is a Java-based software platform
is held by the manipulator. This framework may be used for evalu- (Hentout et al., 2019). The use of the KUKA Sunrise Mobility soft-
ating and improving the ergonomics using mobile manipulators in ware is required for programming the KMP 200 platform. The grip-
shop-floor settings. Most existing HRC applications involving mo- ping tool mounted on the flange of the manipulator is the Robotiq
bile manipulators, focus more on building autonomous robots, on 3-finger adaptive gripper (Robotiq, 2020).
the optimization of task allocation, on gesture and voice controls, For illustration purposes, a relatively short part in the experi-
on the use of machine learning, and on AI tools that can be used ment is presented, but the proposed method may well be applied
to improve the performance in specific tasks (Yang et al., 2019). to longer parts that cannot be easily handled or manipulated by a
A ROS-integrated gesture based control for interaction of Human human operator. These parts would normally require the presence
and KUKA youBot is discussed in (Ahmed et al., 2019). There is of a second operator for moving them in different locations of a
still a lack of applications involving the coexistence of mobile ma- manufacturing environment.
nipulators and human operators, sharing a common workspace to The profile is made up of an aluminum tube with a diameter of
carry-out tasks in a synchronized manner, elaborating on the level 30 mm and a length of 1200 mm. Furthermore, the approximate
of interaction between them (Yang et al., 2019, Hoffman, 2019). A weight of the aluminum profile is 610 g. The profile is placed on
collaborative scenario discussed in (Walter et al., 2016) involves a the rack as shown in the Fig. 1 The goal is that the human operator
mobile manipulator assisting the human operator with the fixing and the mobile manipulator move the part between two distant
of frames. The mobile manipulator positions the one end of the positions in a collaborative manner, while allowing the manipula-
frame in parallel with the other end held by the human operator. tor to follow the human operator.

218
A.K. Ramasubramanian and N. Papakostas Procedia CIRP 97 (2020) 217–223

Fig. 1. (a) Aluminum profile; (b) KUKA KMR iiwa platform; (c) KUKA KMR iiwa R820 and Robotiq 3-finger adaptive gripper.

Fig. 2. (a) Workspace representation with position; (b) Robot positioned in front of the table.

Table 1 Table 2
Position parameters for the mobile platform. Final position of the mobile robot.

Position Home (m) P1 (m) Position Home (m) P1 (m)


X 6.5000 8.7500 X 6.5005 8.7489
Y 10.0000 11.5000 Y 10.0006 11.4995
 90.0000 0.0000  89.9494 0.0457

Table 3
3.3. Implementation Difference between measured and the actual
position of the mobile platform.

The laser scanners attached to the mobile manipulator can be Position Home (m) P1 (m)
used to create a map of the work environment. Fig. 2 gives the X 0.0005 0.0011
Y 0.0006 0.0005
representation of the workspace scanned by the manipulator, in-
 0.0506 0.0457
cluding the home position, as well as target point P1 in front of the
table to pick up the object. Table 1 gives the positional information
(poses) X, Y,  of the home and P1 for the mobile manipulator.
The communication to the collaborative arm is initiated via the how the overall process is implemented for allowing the effective
KUKA NavBox. The NavBox consists of a computer that controls the physical human-robot interaction. The pseudocode of the proposed
activities of the mobile platform. A remote task is set up in or- method is shown in Fig. 4.
der for the platform to establish communication with the Sunrise
controller via the NavBox to manipulate the arm. Once the remote
task is initialized, the robot moves to home position, if not already 4. Results
there. Later, the robot moves to point P1 as shown in the Fig. 2. Af-
ter the mobile platform reaches the position P1, the collaborative 4.1. Localization of the mobile platform
arm picks up the object. The arm grasps one end of the aluminum
profile and then the cartesian impedance control for allowing hand The maximum deviation of error along the x and the y axis are
guidance control is initiated. The other end of the profile is picked 0.01257% and 0.006% respectively. Hence, this deviation is small
up by the operator and the profile is lifted out of the rack. and has not affected the success rate with which the manipula-
The translational offset value of the TCP with respect to the tor picks up the aluminum profile. Table 2 illustrates the measured
point of gripping along the X and the Y axis is computed. A force position of the mobile platform positioned at Home and P1 posi-
condition is set to measure the magnitude of the force in the carte- tions. The average difference between the measured and the actual
sian X and Y coordinates. The flowchart shown in Fig. 3 shows poses are shown in Table 3.

219
A.K. Ramasubramanian and N. Papakostas Procedia CIRP 97 (2020) 217–223

Fig. 3. Process flowchart diagram for co-manipulation task.

Fig. 4. Pseudocode of the method that initiates the co-manipulation task.

4.2. Human-robot team

4.2.1. Safety
The minimum separation distance required to enable safe HRC
with the mobile platform here is approximately 0.5 m. The laser
scanners detect the presence of the human operator if the oper-
ator is within the above range and thereby stops its motion, as
shown in Fig. 5. The force torque sensors enable a safety feature
that is in charge of stopping the motion of the arm when a colli-
sion is detected. Therefore, the collaborative platform is overall safe
to support physical human-robot collaboration (Lee et al., 2016,
Hentout et al., 2019).
The first subtask of the collaborative task involves the mobile Fig. 5. Mobile platform in stop as operator is in proximity.
manipulator grasping the aluminum profile. In Fig. 6 the gripper’s
open and close status is shown.

220
A.K. Ramasubramanian and N. Papakostas Procedia CIRP 97 (2020) 217–223

Fig. 8. Motion of the robot along the negative X axis.


Fig. 6. Collaborative arm gripping the Aluminum profile.

move the platform to the location desired. This case scenario is an


example of how human operators and robots may work together
After the profile is grasped by the gripper, cartesian impedance in transportation tasks in shop-floor environments.
control is initiated. In the next step, the human operator assists the In this paper it is demonstrated how the safety features and
robot to remove the profile from the rack. control parameters of modern mobile manipulators could be used
The arm is at this point in compliance mode, measures the po- for allowing the cooperation with human operators in tasks where
sitional offset and the force condition values. Based on the haptic synchronized motion of the mobile platform, the manipulator and
feedback from the human operator, the mobile platform moves ac- the human operator is required. A number of other similar tasks
cordingly along the X or Y axis. may also be considered in industrial settings.
The next step corresponds to the human operator teaming up
with the mobile manipulator and moving on towards transferring 5. Discussion
the part in a synchronized manner. To initiate the motion along
the X-axis, the operator is required to move the arm up as shown This paper presents a straightforward approach that allows hu-
in Fig. 7. This enables the mobile platform to move relative to its man operators to collaborate with mobile robotic manipulators.
last position. The relative X and Y coordinates for the mobile plat- The capability of these platforms to move in a typical industrial en-
forms are computed based on the offset of the arm from its grasp vironment expands the range of applications that could potentially
position. be carried out jointly by human operators and robots. Instead of
Fig. 8 shows the real time data from the robot moving along using special expensive platforms or devices, such as AGVs, which
the X axis. In that particular point of time, for instance, the X, Y are capable of moving from a predefined position to another, with-
and  are 8.2903 m, 11.5163 m and 0.0524 deg respectively with out interacting with human operators, mobile collaborative plat-
velocities vx =−0.064 m/s, vy =0.0 0 0 m/s and  ˙ =0.006 deg/s. The forms may still be interacting with operators in the context of a
blue light on the platform indicates the direction along the motion plethora of tasks, thus potentially increasing the production flexi-
of the mobile platform shown in Fig. 7. bility and overall manufacturing performance.
Similarly, the operator needs to move the arm to the side to At the same time, replacing human intuition and dexterity with
initiate motion along the Y axis. The human operator moves the fully automated robotic cells may not always be a cost-efficient
arm towards the left to start the motion of the platform to move solution, especially in the case of SMEs, where resources are in
along +Y axis. most cases limited. Depending on the complexity of the task it-
Fig. 9 illustrates the human operator moving the arm towards self in the case of fully automated robotic solutions, special fix-
the left and the blue light indicates the direction along which the tures might need to be built, a multitude of sensors might have
platform moves. to be installed and programmed and specific, highly reliable con-
Fig. 10 displays the real time position and the velocity trol strategies will have to be deployed. On the contrary, combin-
of the mobile platform along Y axis. The X, Y and  are ing the programmability advantages of collaborative manipulators
8.0537 m, 11.6901 m and 0.0953 deg respectively, with vx =0 m/s, with the safe, autonomous operation of mobile platforms may in
vy =0.008 m/s, and  ˙ =0.085 deg/s. many cases prove to be a more cost-efficient solution. Especially in
The result of the co-manipulation task discussed here enables the case of highly flexible production environments, where heav-
the control of the mobile platform based on the haptic feedback ily customized or one-of-a-kind products need to be produced, the
provided by the human operator. The arm acts like a joystick to introduction of fully automated robotic solutions many not always

Fig. 7. Haptic feedback to move the robot along X axis.

221
A.K. Ramasubramanian and N. Papakostas Procedia CIRP 97 (2020) 217–223

Fig. 9. Motion of the robot along the positive Y axis. By pulling the arm towards left, the mobile platform moves along +Y axis.

controlled in an easier way. Those plans include the development


of a ROS-based digital twin model of the process, involving the op-
erator and mobile manipulator carrying out joint co-manipulation
tasks. Such methods can be used for evaluating the performance of
collaborative tasks (including the ergonomic performance) as well
as for assessing the safety of the operation before and after de-
ployment.
Furthermore, a series of experiments could be designed and
conducted to enable the dynamic modification of the stiffness of
the collaborative arm while in the cartesian impedance control,
Fig. 10. Real time positional and velocity parameters of the mobile manipulator
while moving along the Y axis.
computing, at the same time, an optimum force condition to trig-
ger in order for the mobile manipulator to adapt to the intrinsic
characteristics of the operator. Such an approach could contribute
be the best option or may not even be an option at all. Therefore, to the reduction of the operators’ fatigue and would also lead to
keeping the human operator in the loop, while integrating the ca- the more robust control of the system.
pabilities of mobile manipulators may still be the best option for Another challenge would be to use the input of wearable de-
many processes in typical manufacturing environments. vices or vision systems to better understand the status of the oper-
On the other hand, it is important that further research and de- ator as well as to predict in a more accurate manner the intentions
velopment be carried out in the future for allowing mobile robotic of the operator, thus achieving better real-time cooperation.
technologies to be further developed so that it be easier to control Overall, it is expected that more tasks will be jointly executed
and adapt mobile robotic platforms to the needs of human opera- by operators and mobile robots in the future.
tors and to the requirements of manual tasks.
It is important to note that the flawless and reliable operation Acknowledgements
of the mobile robot’s safety subsystems is the only way to fos-
ter the development of trust on the operator’s side, which in turn This work has been partially supported by the H2020 research
would have a profound positive effect on the overall performance project SHERLOCK “Seamless and safe human centered robotic ap-
of human-robot systems. plications for novel collaborative workplaces” (Grant Agreement
820689), and by the ERDF-funded INTERREG NWE project Machin-
6. Future work ing4.0 “Towards the machining shop-floor of the future”.

The approach presented in this paper may in principle be References


adapted to the needs of cases where the part is much longer and
Ajoudani, A., Zanchettin, A.M., Ivaldi, S., Albu-Schäffer, A., Kosuge, K., Khatib, O,
heavier, given that both the robotic arm and gripper can still meet 2018. Progress and prospects of the human–robot collaboration. Autonomous
the corresponding payload requirements. A part of the plans for Robots.
future research is to test the proposed approach in more challeng- Ahmed, S., Popov, V., Topalov, A., Shakev, N, 2019. Hand Gesture based concept of
human-mobile robot interaction with leap motion sensor. IFAC-PapersOnLine.
ing tasks, including cases where parts are much bigger and heavier. Antonelli, D., Astanin, S., Bruno, G, 2016. Applicability of human-robot collaboration
It is this kind of cases where the safety aspects pertaining to the to small batch production. IFIP Adv. Inf. Commun. Technol. 480 (480), 24–32.
process will have to be tested in a structured and thorough way Barattini, P., Federico, V., Virk, G.S., Haidegger, T., Saenz, J, 2019. Workspace sharing
in mobile manipulation. Human–Robot Interact. 81–89.
in order to ensure that the process is entirely safe for the human
Chawda, V., Niemeyer, G., 2017. Toward torque control of a KUKA LBR IIWA for phys-
operators and the associated environment, even when the task is ical human-robot interaction. In: IEEE International Conference on Intelligent
not executed as planned, as in the case of the part slipping, or Robots and Systems. IEEE, pp. 6387–6392.
Hentout, A., Aouache, M., Maoudj, A., Akli, I, 2019. Human–robot interaction in in-
the gripper failing to grasp the part. At the same time, additional
dustrial collaborative robotics: a literature review of the decade 2008–2017. Adv
safety measures might have to be implemented. Robotics 33, 764–799.
Part of the future research plans could also involve the devel- Hoffman, G., 2019. Evaluating fluency in human-robot collaboration. IEEE Trans. Hu-
opment and use of reinforcement learning algorithms in order to man-Machine Syst. 49, 209–218.
Iriondo, A., Lazkano, E., Susperregi, L., Urain, J., Fernandez, A., Molina, J, 2019. Pick
make the mobile manipulator more adaptive to the special charac- and place operations in logistics using a mobile manipulator controlled with
teristics of the environment so that the platform be navigated and deep reinforcement learning. Applied Sciences (Switzerland).

222
A.K. Ramasubramanian and N. Papakostas Procedia CIRP 97 (2020) 217–223

Kim, W., Lorenzini, M., Balatti, P., Wu, Y., Ajoudani, A, 2019. Towards ergonomic Popov, D., Klimchik, A., Mavridis, N, 2017. Collision detection, localization & classifi-
control of collaborative effort in multi-human mobile-robot teams. IEEE Inter- cation for industrial robots with joint torque sensors. In: RO-MAN 2017 - 26th
national Conference on Intelligent Robots and Systems. IEEE International Symposium on Robot and Human Interactive Communication,
Kousi, N., Michalos, G., Aivaliotis, S., Makris, S, 2018. An outlook on future assembly 2017-Janua 2017, pp. 838–843 Janua.
systems introducing robotic mobile dual arm workers. Procedia CIRP 72 (72), Robotiq, 2020. 3-finger adaptive robot gripper. In: Online.
33–38. Schou, C., Andersen, R.S., Chrysostomou, D., Bøgh, S., Madsen, O, 2018. Skill-based
KUKA A.G. KMR iiwa - the mobile, intelligent production assistant, 2017. instruction of collaborative robots in industrial settings. Robot. Comput.-Integr.
Lasi H., Fettke P., Kemper H.G., Feld T., Hoffmann M. Industry 4.0. Vol: 6. 2014; Manuf. 53, 72–80.
Lee, S.D., Ahn, K.H., Song, J.B, 2016. Torque control based sensorless hand guiding Shneier, M., 2015. Literature review of mobile robots for manufacturing literature
for direct robot teaching. In: IEEE International Conference on Intelligent Robots review of mobile robots for manufacturing. National Institute of Standards and
and Systems. IEEE, pp. 745–750 2016-Novem2016-Novem. Technology - U.S. Department of Commerce.
Madsen, O., Bøgh, S., Schou, C., Andersen, R.S., Damgaard, J.S., Pedersen, M.R., et al., Walter, C., Schulenburg, E., Saenz, J., Penzlin, F., Elkmann, N, 2016. Demonstra-
2015. Integration of mobile manipulators in an industrial production. Ind. Robot tion of complex task execution using basic functionalities : experiences with
42, 11–18. the mobile assistance robot, “ANNIE. In: The 26th International Conference on
Makris, S., Michalos, G., Eytan, A., Chryssolouris, G, 2012. Cooperating robots for Automated Planning and Scheduling, 1st Workshop on Planning, Scheduling
reconfigurable assembly operations: review and challenges. Procedia CIRP 3 (3), and Dependability in Safe Human-Robot Interaction (SafePlan 2016), London,
346–351. pp. 10–17.
Mel, A.D Ö, Kriegel, S., Kaßecker, M., Brucker, M., Bodenmuller, T., Suppa, M, 2017. Yang, M., Yang, E., Zante, R.C., Post, M., Liu, X, 2019. Collaborative mobile industrial
Toward fully autonomous mobile manipulation for industrial environments. Int. manipulator: a review of system architecture and applications. In: International
J. Adv. Robotic Syst. 14, 1–19. Conference on Automation & Computing. IEEE, pp. 1–6.
Michalos, G., Kousi, N., Makris, S., Chryssolouris, G, 2016. Performance assessment
of production systems with mobile robots. Procedia CIRP 41 (41), 195–200.

223

You might also like