DRONES USED TO TRANSPORT THINGS

I’ll admit to having been somewhat skeptical about the strategy of dangling payloads
on long tethers for drone delivery. I mean, I get why Wing does it— it keeps the drone
and all of its spinny bits well away from untrained users while preserving the
capability of making deliveries to very specific areas that may have nearby obstacles.
But it also seems like you’re adding some risk as well, because once your payload
is out on that long tether, it’s more or less out of your control in at least two axes.
And you can forget about your drone doing anything while this is going on, because
who the heck knows what’s going to happen to your payload if the drone starts
moving around?
This research is by Guanrui Li, Alex Tunchez, and Giuseppe Loianno at the Agile
Robotics and Perception Lab (ARPL) at NYU. As you can see from the video, the
drone makes keeping rock-solid control over that suspended payload look easy, but
it’s very much not, especially considering that everything you see is running onboard
the drone itself at 500Hz— all it takes is an IMU and a downward-facing monocular
camera, along with the drone’s Snapdragon processor.
To get this to work, the drone has to be thinking about two things. First, there’s state
estimation, which is the behavior of the drone itself along with its payload at the end
of the tether. The drone figures this out by watching how the payload moves using
its camera and tracking its own movement with its IMU. Second, there’s predicting
what the payload is going to do next, and how that jives (or not) with what the drone
wants to do next. The researchers developed a model predictive control (MPC)
system for this, with some added perception constraints to make sure that the
behavior of the drone keeps the payload in view of the camera.
At the moment, the top speed of the system is 4 m/s, but it sounds like rather than
increasing the speed of a single payload-swinging drone, the next steps will be to
make the overall system more complicated by somehow using multiple drones to
cooperatively manage tethered payloads that are too big or heavy for one drone to
handle alone.
For more on this, we spoke with Giuseppe Loianno, head of the ARPL.

We've seen some examples of delivery drones delivering suspended loads.

How will this work improve their capabilities?
Giuseppe Loianno: For the first time, we jointly design a perception-constrained
model predictive control and state estimation approaches to enable the autonomy of
a quadrotor with a cable suspended payload using onboard sensing and
computation. The proposed control method guarantees the visibility of the payload
in the robot camera as well as the respect of the system dynamics and actuator
constraints. These are critical design aspects to guarantee safety and resilience for
such a complex and delicate task involving transportation of objects.
The additional challenge involves the fact that we aim to solve the aforementioned
problem using a minimal sensor suite for autonomous navigation made by a single
camera and IMU. This is an ambitious goal since it concurrently involves estimating
the load and the vehicle states. Previous approaches leverage GPS or motion
capture systems for state estimation and do not consider the perception and physical
constraints when solving the problem. We are confident that our solution will
contribute to making a reality the autonomous delivery process in warehouses or in
dense urban areas where the GPS signal is currently absent or shadowed.

Will it make a difference to delivery systems that use an actuated cable and
only leave the load suspended for the delivery itself?
This is certainly an interesting question. We believe that adding an actuated cable
will introduce more disadvantages than benefits. Certainly, an actuated cable can be
leveraged to compensate for cable's swinging motions in windy conditions and/or
increase the delivery precision. However, the introduction of additional actuated
mechanisms and components come at the price of an increased system mass and
inertia. This will reduce the overall flight time and the vehicle’s agility as well as the
system resilience with respect to the transportation task. Finally, active mechanisms
are also more difficult to design compared to passive ones.

What's challenging about doing all of this on-vehicle?

There are several challenges to solve on-board this problem. First, it is very difficult
to concurrently run perception and action on such computationally constrained
platforms in real-time. Second, the first aspect becomes even more challenging if we
consider as in our case a perception-based constrained receding horizon control
problem that aims to guarantee the visibility of the payload during the motion, while
concurrently respecting all the system physical and sensing limitations. Finally, it has
been challenging to run the entire system at a high rate to fully unleash the system’s
agility. We are currently able to reach rates of 500 Hz.

Can your method adapt to loads of varying shapes, sizes, and masses? What
about aerodynamics or flying in wind?
Technically, our approach can easily be adapted to varying objects sizes and
masses. Our previous contributions have already shown the ability to estimate online
changes in the vehicle/load configuration and can potentially be used to operate the
proposed system in dynamic conditions, where the load’s characteristics are
unknown and/or may vary across consecutive flights. This can be useful for both
package delivery or warehouse operations, where different types of objects need to
be transported or manipulated.
The aerodynamics problem is a great point. Overall, our past work has investigated
the aerodynamics of wind disturbances for a single robot without a load. Formulating
these problems for the proposed system is challenging and is still an open research
question. We have some ideas to approach this problem combining Bayesian
estimation techniques with more recent machine learning approaches and we will
tackle it in the near future.

What are the limitations on the performance of the system? How fast and agile
can it be with a suspended payload?
The limits of the performances are established by the actuating and sensing system.
Our approach intrinsically considers both physical and sensing limitations of our
system. From a sensing and computation perspective, we believe to be close to the
limits with speeds of up to 4 m/s. Faster speeds can potentially introduce motion blur
while decreasing the load tracking precision. Moreover, faster motions will increase
as well aerodynamic disturbances that we have just mentioned. In the future,
modeling these phenomena and their incorporation in the proposed solution can
further push the agility.

Your paper talks about extending this approach to multiple vehicles

cooperatively transporting a payload, can you tell us more about that?
We are currently working on a distributed perception and control approach for
cooperative transportation. We already have some very exciting results that we will
share with you very soon! Overall, we can employ a team of aerial robots to
cooperatively transport a payload to increase the payload capacity and endow the
system with additional resilience in case of vehicles’ failures. A cooperative cable
suspended payload cooperative transportation system allows as well to concurrently
and independently control the load’s position and orientation. This is not possible
just using rigid connections. We believe that our approach will have a strong impact
in real-world settings for delivery and constructions in warehouses and GPS-denied
environments such as dense urban areas. Moreover, in post disaster scenarios, a
team of physically interconnected aerial robots can deliver supplies and establish
communication in areas where GPS signal is intermittent or unavailable.