MTE PROJET

EE 442
ROBOTICS AND MACHINE VISION
VI semester 21-22

Review of an Autonomous Harvesting Robot

for
Picking & Gathering Brinjals

Faculty Advisor : Dr. Rajesh Kumar

Dept. of Electrical Engineering, Delhi Technological University

Angshuk Basu 2K18/EE/027

Anas Ayub 2K19/EE/038
 Introduction

Harvesting is an extremely labour intensive operation and with the

prevalence of technology in rural India, the opportunity for deploying
robotics to do farming activities seems promising. The development of
technology in agriculture has brought forth a new breed of farmers
termed the modern farmers who strive to harness the use of technology
for improved yields and profits.

India is the second largest producer of Brinjals in the world. Given how
harvesting is the most labour intensive activity and the fact that doing
such activities manually in high conditions is physically demanding and are
prone to injuries a robot which can perform harvesting activities can
substantially optimise the process and decrease costs to the farmer.

It is estimated that 30 percent of the entire production cost is spent on

pay for physical labour and with the seasonality in the business sometimes
it could be difficult to obtain appropriate labour. With stagnating demand
, the market is getting more competitive to the farmer and optimising
human capital is crucial for higher efficiency in the production process.

A contemporary farm should rely on autonomous decision-making to (i)

assure asset efficiency, (ii) achieve higher product quality, (iii) lower
costs, (iv) reduce delivery time to consumers, and (v) enhance market
share and profitability and stabilise the labour force.
 Existing Research

The present study in the application of Robotics for agriculture has been
able to solve challenges with respect to cutting and gathering mechanisms
of the robot arm where the harvesting mechanism is automated but the
movement of the robot is along a fixed rail line.

The robot mounted over UGV comprises a 6 DOF manipulator or

placement of the end-effector during the harvest operation.
Cutting Mechanism:

The end-effector has the following parts: a gripper and

suction cup to capture the fruit and a thermal cutting
device to separate the fruit from the plant. A light
weight camera positioned on top of the end-effector.

During the harvest procedure the two fingers of the motor gripper, hold
the stalk of the fruit. Once the fruit has been chopped, the suction cup
located below the gripper fingers immobilises the fruit throughout the
transportation phase and the fruit is put in a horizontal orientation and
gently lowered into the storage crate. Then the gripper releases the
fruit.
The design of the cutting device requires considerable care.
In horticultural practice, the grower uses a knife to cut the stalk of the
Brinjal. By using the same knife over and over again, there is a possibility
of transmission of viruses from one plant to the other.

Thus for cutting of the Brinjal stalks a thermal cutting technique was
devised which involves two electrodes carrying a high-frequency electrical
potential. Once a stalk hits the electrodes, it is severed by the HF
current between the electrodes.

The high water content of the tissue substance alleviates this process.
This technique has two unique advantages. First of all, during the cutting
process viruses are killed due to the noticeable temperature increase at
the cutting surface. Secondly, the wounds of both the fruit and the plant
are closed during the cutting process. This results in less water loss from
the fruit and hence a longer shelf life.
 Innovative Component

1. To connect an autonomous vehicular component to the harvesting robot

where it would proceed on a predefined path and perform harvesting
operation on detection of the crop.

2. To measure and record the quality of produce harvested and store in a

database for analysis.

Rough Depiction of the model

 Objective 1

Unmanned vehicular component

❖ UGV uses 24V Gearhead motors for mobilisation.

❖ VEX Cortex Microcontroller is used to transmit bi directional
communication.
❖ 3 Lidar sensors are used for local path planning to yield better
results along with a pre planned global path.
❖ Field can be prepared with uniform gaps b/w the vegetables so the
robot will make stops periodically and the harvesting function will
occur in that time.
❖ UGV uses global path planning information along with accumulated
sensor data run by A* algorithm to obtain optimum path.
Why A* Algorithm?

Path planning using A* algorithm is a technique used to identify the shortest

path between a source and a destination. A* algorithm is a heuristic method of
search which ensures that navigation is done in the least time and in the most
optimum way, conserving energy and offering an optimised way of accomplishing
activities.

If we employ standard approaches. The path can be found when we check every
conceivable path and then check all the paths and decide the shortest path. But
with A* algorithm when the end point and start point is specified, the path is
determined by inspecting only the path of interest rather than all the points.

The heuristic function is a mechanism so that we may make an educated

prediction regarding the path notion. One popular heuristic function is
Aerial Distance square.
((x1-x2)^2 + (y1-y2)^2)
Another function is the Manchester distance
|y1-y2|+|x1-x2|
When we know which heuristic is better we add it to a function that
gives us the overall path.
A* star function has two part f(x) = g(x) + h(x)
h(x) is the heuristics value Here
g(x) is the value from the start to that point.
 Simulation Of Obstacle Avoidance
And Path Navigation
For the simulation we consider a sample farm in the form of 10x10 grid size.

In the sample farm taken above black boxes are considered obstacles
and brinjal plants are considered to be round blue only which will be
detected by image processing cameras.

A program run in python is used for identifying shorted path:

A* Algorithm Program
 Output
1. The coordinates of occupied grids:

The code return coordinates of the occupied grids in the form of

tuples. The coordinates are represented by the x and y axis. The
local camera on UGV recognizes the brinjal plant and performs
the operation.

2. The minimum path:

The program uses compare_ssim function from scikit-image for

matching object which is nearest to it.. Traversal is done by
moving either horizontally or vertically. A* search is used to find
this shortest path.
Screenshot of the Command O/P :

Customised Screen O/P :

References :
1. Recognition and Localization Methods for Vision-Based Fruit Picking Robots: A Review
2. Lidar Based Off-road Negative Obstacle Detection and Analysis :Jacoby Larson and Mohan
Trivedi IEEE Xplore
3. http://www.igvc.org/reports.htm
4. https://neptune.ai/blog/image-processing-python
5. https://www.researchgate.net/publication/224749848_A_lane_detection_method_for_int
elligent_ground_vehicle_competition
 Work to be shown in II Review

Objective 2
To measure and record the quality of produce harvested and
store in a database for analysis :

The signs of a ripe brinjal are :- Glossy purple skin

Length: 7.5 - 9.5 cm
Width: 5 cm diameter
Weight: around 140 g

The recognition system will measure the colour and dimensions of the
vegetable and store the data in the memory of the microcontroller.

A non-intrusive method can be used to find the weight of the brinjal

through its dimensions. If a brinjal is assumed to be a cylinder then the
volume can be calculated through the height and diameter.
Mass = Volume * Density
Where the density will remain constant

Prior research has shown that

Vol. of vegetable = 0.714 * (Vol. found on cylindrical approximation)
So the following factor has to be accounted for while calculating the
weight.

Paper referred for this objective:

G. Barbieri and M.I. Sifola - Volume and weight prediction models

from linear measurements for eggplant (Solanum melongena L.) fruit

Time of flight (ToF) cameras can be used for imaging of the vegetable
(here, brinjal) with which the dimensions can be obtained and weight can
be resolved.
The ToF camera can be mounted on the end-effector due its light-weight.
This task is to be operated before the brinjal is cut-off from its stem.

Figure Showing : Flow action for harvesting brinjals and storing produce in database

Helios Flex ToF Camera

Image Source: LUCID vision labs
 Conclusion
The project focussed on the implementation of an autonomous component
to the existing harvesting system with a path planning based on signal
detection. Prior research in this field has used a rail system with tracks
for the movement but in many uneven conditions such a rail system may
seem to be difficult to implement. The UGV claims to move in uneven
terrain and every harvesting robot aims to replace 4 workers in the
plantation significantly cutting down recurring costs on worker salaries.

Path planning has been done using A star algorithm to detect obstacles
like rocks and stray animals and figure out the most optimum path. A non
intrusive method for measuring the weight of the brinjal is through its
dimensions considering it a cylindrical shape. Prior research on this has
shown that the weight can be estimated to an accuracy of 90%.