You are on page 1of 9

well as the limited potential market Glancey, J.L., W.E. Kee, T.L.

Wootten,
for specialty harvesters for these minor and D.W. Hofstetter. 1998. Feasibility of Engineering and
once-over mechanical harvest of processing
crops.
Clear interactions exist between squash. ASAE Paper No. 98-1093. Amer. Horticultural
Soc. Agr. Eng., St. Joseph, Mich.
the cultivar, cultural practices, a
mechanical harvester and several
Aspects of Robotic
Glancey, J.L., W.E. Kee, T.L. Wootten,
postharvest requirements. As a result, M.D. Dukes, and B.C. Postles. 1996. Field Fruit Harvesting:
a system-level approach is critical losses for mechanically harvested green peas
for developing economically viable, for processing. J. Veg. Crop Production Opportunities and
highly automated vegetable produc- 2(1):61–81.
tion systems. Furthermore, improve- Kahn, B.A., Y. Wu, N.O. Maness, J.B.
Constraints
ments in plant architectures and yields Solie, and R.W. Whitney. 2003. Densely
and other modifications to crops are planted okra for destructive harvest: III.
required before some vegetables can Effects of nitrogen nutrition. HortScience T. Burks1, F. Villegas2,
be machine harvested. Some of the 38(7):1370–1373.
M. Hannan3, S. Flood3,
attributes requiring further develop- Inman, J.W. 2003. Fresh vegetable harvest-
ment include, but are not limited to, ing. Resource: Engineering and Technol- B. Sivaraman3,
better fruit location within the plant ogy for Sustainable World 10(8):7–8.
structure, more uniform fruit sets, in-
V. Subramanian3, and J.Sikes3
Palau, E. and A . Torregrosa. 1997. Me-
creased mechanical damage resistance, chanical harvesting of paprika peppers in
prevention of premature or difficult Spain. J. Agr. Eng. Res. 66(3):195–201. ADDITIONAL INDEX WORDS. selective
fruit detachment, and more robust harvesting, automated production,
postharvest quality and stability. Roberson, G.T. 2000. Precision agriculture machine vision
The integration of new tech- technology for horticultural crop produc-
tion. HortTechnology 10(3):448–451. SUMMARY. Automated solutions for
nologies including DGPS, automatic fresh market fruit and vegetable
machine guidance, and computer- Upadhyaya, S.K., U. Rosa, M. Ehsani, M. harvesting have been studied by
based vision systems offers significant Koller, M. Josiah, and T. Shikanai. 1999. numerous researchers around the
performance benefits, and is a substan- Precision farming in a tomato production world during the past several decades.
tial component of current vegetable system. ASAE Paper No. 99-1147. Amer. However, very few developments have
production and harvesting research in Soc. Agr. Eng., St. Joseph, Mich. been adopted and put into practice.
the U.S. As costs continue to decrease The reasons for this lack of success are
U.S Dept. Agr. 2004. Vegetables at a
due to technical, economic, horti-
for these new technologies, commercial glance: Area, production, value, unit value,
cultural, and producer acceptance
adoption will increase. trade and per capita use. 25 June 2004.
issues. The solutions to agricultural
<http://www.ers.usda.gov/briefing/veg-
robotic mechanization problems are
etables/vegpdf/VetAtAGlance.pdf>.
Literature cited multidisciplinary in nature. Although
Arndt, G., W.M. Perry, and R. Rudziejew- Vassallo, M., E. Benson, and W.E. Kee. there have been significant technol-
ski. 1994. Advances in robotized asparagus 2002. Evaluation of multispectral im- ogy advances during the past decade,
harvesting, p. 261–266. Proc. 25th Intl. ages for harvester guidance. ASAE Paper many scientific challenges remain.
Symp. Industrial Robots. No. 02-1202. Amer. Soc. Agr. Eng., St. Viable solutions will require engi-
Joseph, Mich. neers and horticultural scientists who
Arndt, G., R., R. Rudziejewski, and understand crop-specific biological
V.A. Stewart. 1997. On the future of Wall, M.W., S. Walker, A. Wall, E. Hugh- systems and production practices,
automated selective asparagus harvesting sand, and R. Phillips. 2003. Yield and qual- as well as the machinery, robotics,
technology. Computers Electronics Agr. ity of machine-harvested red chile peppers. and controls issues associated with
16(2):137–145. HortTechnology 13(2):296–302. the automated production systems.
Wu. Y, B.A. Kahn, N.O. Maness, J.B. Focused multidisciplinary teams are
Cho, S.I., K.J. An, Y.Y. Kim, and S.J.Chang.
Solie, R.W. Whitney, and K.E. Conway. needed to address the full range of
2002. Development of a three-degrees-of-
2003a. Densely planted okra for destructive commodity-specific technical issues
freedom robot for harvesting lettuce using
harvest: II. Effects on plant architecture. involved. Although there will be com-
machine vision and fuzzy logic control.
HortScience 38(7):1365–1369. mon technology components, such
Biosystems Eng. 82(2):143–149.
as machine vision, robotic manipula-
Glancey, J.L. 2003. Machine design for veg- Wu. Y, B.A. Kahn, N.O. Maness, J.B.
etable production systems, p. 1105–1115. Solie, R.W. Whitney, and K.E. Conway. 1
PhD, PE, Assistant Professor, University of Florida,
In: D.R. Heldman (ed.). The encyclopedia 2003b. Densely planted okra for destructive 225 Frazier-Rogers Hall, PO Box 110570, Gainesville,
harvest: I. Effects on yield. HortScience FL 32611-0570. To whom reprint requests should be
of agricultural, food and biological engi- addressed. E-mail: TFBurks@ifas.ufl.edu
neering. Marcel Dekker, New York. 38(7):1360–1364.
2
Postdoctorate, University of Florida, Agricultural and
Glancey, J.L., W.E. Kee, T.L. Wootten, Biological Engineering Department, Gainesville.
and M.D. Dukes. 2004. Effects of plant 3
PhD Candidate, University of Florida, Agricultural and
architecture on the mechanical recovery Biological Engineering Department, Gainesville.
of bush-type vegetable crops. ASAE Paper Acknowledgments. Research conducted at the Uni-
No. 041024. Amer. Soc. Agr. Eng., St. versity of Florida, Institute of Food and Agricultural
Sciences, through funding provided by the Florida
Joseph, Mich. Department of Citrus. Special thanks are given to M.
Wilder for her help in editing this manuscript. This
is a Florida Agricultural Experiment Station Journal
Series R-09821.


January–March 2005 15(1) 79

Jan2005HT.indb 79 12/6/04 4:35:15 PM


WORKSHOP

tion, vehicle guidance, and so on, each of robotic mechanization for horticul- crops requires major design compo-
application will be specialized, due to tural crop harvesting systems. In order nents—machine, variety, and cultural
the unique nature of the biological to provide the reader with sufficient practices. A systems development ap-
system. Collaboration and technology breadth of information, this paper is proach must be followed to insure that
sharing between commodity groups
primarily a literature survey and syn- all three aspects are considered (Sims,
offers the benefit of leveraged research
and development dollars and reduced thesis, which tries to identify the key 1969). The major aspects related to
overall development time for multiple issues that robotic system developers cultural practices that affect fruit and
commodities. This paper presents an and horticultural scientists should vegetable mechanical harvesting in-
overview of the major horticultural consider to optimize plant–machine clude field conditions, plant population
and engineering aspects of robotic system performance. and spacing, and plant shape and size.
mechanization for horticultural crop Efficient harvest mechanization cannot
harvesting systems. Horticultural aspects of be achieved by machine design alone.
robotic harvesting Establishing favorable field conditions
Robotic solutions for fresh market for the harvesting system under devel-

S
everal horticultural commodity fruit and vegetable harvesting have opment has to be considered before
groups around the nation are been studied by numerous research- the harvesting system can be effectively
facing growing global market ers around the world during the past developed (Wolf and Alper, 1983).
pressures that threaten their long- several decades. However, very few Peterson et al. (1999) developed
term viability. For instance, Brazilian developments have been adopted and a robotic bulk harvesting system for
orange (Citrus sinensis) growers can put into practice. The reasons for this apples. They trained the apple trees
produce, process, and ship juice to lack of success are due to technical, using a Y-trellis system and found them
Florida markets cheaper than can economic, horticultural, and pro- to be compatible with the mechanical
Florida growers. In the event that tariffs ducer acceptance issues. In industrial robotic harvesting. Fruit was trained to
are eliminated, numerous horticultural automation applications, the robots’ grow on the side and lower branches to
commodities across the nation will not environment is designed for optimal improve fruit detection and removal.
be able to compete in either domestic performance, eliminating as many They further suggested that pruning
or international markets with their variables as possible through careful could enhance the harvesting process
counterparts in Latin America and systems planning. In agricultural set- by removing unproductive branches
Asia. The combination of low com- tings, environmental and horticultural that block effective harvesting. Further
modity prices both domestically and control can be a significant hurdle to research was suggested to determine
abroad, high labor prices, and low successful automation. Not only must the variety and rootstock combinations
labor productivity presents significant the plant system be designed for suc- most compatible with the training and
challenges for U.S. agriculture. Several cessful automation, but the cultural and harvesting system.
commodity groups, including Florida horticultural practices employed by the The concept of designing a grove
tomato (Lycopersicon esculentum) and producers must often be changed to for optimal economic gain is being
orange, California citrus (Citrus spp.), provide a plant growth environment in considered for citrus production. In the
New York apple (Malus ×domestica), which robotic systems can be success- model grove concept, a grove must be
and northwestern U.S. deciduous tree ful. According to Sarig (1993), “The designed for the optimal combination
fruits recognize that harvesting labor major problems that must be solved of varieties, rootstocks, grove layout,
is one of the most crucial challenges with a robotic picking system include production practices, and harvesting
to maintaining economic viability for recognizing and locating the fruit, and methodologies, which will provide
U.S. horticultural crops. According to detaching it according to prescribed maximum economic yield.
economic studies, harvesting labor rep- criteria, without damaging either the PLANT POPULATION AND SPACING.
resents over 40% of citrus production fruit or the tree. In addition, the ro- Harvesting equipment can operate
cost and will need to be cut by 50% in botic system needs to be economically at maximum productivity when the
order to maintain global competitive- sound to warrant its use as an alterna- workspace has been organized to mini-
ness (Brown, 2002). tive method to hand picking.” If the mize inefficient obstacles, standardize
The potential societal benefits plant growth systems can be modified fruit presentation, provide sufficient
from agricultural robotic mechani- to improve harvestability, the robotic alleyways, and maximize fruit density
zation are numerous. By sustaining system will have a much better chance on uniform growth planes.
crucial commodities, the economic of being successful. Certain tree species and even
infrastructure which supports these Modifications and improvements certain varieties within species have an
industries will be reinvigorated. Rural of cultural practices for mechanization optimal subsistence area for best fruit
communities will have new oppor- are continually being made through production, which provides a proper
tunities for better jobs that have less research and experience (Sims, 1969). ratio between the number of leaves
drudgery than traditional manual In order to have a successful auto- needed to produce carbohydrates and
field labor. Opportunities to improve mated/mechanized system, the cul- other organic compounds, and the
worker health and safety by automating tural practices must be designed for number of developing fruits (Monselise
dangerous operations have significant the machine and the variety (Davis, and Goldschmidt, 1982). The woody
potential. 1969). Cultural practices are a critical mass—roots, trunk, scaffolds, and
The objective of this paper is factor in mechanization of fruit and branches—supports the tree canopy,
to present an overview of the major vegetable production and harvesting. but contributes minimally toward fruit
horticultural and engineering aspects Mechanization of fruit and vegetable development once nutrient uptake and

80 ●
January–March 2005 15(1)

Jan2005HT.indb 80 12/6/04 4:35:16 PM


moisture demands are met. However, pruning. The trees can be pruned to plant interior. In both cases, a plant
the woody mass continues to use the the desired shape before fruit set and system that presented the majority
tree’s resources to maintain itself, allowed to grow during the remainder of the fruit at the canopy surface
presenting obstructions to robotic har- of the year. In some limited cases, severe would improve harvestability. There
vesting. Ben-Tal (1983) suggested that pruning is being tested. Under this are two possible solutions. The first
maximum yield per unit area would be practice, alternating sides of the tree would suggest a thin leaf canopy so
achieved by a large number of relatively are pruned each year and allowed to that the detection systems could more
small trees, suggesting that smaller set fallow, while the other side of the easily view the plant interior, and the
robotic systems may actually provide tree produces the current year’s crop. second suggests a dense canopy that
a better economic return. When the canopy returns the following might force more fruit to grow at the
Scalability of robotic systems is year, the woody mass is covered by the surface. The two strategies seem to be
an important economic factor, which new growth and a relatively uniform in conflict under normal tree behavior.
impacts the design of the plant growth vertical wall is achieved. Impact of an- Sparsely leafed trees tend to have more
system. The productivity of large mul- nual fruit yield has not been reported interior fruit, which reduces fruit ac-
tiple-arm systems vs. smaller, more ag- on this technique to date. cessibility, while densely leafed trees
ile, human-like robots is an important Experiments conducted on ap- will make it more difficult to sense the
economic question. Large equipment ples demonstrated that tree shape interior fruit. A tree which is naturally
systems require wide row spacings contributes toward the suitability of fruited at the limb extremities with
while smaller systems can work in a mechanical harvesting (Zocca, 1983). minimal interior fruit might resolve
more confined grove configuration. Modifications to cultural practices this problem.
Optimally, the fruit should be grown for growing and harvesting fruit are Another primar y concern is
in a hedge row configuration where the important for successful mechanical canopy uniformity. Factors affect-
plants produce a maximum number of harvesting. A mechanized pruner was ing uniformity in emergence, stand,
fruits over the surface area (Ben-Tal, developed that not only reduced the growth, and maturity must be clearly
1983). This suggests that the trees labor required for pruning, but also understood in order to develop viable
or plants be grown at a close spacing properly shaped the hedgerow for plant systems for mechanical harvesting
so that the growth plane is uniform, maximum harvesting efficiency of erect (Davis, 1969). Cultural practices have
with minimal scalloping of the hedge cane fruits (Morris, 1983). been discussed that could produce a
between plants. Ben-Tal (1983) pointed out sev- hedge-row system. However, trees
PLANT SHAPE AND SIZE. The ideal eral problems that can arise when an that require severe hedging to main-
configuration for efficient robotic har- orchard is prepared through pruning tain their shape often develop woody
vesting would be a vertical or slightly for a specific kind of equipment, such structures near the surface, which could
inclined hedge wall, 10 to 12 ft (3.0 to as reduced yield, fruit quality, and the be an obstacle to robotically harvesting
3.7 m) tall, that is relatively uniform, number of years of production. Ad- interior fruit. A tree that grew to an
smooth, and continuous from start to ditional issues, such as canopy light appropriate mature height and shape
row end. The fruit would be located exposure and maximum height of a and then maintained its size with ei-
on the canopy surface with minimal tree for proper spraying, pruning, etc., ther minimal hedging or woody mass
occlusion. In reality this would not should be considered. The question of build-up would be ideal.
be the case, but the example provides plant geometry and its relationship to Several plant breeding projects
some insight into what a robot would productivity needs to be thoroughly have contributed favorably to mechani-
need in order to maintain fast harvest examined (Rohrbach, 1983). cal harvesting. Peach (Prunus persica)
cycle times and maximum fruit re- TREE GENETICS FOR OPTIMAL breeders increased fruit harvest by
moval. Deviations from the ideal will HARVESTING. Plant breeders developing releasing varieties with varying maturi-
cost removal efficiency and cycle time new varieties of fruit must consider if ties, effectively doubling or tripling the
performance. the variety will be accepted at market length of the peach season in many
Orchards should have uniform and if it will be durable under machine production areas (Carew, 1969).
plant sizes and predictable shapes for handling. Attractive appearance and Dwarfing rootstocks in combination
efficient robotic harvesting (Cargill, long shelf life are imperative in the fresh with apple varieties have provided size
1983). Standardization of tree sizes market. Varieties must be resistant to control of apple trees. Plant improve-
would significantly improve harvesting bruising, cracking, and rupturing dur- ment through breeding can modify
throughput and thus economic benefit. ing machine handling. The fruit must crop characteristics and assist in the
These standard sizes should consider be relatively easy to remove from the introduction of mechanical harvesting
tree height, tree thickness, tree shape, plant and the peduncle must remain systems (Carew, 1969).
and tree spacing within and between attached (Davis, 1969; Lapushner et
rows so that the robotic equipment al., 1983). Engineering design aspects of
can maintain continuous harvesting, In addition to fruit-related issues, robotic harveting
with minimal idle harvest time when there are a number of tree factors Robotic systems developers from
traveling between trees. A number of that can be improved genetically that the U.S., Europe, Israel, and Japan have
these features are designed into the can enhance robotic harvestability. conducted independent research and
grove at planting, while others must Two major obstacles impede efficient development on harvesting systems for
be maintained mechanically. A com- robotic harvesting: 1) locating fruit apples and citrus achieving harvesting
mon modern approach for maintaining occluded by the leaf canopy; and 2) efficiencies of 75%. These low levels
both tree size and shape is mechanical harvesting fruit located in the tree or of performance were attributed to


January–March 2005 15(1) 81

Jan2005HT.indb 81 12/6/04 4:35:16 PM


WORKSHOP

poor fruit identification and the in- varieties (C. unshiu) tended to be more still, however, susceptible to injury.
ability to negotiate natural obstacles susceptible to undesirable detachment Injury is more prevalent in less mature
inside the tree canopy (Sarig, 1993). conditions than other citrus varieties. oranges, as was found by Juste et al.
Harvesting cycle times for citrus were ‘Clausellina’ exhibited 8% to 19% (1988). The resistance to pressure
estimated at 2 s/fruit for a two-arm calyxless conditions, depending on was found to stabilize as the oranges
machine (or 4 s/fruit for a single- fruit maturity, while ‘Navelina’ was matured at around 343.2 kPa (49.78
arm machine). Cycle time should be relatively constant at 2% to 3% calyx- psi) for ‘Salustiana’ and 441.3 kPa
higher for apples due to a more open less. Fornes et al. (1994) also reported (64.01 psi) for ‘Washington’ using a
canopy (Sarig, 1993). These levels of that detachment method and damage circular surface area of 1.1 cm2 (0.17
harvesting performance and economic varied with rotational speed for the inch2). Rind penetration tests were
return prevented producer acceptance same three varieties. also performed using a 4.7-mm-diam-
of robotic harvesting. Juste et al. (1988) researched the eter (0.19 inch) punch. Penetration
PHYSICAL PROPERTIES AND FRUIT detachment forces of ‘Salustiana’ (C. si- force was found to be 28.9 N (6.50
REMOVAL. A robotic harvester must nensis) and ‘Washington’ navel oranges lbf) for ‘Salustiana’ oranges and 26.6
be able to quickly remove the fruit (C. sinensis). ‘Salustiana’ was found to N (5.98 lbf) for ‘Washington’ navel
without damaging the fruit or the tree. have a detachment force 73.1 ± 0.6 N oranges.
An integral part of the harvester is the (16.43 ± 0.13 lbf) and ‘Washington’ When manually harvesting or-
end-effector, which is a tool or device was found to have a detachment force anges the fruit is detached using one
attached to the end of the manipulator of 54.4 ± 4.7 N (12.23 ± 1.06 lbf). The of three methods, depending on the
that grabs and removes the fruit from detachment force of ‘Washington’ was variety and cultural practice. The la-
the tree. Because of its direct interac- found to substantially increase with an borer can use a set of clippers to detach
tion with the fruit and tree structure, increase in maturity. All of these forces the fruit, usually leaving as short a
it must be designed with the specific were measured along the stem axis. stem as possible. Secondly, the laborer
physical properties of the commodity Fornes et al. (1994) also researched can lift the fruit so that the stem axis
to be harvested in mind. detachment forces on ‘Clausellina’ and is rotated 90° and then pull down so
There are several ways that a robot ‘Clemenules’ mandarins, and ‘Nave- that the force is perpendicular to the
might damage the fruit or tree: 1) end- lina’ oranges. Detachment forces for all stem axis. Lastly, the laborer can add a
effector applying excessive positive or three varieties were found to decrease twisting motion to the second method.
negative pressure or force to the fruit as maturity increased. Although the end-effector does not
during pick and place operations; 2) Juste et al. (1988) measured necessarily have to follow one of these
inappropriate stem separation tech- torsional detachment by counting the methods, an understanding of manual
niques for the type of fruit; 3) fruit number of turns required to detach the procedures gives insight into some of
damage during retraction from the tree fruit. ‘Salustiana’ displayed an average the potential methods.
canopy or conveyance to bulk storage; twist of 2.48 ± 0.12 revolutions and The first type of robotic orange
or 4) manipulator contact with the ‘Washington’ displayed an average harvesting end effectors that has been
tree structure. Fruit damage may not twist of 2.36 ± 0.11 revolutions. The developed is the cutting end-effector.
be physically evident immediately, yet maximum twist was 4.75 revolutions Several cutting end-effector designs
bruising, scratches, cuts, or punctures for both varieties. Approximately 60% have been developed as described in
will result in decreased shelf life. A of ‘Salustiana’ still had a stem, which Ito (1990), Sarig (1993), Pool and
properly designed end-effector will was on average 3.82 ± 0.36 mm (0.150 Harrell (1991), and Bedford et al.
attempt to minimize fruit damage. ± 0.014 inch) in length. Approximately (1998). This method is prevalent in
The fruit removal technique em- 78% of ‘Washington’ navel oranges several agricultural applications since
ployed is typically the largest cause of still had a stem, which was on aver- it produces the least amount of stress
fruit injury. In the case of oranges, the age 6.33 ± 1.14 mm (0.249 ± 0.044 on the actual fruit. The basic premise
fruit must be harvested with the calyx inch) in length. Rabatel et al. (1995) is to first capture the fruit using a
intact and the stem removed flush with stated that a stem length of 5.0 mm suction cup or gripper, and then use
the calyx. If the peel is torn away from (0.20 inch) or less was desirable. It a cutting device to sever the stem that
the caylx, the resulting fruit is unus- should be noted that in these tests the is holding the fruit onto the tree. This
able for the fresh fruit market due to calyx remained intact on the torsional can either be done blindly by swing-
contamination and reduced shelf life. detachments, but 70% of the direct ing a blade around the outer edge or
This condition is referred to as “plug- pulling detachments had calyx separa- by detecting the stem’s location and
ging.” If a long stem remains on the tion or displayed “plugging.” Coppock cutting it with a scissor device. The
fruit, the packer will either reject the (1984) observed a near 50% reduction stem’s location can either be detected
fruit or require stem removal post- in pulling detachment force when the through machine vision or through
harvest. Fornes et al. (1994) reported force was applied at a 90° angle from force/torque sensors.
citrus detachment conditions for fruit the stem axis for ‘Valencia’ oranges (C. In the blind system a blade passes
removed using a prototype vacuum- sinensis). This reduction in detachment around the encased fruit to sever the
grip rotational-separation end-effector. force might decrease the amount of stem without damaging adjacent fruit
Fruit detachment conditions varied plugging exhibited. or the tree. The blade must be large
with maturity for the three citrus va- The rind of oranges makes them enough to encircle the fruit, and must
rieties reported: ‘Clausellina’ (Citrus one of the more durable fruits, in maintain sharpness to achieve a clean
unshiu), ‘Clemenules’ (C. unshiu), and contrast with more delicate-skinned cut. The scissor method reduces the
‘Navelina’ (C. sinensis). The mandarin products, such as tomatoes. They are chance of fruit damage but is sub-

82 ●
January–March 2005 15(1)

Jan2005HT.indb 82 12/6/04 4:35:17 PM


stantially more complex, requiring a In robotic fruit harvesting sys- images that contained 673 oranges that
larger end-effector, more sensors, and tems, machine vision has become one were taken at an actual orange grove.
more time. of the most popular sensing systems Their algorithm correctly identified
The second type of end-effector for fruit identification. With the advent 87% of the oranges, while 15% of the
is the pull and cut end-effector. This of color cameras, differences between detected regions were incorrectly clas-
method was proposed by Pool and leaf canopy and mature fruit can be sified as oranges. Their approach had
Harrell (1991). In this method, the discriminated. The vision system is able difficulty with both brightly and poorly
fruit is grasped either through suction to determine either two-dimensional lit oranges, brightly lit leaves, and
or a type of collection sock. The stem (2D) or three-dimensional (3D) posi- certain types of occlusion. Bulanon et
is severed as the end-effector retracts. tion in the canopy, depending on the al. (2001) presented an algorithm that
This method disturbs the surrounding type of system employed. However, used a 240 × 240 pixel color image to
limb structure, making subsequent the process of separating the desir- detect apples. The apples were detected
harvesting more difficult since the fruit able fruit from other objects, such as by thresholding the image using both
is in motion, and still has some of the leaves, branches, and unripe fruit, is the red color difference and luminance
limitations of the cutting end effectors very difficult due to the large amount values. It was determined that the red
previously mentioned. of information that must be processed. color difference values were much more
The third type of end-effector This is a difficult task even with today’s effective at detecting the apples than
design is the twisting method. This relatively high-speed computers. To the luminance values.
method was suggested by Juste et al. put this into perspective, one 640 × There are three major problem
(1998) and Rabatel et al. (1995) to be 480-pixel color image has almost one areas associated with the use of ma-
the most promising of the three. This million data points, and a standard chine vision-based sensing: 1) partial
involves twisting the fruit, preferably video camera runs at 30 frames per and totally occluded fruit are difficult
perpendicular to its attachment axis, second, resulting in 30 million data to accurately detect; 2) light variability
until the stem is severed. Twisting points to process every second. can result in low detection rates of ac-
the fruit in this manner reduces the Machine vision systems common- tual fruit as well as high levels of false
amount of disturbance to the tree ly applied to agricultural applications detections; and 3) the computational
and thus to the surrounding fruit. acquire reflectance, transmittance, time required to process images as
Twisting involves the least amount of or fluorescence images of the object influences real-time control.
force of the three methods and has the under ultraviolet, visible, or infrared Numerous other sensors are com-
lowest plugging rate. Like the other illumination. A basic machine vision monly employed in robotic harvesting
two types, fruit size is a consideration system includes a camera, optics, light- systems, such as ultrasonic range, laser
here as well. Generally, the twisting ing, a data acquisition system, and a range, capacitive proximity, light emit-
action is achieved by use of a rotating processor. Fujiura (1997) developed ting diode (LED) range, and so on.
suction cup. This cup must be of the robots having a 3D machine vision It is not likely that a single sensor will
right size to create a good seal while system for crop recognition. The vi- solve the complete sensing problem,
still providing enough force to keep sion system illuminated the crop using but several sensors will need to be in-
the orange from slipping. One of the red and infrared laser diodes and used tegrated together to form a complete
major advantages of this method is that three position-sensitive devices to system.
there is a large flexibility in the angle of detect the reflected light. The sensors R OBOTIC MANIPULATION AND
approach. Except at the stem, the cup selected were suitable for agricultural CONTROL. The kinematic and geomet-
can attach to any part of the fruit. robots that are required to measure the ric description of a robotic manipulator
Tutle (1985) suggested an ap- 3D shape and size of targets within a is one of the key tasks in building a
proach that combined the twisting limited measuring range. Jimenez et al. robotic system to work, especially in
and pulling approach in U.S. Patent (2000) developed a laser-based vision unstructured environments. Harvest-
4,532,757. The end-effector design system for automatic fruit recognition ing of fruits (orange, apple, etc.) is
selected for a given application should to be applied to an orange-harvesting highly unstructured and the robotic
be developed in conjunction with the robot. The machine vision system arm should be flexible enough to adapt
manipulator, sensors, and control de- was based on an infrared laser range- to changes in the environment. Various
velopment to optimize the capabilities finder sensor that provides range and geometric coordinate configurations
of the harvester. reflectance images and was designed used in industrial applications are avail-
MACHINE VISION AND SENSING to detect spherical objects in non- able: cartesian, cylindrical, spherical,
TECHNOLOGIES. According to Sarig structured environments. The sensor articulated, and redundant.
(1993), “While major progress has output included 3D position, radius, In a 1968 review of mechanical
been made with the identification of and surface reflectivity of each spheri- citrus-harvesting systems (Schertz and
fruit on the tree and determination of cal target, and had good classification Brown, 1968), the basic principles for
its location, only 85% of the total fruits performance. utilizing robots to pick fruits were laid
on the tree are claimed to be identified. Plebe and Grasso (2001) pre- out. The earliest laboratory prototype
Variability in lighting conditions and sented a color-based algorithm for was an apple harvester (Parrish and
obscurity of fruits because of leaf and detecting oranges and determining the Goksel, 1997) consisting of a simple
branch coverage (especially in citrus target centers. They also applied stereo arm with a pan-and-tilt mechanism
trees), require further development imaging to these processed images to and a touch sensor in place of an end-
of identification techniques, or major determine the range to the detected effector, which made contact with
changes in tree shape.” fruit. They presented results from 50 modeled fruit.


January–March 2005 15(1) 83

Jan2005HT.indb 83 12/6/04 4:35:17 PM


WORKSHOP

The first field prototype for har-


vesting apples was developed in France
(Grand D’Esnon, 1985). The mechani-
cal system consisted of a telescopic arm
that moved up and down in a vertical
framework. The arm was mounted on
a barrel that could rotate horizontally.
In 1986, a new prototype (MAGALI)
was built (Grand D’Esnon et al., 1987)
that used a spherical manipulator ser-
voed by a camera set at the center of
the rotation axes. Figure 1 shows the
spherical manipulator that can execute
a pantographic prismatic movement
(only rotational joints) along with two
rotations.
In 1986, the University of Florida,
along with other collaborators, initi-
ated a program to develop a robotic
system for citrus harvesting (Harrell et
al., 1988). The outcome of this research
was a 3-degree of freedom (DOF) ma-
nipulator actuated with servo-hydraulic
drives. Joints 0 and 1 were revolute and
joint 2 was prismatic. This geometry
was characteristic of a spherical coordi-
nate robot (Fig. 2). The feasibility of a Fig. 1. MAGALI apple-picking arm developed by Grand D’Esnon et al. (1987).
robotic citrus harvester was ascertained
by this research work.
The French-Spanish Eureka
project (Rabatel et al., 1995), started
in 1991, was based on the feasibility
study done at the University of Florida.
The proposed robotic system had a
dual harvesting arm configuration
to achieve greatest economic return;
however, the prototype consisted of
only one harvesting arm. The arm had
two modules, an elevating arm and a
picking arm. The picking arm was of
a pantographic structure rather than a
linear structure.
The elevating arm supported the
picking arm and the associated camera.
The elevating arm was equipped with a
lateral DOF to avoid collision with the
picking arm with the vegetation, while
acting as a fruit conveyor as well.
Several manipulator architectures
Fig. 2. Citrus-picking robot developed by Harrell et al. (1988).
have been attempted for fruit harvest-
ing. Of these, the articulated joint (6
DOF) seems to work the best, since reduced burden for the operator. Ve- with success, based on the degree of
it closely resembles a human arm. In hicle position, heading, steering effort, accuracy required in the navigation
order to avoid obstacles and to harvest and speed with respect to the desired system. There is a tradeoff between
interior canopy fruit, the optimal con- path are the most important issues that accuracy and cost in the selection of
figuration for a robotic harvester may must be considered. Global positioning DGPS and RTKGPS, with the latter
require more degrees of freedom than systems (GPS) in combination with being more accurate and expensive.
a standard articulated joint. inertial navigation systems have been RTKGPS has been giving very accurate
AUTONOMOUS VEHICLE GUID- widely used as positioning and heading results (Benson et al., 2001; Nagasaka
ANCE. Autonomous vehicles are being sensors in traditional field agriculture et al., 2002; Noguchi et al., 2002).
developed for several applications, application. Both realtime kinematic Gyros have been widely used for in-
including agricultural vehicles. The GPS (RTKGPS) and realtime differ- clination measurement (Mizushima et
major advantages are precision and ential GPS (DGPS) have been tested al., 2002). Fiberoptic gyro (FOG) has

84 ●
January–March 2005 15(1)

Jan2005HT.indb 84 12/6/04 4:35:18 PM


given the best performance (Nagasaka al. (1992) pointed out that the fuzzy there are still major technological chal-
et al., 2002). At present, gyros and controller could achieve better tracking lenges that need to be addressed, such
inclinometers are available together as performance than the PI controller. It as locating occluded fruit, reaching the
inertial measurement units (IMU) for has wider adaptability to all kinds of interior canopy, maintaining high cycle
pitch, roll and yaw, and linear velocity inputs. Qiu et al. (2001) verified that rates in difficult harvesting conditions,
measurements. With the combination the fuzzy steering control provided cost affordability, and maintainability
of RTKGPS and FOG, accuracy of a prompt and accurate steering rate of high-tech equipment.
±5 cm (2.0 inches) has been achieved control on the tractor. Kodagoda et al. The current citrus robotics devel-
(Noguchi et al., 2002). GPS cannot (2002) found fuzzy control to be better opment initiative, which is being led by
be used alone for positioning in citrus than PID for longitudinal control. PID the University of Florida, is seen as a
applications, as it gives errors when the was also found to have large chatter, multidisciplinary effort where horticul-
vehicle moves under tree canopies. high saturation. A combination of turalists, economists, and agricultural,
In addition to sensing global fuzzy and PID control holds significant mechanical, and electrical engineers are
positions, the vehicle must be able to promise (Benson et al., 2001). Efficient working together to solve some of the
detect local obstacles that may impede guidance can be achieved using a fuzzy- complex problems mentioned previ-
the path. Several sensing technologies PID control system with vision, laser ously. From a horticultural perspective,
have been explored for this task. Ul- radar and IMU as sensors. research has begun to develop special
trasonic sensors can map tree canopies orange varieties, which may be more
while traveling at speeds of 1.8 m·s–1 Results and discussion favorable toward robotic harvesting.
(5.91 ft/s); measurement accuracy is In Summer 2001, the Florida Additionally, research is beginning on
better at lower speeds (Iida and Burks, Department of Citrus began an in- the development of a model grove,
2002). The development of machine vestigation into the potential for using which will attempt to optimize the
vision guidance techniques has become robotics to harvest citrus. Current grove design and cultural practices for
a very attractive sensing alternative, mass harvesting programs have proven mechanical/robotic harvesting and
especially when combined with other viable for process citrus, but cannot maximum economic potential. From an
proximity-based sensors (Benson et be used for fresh fruit markets and economic standpoint, studies are pro-
al., 2001; Zhang et al., 1999). They remain questionable for late season posed that will evaluate the production
have proven to be reliable in several ‘Valencia’, pending abscission chemical system for optimum economic return
row-crop applications, but have not development and approval. A fact-find- on investment. The potential labor
performed well in sparsely populated ing team evaluated past horticultural productivity gains, projected capital
crops. Their reliability reduces with robotics efforts, and talked to experts equipment cost, harvesting efficiency,
low lighting, shadows, dust, and fog. in robotics, agricultural mechaniza- value added benefits, and long-term
Benson et al. (2001) overcame this tion, horticulture, and economics to yield impacts of robotic systems will be a
by using artificial lighting. Laser radar determine if there had been sufficient few of the factors that will be considered
has been used for ranging and obstacle advances in technology and changes in evaluating the economic potential.
avoidance. It has higher resolution in the economic potential for robotic In terms of engineering development
than ultrasonic sensing, and requires harvesting to suggest that a renewed issues, the primary thrusts will be in
fewer computations than vision. Its effort was warranted. The consensus the end-effector, manipulator system,
performance degrades with dust and opinion of a Forum on Robotic Citrus sensing technology, material handling,
rain like vision and it is costlier than Harvesting, held in Apr. 2002, was that vehicle guidance, and machine intel-
ultrasound. It provides planar data there was an urgent need for harvest- ligence development.
of the path, but can generate 3D by ing solutions for the fresh fruit market, There is a growing interest among
rotating the laser source to give a 3D that significant long-term financial researchers working in other tree fruit
view. O’Connor et al. (1995) found commitment would be required, and industries, along with their growers, to
that sensor data is noisy, and can be although it is a difficult problem, pursue automation solutions to reduce
filtered using Kalman filters to obtain enough technical progress has been the increasing disparity between U.S.
robust sensor fusion. made in the past decade to warrant a production labor cost and those of
Steering control is a major factor new robotics program. developing countries. However, it is
for accurate guidance. PID control During the past two decades, clear that novel approaches need to
(proportional, integral, derivative) has since the beginnings of research in be taken to solve robotics technology
given satisfactory performance (Zhang agricultural robotics, there have been problems, as well as the manufacturing
et al., 1999). Neural networks have numerous technological advances. and maintenance challenges that will
the inherent disadvantage of learning The speed of computers has increased surface as high-tech equipment systems
only what the driver does, so they are exponentially over the last 10 years. are implemented in harsh agricultural
not robust. Behavior-based control is Manipulator technologies have im- environments. These challenges will
a new development that has been suc- proved so that current manipulators require a high level of cooperation
cessfully used in small mobile robots. A are faster and more dexterous. Machine among engineers, researchers, indus-
behavior-based system in combination vision and other sensing technologies, try, and growers to insure that these
with a real-time control system (RTCS) along with image and signal processing new systems will be able to meet their
is expected to do well in vehicle guid- technologies, have greatly advanced. design objectives. An additional major
ance. Fuzzy control has recently been Our understanding of horticultural issue is safety, since agriculture has been
tried, with results comparable with practices and their impact on harvest- a notoriously dangerous workplace.
PID (Benson et al., 2001). Senoo et ability has greatly improved. However, Thoughtful design and appropriate


January–March 2005 15(1) 85

Jan2005HT.indb 85 12/6/04 4:35:20 PM


WORKSHOP

industry standards will be required to ogy sharing between commodity Grand D’Esnon, A., G. Rabatel, R. Pel-
build safety devices into the automated groups will offer the benefit of lever- lenc, A. Journeau, and M.J. Aldon. 1987.
systems that will protect operators and aged research and development dollars MAGALI: A self-propelled robot to pick
casual observers. and reduced overall development time apples. Amer. Soc. Agr. Eng. Paper No.
87-1037.
Some expected outcomes are the for multiple commodities.
creation of new jobs associated with the Harrell, R.C., P.D. Adsit, T.A. Pool, and
manufacture, operation, and service Literature cited R. Hoffman. 1988. The Florida robotic
of machinery and instrumentation; grove-lab. Amer. Soc. Agr. Eng. Paper
Bedford, R. Ceres, J.L. Pons, A.R. Jimenez, No. 88-1578.
improvement of working conditions, J.M. Martin, and L. Calderon. 1998.
and ability of laborers to become more Design and implementation of an aided Ito, N. 1990. Agricultural robots in Japan.
skilled; and elimination of monotonous fruit-harvesting robot (Agribot). Ind. Inst. Electrical and Electronics Eng. Intl.
and onerous tasks, such as heavy lifting, Robot 25(5):337–346. Wkshp. Intelligent Robots and Systems.
digging, handling of sharp objects, and Ibaraki, Japan 1:249–253.
Benson, E.R., J.F. Reid, and Q. Zhang.
repetitive tasks. 2001. Machine vision based steering system Iida, M. and T.F. Burks. 2002. Ultrasonic
Initial optimistic estimates have for agricultural combines. Amer. Soc. Agr. sensor development for automatic steering
suggested that a 7- to 10-year multi- Eng. Paper No. 01-1159. control of orchard tractor. Proc. of the
million dollar program will be required Conf. Automation Technol. for Off-Road
to bring forth a market-ready citrus Ben-Tal, Y. 1983. Horticultural aspects Equipment. Amer. Soc. Agr. Eng. Publ.
of mechanical fruit harvesting. Proc. Intl. 701P0502:221–229.
harvesting system. It is recognized Symp. on Fruit, Nut, and Veg. Harvest-
that significant federal funding will be ing Mechanization. Amer. Soc. Agr. Eng. Jiminez, A.R., R. Ceres, and J.L. Pons.
required to sustain the program for the SP-5:372–375. 2000. Vision system based on a laser
duration of the development process. range-finder applied to robotic fruit har-
Even with major federal research dol- Brown, G.K. 2002. Mechanical harvest- vesting. Machine Vision and Applications
lars, there is a degree of uncertainty ing systems for the Florida citrus juice 11(6):321–329.
industry. Amer. Soc. Agr. Eng. Paper No.
regarding our ability to overcome all 021108. Juste, F., C. Gracia, E. Molto, R. Ibanez,
technological barriers to high-speed and S. Castillo. 1988. Fruit bearing zones
and high-efficiency robotic harvesting. Bulanon, D.M., Kataoka, T., Zhang, S., and physical properties of citrus for me-
The questions still remain of whether or Ota, Y., and Hiroma, T. 2001. Optimal chanical harvesting. Citriculture. Proc.
not these new systems will be accepted thresholding for the automatic recognition Sixth Intl. Congr. Middle-East, Tel Aviv,
by growers and provide an adequate of apple fruits. Amer. Soc. Agr. Eng. Paper Israel 4:1801–1809.
No. 01-3133.
return on investment. If not, these
Kodagoda, K.R.S, W.S. Wijesoma, and
robotics development efforts may meet Carew, J. 1969. The prospects for fruit E.K. Teoh. 2002. Fuzzy speed and steer-
the same fate as the ones of the 20th and vegetable mechanization: Horticul- ing control of an AGV. Inst. Electrical and
century. One major factor looms on ture outlook. Fruit and vegetable harvest Electronics Eng. Trans. on Control Systems
the horizon: most major automation mechanization. Rural Manpower Ctr., Technol. 10(1):112–120.
and mechanization efforts have been Michigan State Univ., East Lansing. Rpt.
No. 16:79–83. Lapushner, D., R. Frankel, and E. Edel-
driven by high labor cost and/or in-
man. 1983. Genetical and cultural aspects
adequate labor supply, both of which Cargill, B.F. 1983. Harvesting high density for mechanical harvested fresh market
could become reality. red tart cherries. Proc. Intl. Symp. on Fruit, tomatoes. Proc. Intl. Symp. on Fruit,
Nut, and Veg. Harvesting Mechanization. Nut, and Veg. Harvesting Mechanization
Conclusions Amer. Soc. Agr. Eng. SP-5:195–200. 5(12):404–407.
The solutions to agricultural Coppock, G.E. 1984. Robotic principles Mizushima, A., N. Noguchi, K. Ishii, and
robotic mechanization problems are in the selective harvest of Valencia oranges. H. Terao. 2002. Automatic navigation of
multidisciplinary in nature. Although Proc. First Intl. Conf. on Robotics and In- the agricultural vehicle by the geomag-
there have been significant technology telligent Machines in Agr., Amer. Soc. Agr. netic direction sensor and gyroscope. Proc.
advances, many scientific challenges Eng., Tampa, Fla. ASAE. p. 138–145. Conf. Automation Technol. for Off-Road
remain. Viable solutions will require Davis, V. 1969. Mechanization of fruit Equipment. Amer. Soc. Agr. Eng. Publ.
engineers and horticultural scientists and pod vegetables introduction. Fruit and 701P0502:204–211.
who understand crop-specific biologi- vegetable harvest mechanization. Rural Monselise, S.P. and E.E. Goldschmidt.
cal systems and production practices, Manpower Ctr., Michigan State Univ., East 1982. Alternate bearing in fruit trees. Hort.
as well as the machinery, robotics, Lansing. Rpt. No. 16:205–213. Rev. 4:155–158.
and controls issues associated with Fornes, I., E. Molto, F. Juste, and A.
automated production systems. Clearly Morris, J.R. 1983. Effects of mechanical
Segura. 1994. Mechanical harvesting of harvesting on the quality of small fruits
focused multidisciplinary teams are citrus: Evolution of damage produced and and grapes. Proc. Intl. Symp. on Fruit,
needed to address the full range of fruit behaviour during post-harvest. EurAg Nut, and Veg. Harvesting Mechanization
commodity-specific technical issues Eng 94. Rpt. N-94-G061. 5(12):332–348.
involved. Although there will be com- Fujiura, T. 1997. Agricultural robots us-
mon technology components, such as Nagasaka, Y., N. Umeda, and Y. Kanetai.
ing 3D vision sensor. Robot. July 1997. 2002. National Agr. Res. Center, Japan:
machine vision, robotic manipulation, 117:32–38.
vehicle guidance, and so on, each ap- Automated rice transplanter with GPS and
Grand D’Esnon, A. 1985. Robotic FOG. Proc. Conf. Automation Technol.
plication will be specialized due to the
harvesting of apples. Proc. Agri-Mation for Off-Road Equipment. ASAE Publ.
unique nature of the biological system. 701P0502:190–195.
1:210–214.
However, collaboration and technol-

86 ●
January–March 2005 15(1)

Jan2005HT.indb 86 12/6/04 4:35:20 PM


Noguchi, N., M. Kise, K. Ishii, and H. Rabatel, G., A. Bourely, F. Sevila, and F. Sims, W.L. 1969. Cultural practices for fruit
Terao. 2002. Field automation using robot Juste. 1995. Robotic harvesting of citrus: vegetables. Fruit and Vegetable Harvest
tractor. Proc. Conf. Automation Technol. State-of-art and development of the French Mechanization. Rural Manpower Ctr.,
for Off-Road Equipment. ASAE Publ. Spanish Eureka Project. Harvest and Post- Michigan State Univ., East Lansing. Rpt.
701P0502:239–245. harvest Technol. for Fresh Fruits and Veg.: No. 16:225–237.
Amer. Soc. Agr. Eng. Proc. Intl. Conf.,
O’Connor, M., G. Elkaim, Jr., and B. Par- Guanajunto, Mexico: 232–239. Tutle, E. 1985. Robotic fruit harvester.
kinson. 1995. Kinematic GPS for closed- U.S. Patent No. 4,532,757. U.S. Patent
loop control of form and construction Rohrbach, R.P. 1983. Mechanized main- Office, Washington, D.C.
vehicles. Stanford Univ., Stanford, Calif. tenance of blueberry quality. Proc. Intl.
Paper No. 10NGPS-95. Symp. on Fruit, Nut, and Veg. Harvest- Wolf, I. and Y. Alper. 1983. Mechaniza-
ing Mechanization Amer. Soc. Agr. Eng. tion of paprika harvest. Proc. Intl. Symp.
Parrish, E.A., Jr., and A.K. Goksel. 1997. SP-5:134–140. on Fruit, Nut, and Veg. Harvesting
Pictorial pattern recognition applied to fruit Mechanization Amer. Soc. Agr. Eng. SP-
harvesting. Trans. ASAE 20(5):822–827. Sarig, Y. 1993. Robotics of fruit harvesting: 5:265–275.
A state-of-the-art review. J. Agr. Eng. Res.
Peterson, D.L., B.S. Bennedsen, W.C. 54(4):265–280. Zhang, Q., J. Reid, and N. Noguchi. 1999.
Anger, and D. Wolford. 1999. A systems ap- Automatic guidance control for agricul-
proach to robotic bulk harvesting of apples. Schertz, C.E. and G.K. Brown. 1968. tural tractor using redundant sensor. Soc.
Trans. of the ASAE 42(4):871–876. Basic consideration in mechanizing cit- Automation Eng. Trans., J. Commercial
rus harvest. Trans. Amer. Soc. Agr. Eng. Vehicles (108):27–31.
Plebe, A. and G. Grasso. 2001. Localiza- 11:343–346.
tion of spherical fruits for robotic harvest- Zocca, A. 1983. Shake harvesting trials with
ing. Machine Vision and Applications. Senoo, S., M. Mino, and S. Funabiki. 1992. market apples. Proc. Intl. Symp. on Fruit,
13(2):70–79. Steering control of automated guided Nut, and Veg. Harvesting Mechanization
vehicle for steering energy saving by fuzzy 5(12):65–72.
Pool, T.A. and R.C. Harrell. 1991. An reasoning. Proc. 1992 Inst. Electrical and
end-effector for robotic removal of citrus Electronics Eng. Ind. Applications Soc.
from the tree. Trans. ASAE 34(3/4):373– Annu. Mtg.:1712–1716.
378.
Qiu, H., Q. Zhang, and J. Reid. 2001.
Fuzzy control of electrohydraulic steering
systems for agricultural vehicles. Trans.
ASAE 44(6):1397–1402.


January–March 2005 15(1) 87

Jan2005HT.indb 87 12/6/04 4:35:21 PM

You might also like