2015 8th International Congress on Image and Signal Processing (CISP 2015)
Grading system of tomato grafting machine
based on machine vision
Xiurong Zhao
College of Mechanical and Vehicle Engineering
Shenyang Institue of Technology
Fushun, China
Abstract—Grafting technique helps to reduce soil-borne diseases,
overcome the obstacles of cropping cultivation, increase yield and
fruit quality. But grafting requires precise technique; the
proficiency of operation will also directly affect the productivity
and the survival rate of grafted seedlings. With the wide
application of grafting technology, grafting robots were
developed continually. However, the current grafting robots
could not realize the production automation completely, and it is
necessary for them to grade seedlings by hand and match
different diameters of the rootstock and scion, and it is urgent to
realize grading automatically before grafting. So, the machine
vision system of the grafting robot will be designed to grade
grafting seedling automatically to improve grading accuracy,
reduce labor and improve the quality and speed of grafting
effectively.Firstly, the hardware structure of the grading system
was designed based on machine vision, which includes selection
of Japanese CIS industrial cameras, whose type is UXGA, with a
resolution of 1600 × 1200, a frame rate of 20fps, 1/1.8CCD, and
8mm lens and so on. Then using proper lighting resource, the
algorithm of the image acquisition and segmentation about
tomato grafting classification was analysed using the VC++
software and OpenCV tools. Taking the tomato seedlings for the
research object, the statistical methods that gray value
accumulation of pixels horizontally extracting stem diameter of
seedlings features were used in this paper. Thus the proper joint
point was detected and determined according to the middle point
of straight and the smooth part of pixels’ curve. Finally, the
grading system software was designed and debugged, and 100
tomato seedlings were acquired and processed using the visual
grading system for tomato seedling and classified by A, B, C, D
levels, the results show that the system classification success rate
of 96%, because of irregular blade barrier stalks or stalks
suddenly bend, the four seedlings grading failure. The results
also show the grading system was designed reasonably and
performed reliably.
Keywords-Machine vision; Grafting Machine; Grading,
Tomato; Seedling
I. INTRODUCTION
Grafting is a technique whereby branches or buds from one
plant are inserted into appropriate parts of another plant which
have strong affinity. In the process of development, the branch
or bud become a plant of independent by absorbing nutrients.
The branch or bud that grafted is called the scion and the
grafted plants is called the rootstock. Grafting technique helps
978-1-4673-9098-9/15/$31.00 ©2015 IEEE
Zifan Wang, Siyao Liu, Ruili Wang*, Subo Tian*
College of Engineering
Shenyang Agricultural University
Shenyang, China
* Corresponding Author
to reduce soil-borne diseases, overcome obstacles cropping
cultivation, improve fertilizer use efficiency, enhance
resistance, increase yield and fruit quality, and is currently
wide to use in agricultural production[1]. However, grafting
requires precise technique, the proficiency of operation will
also directly affect the survival rate of grafted seedlings. The
majority of grafting machines is semi-automatic, so they
usually finish grafting operation under artificial grading
seedling, and it is difficult to realize full-automatic grafting.
Grafting Robot have been developed during the past twenty
years,and the machine vision system of grafting robot have
been studied recently.Yang Haijun studied the automated
quality inspection and grading methods of the grafted
vegetables[2]. Gu song et al. gave examples of grafting
machine developed in Japan and analyzed its theory and its
current situation[3]. Qiu Yizhi et al. developed several grafting
robot system in Taiwan[4]. Ashraf et al. developed a tomato
seedling classification algorithm based on machine vision.
Zhang Tiezhong et al. presented the method of testing the
growing point of melon flesh seedling and southern melon
seedling based on machine vision recognition and classification
research on robots grafted seedlings[5-6]. Liu chengliang
proposed a design idea to replace manual labor with a robot
vision system[7]. Liu kai designed a visual software program
based on the research of machine vision recognition of the
grafted seedlings[8]. Zhang Tiezhong and Gong Zhihua
developed a grafting machine automatic positioning system
individually with the help of machine vision and image
processing technology,and then the automatic detection of
mechanical device was designed and then was developed a set
of automatic detection algorithm, the single-chip
microcomputer control system is developed at the same time[9-
10]. Li Zhongqiu conducted the research on vegetable grafting
rootstocks automatic positioning and cutting machine systems
based on Machine Vision. He developed an automatic
detection algorithm and program, and designed the
corresponding device[11]. Qiu shuo carried out a study of
healthy seedlings recognition based on image processing
technology[12].
Tomato grafting adopts different types of tomato rootstock
and scions seedling. The average size of rootstock and scion is
about 2~5mm. Enlarging the joint area of scion and stock is the
key of grafting survive. Therefore before grafting, choosing the
604
basic equal plant size stock and scion is an efficient method.
We usually adopted artificial classification method in the past.
But because of the fatigue resistance and limitations of people,
long time tedious work could not assure the grafted seedlings
to be divided into the right level accurately. It not only affects
the quality of grafting but also grafting survival rate. So before
grafting, the grafting robot need to grade grafting seedling
automatically to improve classification accuracy, reduce labor
and improve the quality and speed of grafting effectively.
Therefore, we use the machine vision to replace the "eye" to
identify grafting seedling, and then use the image processing to
grade the grafting seedling automatically in this paper. Taking
tomato seedlings as the research object, the seedlings will be
graded by the stem diameter of the cutting point before grafting
with the help of image processing and machine vision
technology. The machine vision system has great significance
for improving the level of automation and mechanization of
vegetable grafting robot.
II. PRINCIPLES AND METHODS
2.1 Hardware design of machine-vision grading system
A typical machine vision system generally includes camera,
lens, light source, the image acquisition card, image processing
platform, and control execution parts, etc. Based on the
working conditions, inspection object, performance, and cost
requirements, the appropriate grading machine vision system
was designed in this paper, which is shown in Fig. 1.
Fig. 1 The composition of the machine visual hardware system
2.1.1 Selection of the camera
Camera in the machine vision system plays a crucial role:
it is a kind of photoelectric converter that converts optical
signals into electrical signals and outputs in a certain format.
The current imaging devices are mainly the Charge Coupled
Device (CCD) and Complementary Metal Oxide
Semiconductor (CMOS). This study selected CCD camera,
which has better image quality [13].
In this study, we chose the Japanese CIS industrial
camera, the screen is UXGA, with a resolution of 1600 ×
1200(pixels × pixels), a frame rate of 20fps, 1/1.8CCD and the
image area of 6.9 × 5.2 (mm × mm).
When selecting a monocular camera to inspect and grade
the tomato seedlings, a simple and fast calibration method
should be used. The system takes a standard square-shape
object as a calibration reference and adopts an experiment
calibration method. After determining the working distance
between the measured object and the detection system, we
capture the images of objects, obtain standard parts side-length
a'(unit: pixel), while its actual side-length is a (unit: mm),then
the system calibration coefficient k is
a
k= ᧤1᧥
a㧓
For accurate calibration, multiple calibration should be
done in different orientations of the camera field of the view,
and then the average value should be the final calibration
coefficient k, which can reduce a distortion and the random
errors.
2.1.2 Selection of lens
The role of the lens is similar to the human eye lens, the
quality of lens will directly affect the performance of the
whole system. For the whole machine vision system, the lens
is an important image acquisition part, so it should be a
reasonable choice to install the lens. Seedlings classification
and detection parameters are the features of stem-diameter.
Thus, each pixel of the image for precise measurement of the
diameter of the seedlings is important. In the study, according
to the position of object and camera, 8mm lens was selected
and the proper distance between the lens and the object is
120mm. Depending on the size and the principle of the
imaging, the distance relationships between the lens and the
object were shown in Fig. 2.
6.9 mm
L
Fig. 2 The relationship of the focal length, the actual distance and the size of
the object
As shown in the Fig.2, we can calculate the actual length
L of the object, and the total pixel in this direction is 1600, so
each pixel point represent the actual length is 0.065mm.
Calculations are as follows:
§ 6.9 ·
θ = arctan ¨¨ 2 ¸ = 23.33° ˄2˅
8 ¸
© ¹
L = 2 × 120 × tan θ = 103.5 mm ˄3˅
103.5
l pixel = = 0.065 mm ˄4˅
1600
Because the actual length of each pixel has been
calculated, the actual size of the object can be gotten . If we
can obtain the pixels of stem-diameter, we will easily get
tomato actual stem diameter of grafting seedling.
2.1.3 Selection of light source
In the visual system, we usually use the light source and
the controller matching the lighting to stand out the
characteristic of object, remove the influence of environment
and acquire a steady image. The way of lighting commonly
include back light and front lighting. According to the
purpose—the detection of stem diameter, we select back light,
605
front lighting and both back light and front lighting, then carry
out contrast experiments. With the help of Win Roof software
and other tools, we adjust the different light source and
acquire different images of the same object, and get the gray
value curve to contrast.According to the principle of
maximum contrast difference of gray image, the appropriate
light source will be selected in the paper. When we use the
back light to take images, compare with other two cases, the
gray value curve is very steep. We can conclude that the back
light is the best chosen, so we make use of back light to take
pictures and conduct the further research and experiment [14].
2.1.4 Computer and Image capture card
The computer system is the guarantee of normal
operation of image processing software, which can be used to
storage grafting seedlings images, process and analyze the
stem characteristics and grade seedling [15-16]. The computer
system should configure with the memory of 256M, the unit
frequency of 2.93GHz, hard software of 40G and the graphics
card of 64M.
The function of imagine capture card is to take digital
video image information, storage and running, which is the
interface of the image acquisition and processing parts. Many
camera with image capture card don’t need another one, such
as the FireWire Camera and Gigabit Ethernet Camera.
2.2 Software design of machine-vision grading system
2.2.1 Selection of software
The key technology of this system is computer image
processing technology [14]. The ultimate goal of the study is to
achieve sorted seedlings by automatic grafting machine, so
Microsoft Visual c++ is used as the development software,
which is more flexible and wide function, and match with
OpenCV that is a professional computer vision library and
other tools such as Matlab and Winroof, so it is easier to
realize the development of grading algorithm and program.
2.2.2 Grading criterion of grafting seedling
Seedling shape is characterized by its phyllotaxis,but
these phyllotaxis do not provide any useful quantitative
information enough to pattern recognition.Because the shape
of seedling have even changed a lot in the same species of
seedlings. The parameters that can identify dimensions of the
plant body are inter-nodal length, stem diameter and leaf
arrangement, which always differ from one seedling to
another. Usually, each of seedlings has to be measured by the
stem-diameter separately to determine its size and shape,
thereby to recognize the required characteristics. So, the main
feature of the seedlings to be graded is the stem-diameter.
Because the diameter is very thin, each pixel of an image is
important to measure the exact seedling diameter.
Generally, the stem diameter of grafting seedling is
between 2.7mm and 5.1mm, so we determine that "A"
represents the stem-diameter range of more than 4.3mm , "B"
represents that of 3.5 mm to 4.3 mm, "C" represents that of
less than 3.5 mm. According to the above grading criterion, if
some reasons such as irregular blade shelter stem, we can’t
normally recognize the seedling stem, then these seedlings are
listed as "D" and that is the "unqualified".
2.2.3 Image binaryzation processing
Image binaryzation is one of the most common way
to gray-scale transformation, which can make the visual effect
of the whole image display only black and white. In addition,
it can increase the speed of image processing, and takes
resources as less as possible. At present, common binaryzation
algorithm contain threshold limit binaryzation algorithm and
adaptive threshold binaryzation algorithms. In this study, the
seedlings of graphic information, relative to the background, is
a clear outline and uniform gray value area, so we choose the
threshold limit binaryzation algorithm. The algorithm is
relatively simple, according to the image grey value range,
choosing a grey value as a threshold T. The image is divided
into two parts, then the grey value g (i, j) of any point (i, j) in
each image will be compared with T, if it is greater than or
equal to T, this grey value will be set to 1; otherwise, if it is
less than T, it is set to 0, (cf. Equation (5)):
­1, g (i, j ) ≥ T ᧤5᧥
f (i, j ) = ®
¯0, g (i, j ) < T
where g (i, j) denotes the grey value of original image, f (i, j)
denotes the grey value of binary image
In this study, the image is processed in binaryzation, the
grey value of stem characteristic is 1, the rest is 0, as shown in
Fig.3.
Fig. 3 Binary image
2.2.4 Image characteristics extracting algorithm
This experiment adopts the method of grey accumulation
by all horizontal pixels to extract the diameter characteristics
of seedlings stem. Image is a two-dimensional signal, and all
pixels form a matrix. The value of each pixel in the horizontal
direction adds, that is to say pixel value in the same horizontal
lines will plus together. Since the value of each pixel in the
image is 1 or 0, the accumulation is to calculate the sum of the
number of pixel value 1, then we capture the stem-diameter
distribution histogram characteristic of g (y), as shown in
formula (6):
W −1
g ( y ) = ¦ f ( x, y ) y = 0,1,2, …, W - 1 ˄6˅
x=0
where f (x, y) denotes image pixel values (x, y), W denotes
width of image.
2.2.5 Design of software program
This study selects the Windows XP operating
system,development tool with powerful Microsoft Visual
Studio 2008,as well as open source computer design library
606
OpenCV2.0 which is embed and integrated into development
system formed in the Microsoft Visual Studio 2008. The
process of the system running roughly was divided into three
parts:image preprocessing,image segmentation and
detection.Image preprocessing include image acquisition and a
series of processing before binaryzation.Image segmentation is
an important part of the process,and the processed image
quality directly decides whether the feature extraction and
analysis can be successfully done.The purpose of detecting
process is the extraction and analysis of the features.
2.3 Experiment design of machine-vision grading system
The tomato seedlings called "momotaro" were selected as
the experiment material. The age of seedlings is about two
months. After selecting the device and lighting of machine
vision system of grafting robot, some images of the tomato
seedlings will be captured and processed for grading.
In this study, grafting seedling is between camera and
back light, and the distance is 120mm. In this experiment, we
will rotate the 90 ° direction for every seedling and capture
two images in two direction, therefore, we get two different
images for each seedling and select the better one to process.
III. RESULTS AND DISCUSSIONS
3.1 Hardware structure of machine-vision grading system
The hardware structure of grading system was designed
based on machine vision, which includes selection of Japanese
CIS industrial cameras, whose type is UXGA, with a
resolution of 1600 × 1200, a frame rate of 20fps, 1/1.8CCD,
and 8mm lens and so on. After compared with different
lighting resource such as front lighting, back light, both back
light and front lighting, experiment results show that blue back
lighting system was the best lighting resource and was chosen
for acquiring images of seedlings.
3.2 Running result of software program of machine-vision
grading system
This program is a collection of images to be processed to
extract characteristic gray values, and the data of seedling
characteristics was obtained and stored as an Excel file in
order to provide the necessary data basis for the following
analysis. After completed, program calls the images and
analyze the data generated Excel form, then use the mapping
function to draw the curve of accumulated gray value of image
pixel along the horizontal axis,as shown in Fig.4. Finally we
can conclude that the proper joint point was detected and
determined, that is the middle point of straight and smooth
part of pixels’ curve.
3.3 Experiment results and discussion
According to the experimental design, we acquire two
images of each seedling in 90rdirection and choose the better
image to be graded, then total 100 images of 100 tomato
grafted seedlings were processed. Finally, classification
results of the 100 seedlings are as shown in Tab.1. From the
table, it can be seen that the classification success rate is
96%,and 4% of the image classification fails, that is, 4
seedlings’ diameter can not be detected. We can find that
failure of classification is mainly due to irregular seedling
leaf stalk(Fig.5) or stem suddenly bending.

200

400

600

800

1000
0 50 100 150 200 250 300 350 400

Fig. 4 The curve of accumulated gray value of the image pixel along the horizontal axis

200

400

600

800

1000
0 50 100 150 200 250 300 350 400

Fig. 5 The seedling image of grading failure

607
 Pixel along the vertical axis

Accumulated gray value of image pixel along the horizontal axis

Fig.6 (a) Original image; (b) Binary image; (c) Accumulated gray value of image pixel along the horizontal axis
Pixel along the vertical axis

Accumulated gray value of image pixel along the horizontal axis

Fig.7 (a) Original image; (b) Binary image; (c) Accumulated gray value of image pixel along the horizontal axis

608
 There are five classification success graft examples in
Fig.6,and two classification failure graft examples in Fig.7.
From the tab.1, we can also find that the number of "A"
grade seedlings is 20, "B" is 62, and “C" is 14, which is the
middle diameter of the seedlings accounted for 62% of the
total seedlings, so it is very suitable for mechanical grafting
seedlings.
Tab. 1 The grading result
The success rate The failure rate
Grade A B C D of grading of grading
(100%) (4%)
Number 20 62 14 4 96 4
IV. CONCLUSIONS
Based on a review of domestic and foreign application
status of grafting robot and machine vision in agriculture,the
hardware of the machine vision grading system were selected
and developed, including the selection of camera, lens, blue
back light, and the image information acquisition processing
card,etc. Thus a hardware platform of the tomato grafting
machine vision grading system was built. Then by using
VC++ programming language and computer vision library
Open CV the eigenvalue algorithm of stem diameter of grafted
tomato seedlings was determined. On this basis, the machine
vision software of tomato seedling grading was designed, and
experiment study were carried out. By the hardware platform
of machine vision grading system and grading software, 100
tomato seedling images were acquired, processed and graded,
then the success rate of the grading system for tomato
mechanized seedling reaches 96%, which shows the grading
system is designed reasonably and performed reliably.
ACKNOWLEDGMENT
This work was financially supported by the National
Natural Science Foundation of China (31101076), Program for
Liaoning Excellent Talents in University(LJQ 2012061) and
Scholars of Mount Tianzhu in Shenyang Agricultural
University(2014).
REFERENCES
[1] Lee J, Kubota C, Tsao S, et al. Current status of vegetable grafting:
Diffusion, grafting techniques, automation. Science Horticulture, 2010,
127(2):93-105.
[2] Yang H. Studies on the automatic quality estimation and grading of
grafting seedling. Changsha: Hunan Agricultural University, 2003.
[3] Gu S, Jiang L. Development of domestic and foreign vegetable grafting
robot. Journal of Northeast Agricultural University, 2007, 38(6):847-851.
[4] Chiu Y, Chen S, Chang Y. Development of a circular grafting robotic
system for watermelon seedlings. Applied Engineering in Agriculture,
2010, 24(1): 1077-1084.
[5] Zhang T, Wei J. Study on vision system of vegetable grafting robot (I):
Searching for growth point of cucurbit seedling with morphological
method. Journal of China Agricultural University, 1999, 4(4):45-47.
[6] Zhang T, Wei J. Study on vision system of vegetable grafting robot (Ċ):
Searching for growth point of pumpkin seedling with geometrical
method. Journal of China Agricultural University, 1999, 4(4):48-50.
[7] Liu C, Wang H, Cao Q, et al. Study on robot vision and judgment
system for seedling grafting robot. Transactions of the Chinese Society
for Agricultural Machinery, 2001, 32(4):38-41.
[8] Liu K, Gu S. The research on identification of seedling based on
machine vision. Journal of Agricultural Mechanization Research,
2009(11):46-48.
[9] Zhang T, Sun G, Gong Z. Study on automatic orientation system of
stock in vegetable automation drafting based on image processing
technology. Journal of Weifang University, 2003, 3(6):34-36.
[10] Gong Z. Study on automatic orientation system of stock in vegetable
automation grafting based on image processing technology. Beijing:
China Agricultural University, 2003.
[11] Li Z. Studies on automatic orientation cutting system of stock in
vegetable automation grafting based on machine vision. Changsha:
Hunan Agricultural University, 2007.
[12] Qiu S. Research on methodology to identify seeding healthy condition
based on image processing technology. Changchun: Jilin University,
2009.
[13] Anna Fabija ´nska, Dominik Sankowski. Computer vision system for
high temperature measurements of surface properties. Machine Vision
and Applications, 2009, 20:411-421.
[14] Tian S, Ashraf M A, Kondo N, et al. Optimization of Machine Vision
for Tomato Grafting Robot. Sensor Letters, 2013, 11(6/7):1190–1194.
[15] Chandra Sekhar Nandi, Bipan Tudu, and Chiranjib Koley,et al.A
Machine Vision-Based Maturity Prediction System for Sorting of
Harvested Mangoes.IEEE Transactions on instrumentation and
Measurement,2014,63(7):1724-1727.
[16] Brian L. DeCost, Elizabeth A. Holm. A computer vision approach for
automated analysis and classification of microstructural image data.
Computational Materials Science, 2015, 110:126-133.
609