Professional Documents
Culture Documents
Computers in Industry
journal homepage: www.elsevier.com/locate/compind
A R T I C L E I N F O A B S T R A C T
Article history:
Received 31 August 2017 Intelligent detection is a key technology in precision agriculture. As items of different color cluster in
Received in revised form 9 March 2018 different non-overlapping elliptical regions, this study proposed a method for constructing a multi-
Accepted 15 March 2018 elliptical boundary model in Cr-Cb co-ordinates to detect citrus fruit and tree trunks in natural light
Available online xxx environments. Here, the detected citrus variety was spring sweet tangerine, and the parameters of the
elliptical boundary models for detecting these fruit and tree trunks solved by color-space transformation
Keywords: and ellipse fitting. A series of image detection experiments were performed to evaluate the method’s
Computer vision performance. The experimental results showed that the correct and false positive percentages in fruit
Natural environments
identification from images were 90.8 and 11.2%, respectively. The number of correctly detected images in
Elliptical boundary model
distinguishing tree trunks from background was 44 of 50 images.
Citrus detection
© 2018 Elsevier B.V. All rights reserved.
https://doi.org/10.1016/j.compind.2018.03.007
0166-3615/© 2018 Elsevier B.V. All rights reserved.
10 T.-H. Liu et al. / Computers in Industry 99 (2018) 9–16
saved as 24-bit color JPG format and resized to as 1280 960 2.4. Construct elliptical boundary model
pixels. Microsoft Visual C + + 2013 and Open Source Computer
Vision Library (openCV2.4.9, Intel Corporation) were used in The performance of the elliptical boundary model was
developing software to realize the proposed detection algorithm comparable to that of Gaussian mixture models, and yet, its
on an Intel(R) Core(TM) i7-4600 CPU @ 2.10 GHz 2.69 GHz, 4.00GB computational complexity was as simple as training a single
RAM laptop. Gaussian model [24]. Hsu et al. [25] have concluded that skin color
was distributed as an ellipse in Cr-Cb co-ordinates. The fruit pixels
2.3. Convert images from RGB color space to Y0 CbCr color space colors clustered in the Cr-Cb co-ordinates were within a region
similar to an ellipse (Fig. 5a).
RGB space is the most common color space. However, high In this subsection, the construction of elliptical boundary model
correlation exists between R, G and B colors. After images are was discussed. The moment of order (p and q), with p 0 and q 0,
converted to Y0CbCr color space from RGB color space, the resultant was defined as
signals range from 16 to 235 for Y0 and from 16 to 240 for Cb and Cr. ðð X
q q
Y0CbCr color space is often used in segmenting objects that are mp;q ¼ Crp Cb dCrdCb ¼ Crp Cb ð2Þ
difficult to separate in RGB color space. Here, RGB image was ðCr;CbÞ2R
ðCr;CbÞ2R
converted to Y0CbCr image using Eq. (1)
If p + q 1, the normalized moments of mp,q were given by
Y0 65:481 128:553 24:966 R 16
½ Cr ¼ ½ 37:797 74:203 112:000 ½ G þ ½ 128 ð1Þ 1 X q
np;q ¼ Crp Cb ð3Þ
Cb 112:0 93:786 128:214 B 128 aðCr;CbÞ2R
Fig. 5. (a) Color region in Cr-Cb co-ordinates for a fruit skin, and (b) the parameters for an ellipse.
12 T.-H. Liu et al. / Computers in Industry 99 (2018) 9–16
Fig. 6. (a) Template image, (b) Sampled skin, and (c) Color region of skin in Cr-Cb co-ordinates for fruit.
Fig. 7. (a) Template image, (b) Sampled skin, and (c) Color region of skin in Cr-Cb co-ordinates for tree trunk.
where a represents the region’s area. to define the region’s orientation and extent. According to Steger
If p + q 2, the moments relative to the region’s center of et al. [26], the major and the minor axes r1 and r2, as well as the
gravity, which is defined as the central moments, were given by angle u of the ellipse with respect to the column axis (Fig. 5b) were
given by
1 X
mp;q ¼ ðCr n1;0 Þp ðCb n0;1 Þq ð4Þ 8 rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
aðCr;CbÞ2R >
>
> r ¼ 2ðm2;0 þ m0;2 þ ðm2;0 m0;2 Þ2 þ 4m21;1 Þ
>
> 1
>
> rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
The co-ordinates of the region’s center were given by <
8 X X r2 ¼ 2ðm2;0 þ m0;2 ðm2;0 m0;2 Þ2 þ 4m21;1 Þ ð6Þ
>
>
>
> Cr Cb >
>
< m1;0 ðCr;CbÞ2R m0;1 ðCr;CbÞ2R >
>
> 1 2m1;1
Cr ¼ ¼ X Cb ¼ ¼ X ð5Þ : u ¼ arctan
>
> m0;0 1 m0;0 1 2 m2;0 m0;2
:
ðCr;CbÞ2R ðCr;CbÞ2R
A pixel was classified as a fruit pixel if its chrominance
Assuming a region’s moments of order 1 and 2 were obtained vectors (Cr and Cb) were inside the fitted ellipse region of fruit
from an ellipse, the second central moments (p + q 2) were used skin, or it was classified as a tree trunk pixel if its chrominance
T.-H. Liu et al. / Computers in Industry 99 (2018) 9–16 13
Fig. 9. Fruit detection results from (a) a thin density image and (b) a thick density image.
Fig. 10. Fruit numbers counted by human, the correct and the finally counted by computer along with the false positive and the missed.
Fig. 11. The performance of the algorithm in detecting each of those 50 images.
vectors were inside the fitted ellipse region of tree trunk skin. MATLAB 2016a software (Mathworks, Inc., Natick, MA, USA)
Otherwise, it was classified as “other” pixels. The criteria was was used for designing program calculating parameters for these
expressed as elliptical boundary models. Template images and sample skins for
8 calculating the elliptical boundary model of fruit and tree trunks,
> Fruit pixel; if vectors ðCr; Cb Þare inside the ellipse region of fruit skin .
< along with regions of color clustered in Cr-Cb co-ordinates for
Tree trunk pixel; if vectors ðCr; Cb Þare inside the ellipse region of tree trunk skin .
>
: those two sampled skins are shown in Figs. 6 and 7, respectively.
Other; otherwise:
The fitted elliptical boundary models and their parameters are
ð7Þ shown in Fig. 8.
14 T.-H. Liu et al. / Computers in Industry 99 (2018) 9–16
Table 1 Table 2
Fruit detection accuracy. Sampled fruit skins along with their lightness and their fitted elliptical boundary
models.
Items Results Standard deviation
Images 50 – Fruit skin Lightness Fitted elliptical model
Total fruit number 1299 – 34%
CDP 90.8% 0.044
FPP 11.2% 0.053
MP 9.2% 0.044
51%
Fig. 12. (a) The original tree trunk image, and (b) resulting image after segmentation.
T.-H. Liu et al. / Computers in Industry 99 (2018) 9–16 15
Fig. 13. (a) Template image, (b) Sampled skin, and (c) Color region in Cr-Cb co-ordinates after brightness correction.
16 T.-H. Liu et al. / Computers in Industry 99 (2018) 9–16