Article
Guava Detection and Pose Estimation Using
a Low-Cost RGB-D Sensor in the Field
Guichao Lin
1,2
, Yunchao Tang
3,
*, Xiangjun Zou
1,
*, Juntao Xiong
1
and Jinhui Li
1
1
Key Laboratory of Key Technology on Agricultural Machine and Equipment, Ministry of Education,
South China Agricultural University, Guangzhou 510642, China; guichaolin@126.com (G.L.);
xiongjt@scau.edu.cn (J.X.); lijinhui29@163.com (J.L.)
2
College of Mechanical and Automotive Engineering, Chuzhou University, Chuzhou 239000, China
3
School of Urban and Rural Construction, Zhongkai University of Agriculture and Engineering,
Guangzhou 510006, China
* Correspondence: ryan.twain@gmail.com (Y.T.); xjzou1@163.com (X.Z.)
Received: 27 December 2018; Accepted: 18 January 2019; Published: 21 January 2019
Abstract:
Fruit detection in real outdoor conditions is necessary for automatic guava harvesting, and
the branch-dependent pose of fruits is also crucial to guide a robot to approach and detach the target
fruit without colliding with its mother branch. To conduct automatic, collision-free picking, this study
investigates a fruit detection and pose estimation method by using a low-cost red–green–blue–depth
(RGB-D) sensor. A state-of-the-art fully convolutional network is first deployed to segment the RGB
image to output a fruit and branch binary map. Based on the fruit binary map and RGB-D depth
image, Euclidean clustering is then applied to group the point cloud into a set of individual fruits.
Next, a multiple three-dimensional (3D) line-segments detection method is developed to reconstruct
the segmented branches. Finally, the 3D pose of the fruit is estimated using its center position
and nearest branch information. A dataset was acquired in an outdoor orchard to evaluate the
performance of the proposed method. Quantitative experiments showed that the precision and recall
of guava fruit detection were 0.983 and 0.948, respectively; the 3D pose error was
23.43
◦
± 14.18
◦
;
and the execution time per fruit was 0.565 s. The results demonstrate that the developed method can
be applied to a guava-harvesting robot.
Keywords:
guava detection; pose estimation; fully convolutional network; branch reconstruction;
RGB-D sensor
1. Introduction
Guava harvesting is labor-intensive, time-consuming, and costly work. The aging population
and growing urbanization in China have resulted in an older agricultural labor force [
1
], which is
becoming a potential threat to fruit harvesting. Therefore, it is urgent to develop an automatic
guava-harvesting robot that can work in the field. In-field fruit detection is an important aspect of
a harvesting robot [
2
], containing many challenges including varying illuminations, occlusion caused
by leaves and branches, and color variations in fruit. Additionally, if only the fruit position information
is available, the end-effector of the harvesting robot is likely to have collisions with the mother branch
of the fruit when moving toward a fruit, hence lowering the harvest success rate. Thus, for each fruit,
estimating a three-dimensional (3D) pose relative to its mother branch along which the end-effector
can approach the fruit without colliding with the branch is very important. In this work, the fruit pose
is defined as a vector that passes through the fruit center and is perpendicular to the mother branch
of the fruit. Bac et al. has shown that such a pose could increase the grasp success rate from 41% to
61% [3]. Guava fruit detection and pose estimation were investigated in this study.
Sensors 2019, 19, 428; doi:10.3390/s19020428 www.mdpi.com/journal/sensors
