Abstract:For most fruit-harvesting robots, an essential task of the machine vision system is to provide the manipulator with an accurate three-dimensional location of the target. However, the accuracy of this location can be affected by various factors. This study aimed to develop seven location methods, to investigate their effectiveness, as well as the influences of different camera modes and camera types, and, ultimately, to ascertain which was the optimal method. These methods utilized the pixels of the detected targets in each image, the corresponding depth values, as well as the locations of the 2D bounding boxes extracted from the detection results. These location methods differed in the way that they obtained the position of the 3D bounding box, and in their use of point clustering or colour thresholding. The images were collected via two types of 3D camera, patterned structured light and time-of-flight. Comparative analysis showed that methods using the 2D bounding box and the selected depth value to calculate the 3D bounding box were faster (0.2–8.4 ms compared to 151.9–325.2 ms) and performed better than the 3D clustering methods. In addition, four modes of the structured light camera were tested and compared. The results showed that the high-accuracy mode had fewer noise points but a lower location rate (89.2–89.9%), while the high-density mode created more noise points but a higher location rate (98.9%). Evaluations also indicated that the data from the time-of-flight camera better represented the 3D shape (26.3% more accurate along the camera's depth direction). Therefore, time-of-flight camera was considered better for the applications that required more accurate 3D shape. This paper, thus, provided references in the selection of location methods, cameras and corresponding modes for related work.

Symmetry-based 3D shape completion for fruit localisation for harvesting robots

Research on 3D Reconstruction Algorithm of Citrus Seedlings Based on Multi-View Geometry

Contrastive 3D Shape Completion and Reconstruction for Agricultural Robots Using RGB-D Frames

Fruit Mapping with Shape Completion for Autonomous Crop Monitoring

A Dataset and Benchmark for Shape Completion of Fruits for Agricultural Robotics

Three-dimensional location methods for the vision system of strawberry-harvesting robots: development and comparison

Horticultural Temporal Fruit Monitoring via 3D Instance Segmentation and Re-Identification using Point Clouds

Three-dimensional reconstruction of guava fruits and branches using instance segmentation and geometry analysis

Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting

3D shape sensing and deep learning-based segmentation of strawberries

Panoptic Mapping with Fruit Completion and Pose Estimation for Horticultural Robots

Fruit Localization and Environment Perception for Strawberry Harvesting Robots

Geometry-Aware Fruit Grasping Estimation for Robotic Harvesting in Orchards

FF3D: A Rapid and Accurate 3D Fruit Detector for Robotic Harvesting

NBV-SC: Next Best View Planning based on Shape Completion for Fruit Mapping and Reconstruction

Accurate Fruit Localisation for Robotic Harvesting using High Resolution LiDAR-Camera Fusion

Key Point-based Orientation Estimation of Strawberries for Robotic Fruit Picking

Incomplete Laser 3D Point Cloud Classification and Completion Approach Based on Object Symmetry under Occlusion Conditions

Single-Shot 6DoF Pose and 3D Size Estimation for Robotic Strawberry Harvesting

An Active Structured Light 3D Point Cloud Measurement for Orange Shape Based On Adaptive Co-Opposite-Direction Slicing Method