Abstract:Highlights • A novel Active Laser-Camera Scanning system,2 consisting of a red line laser, an RGB-D camera, and a linear motion slide, is designed, and developed for accurate fruit 3D localization. • A Laser Line Extraction (LLE) algorithm is proposed and implemented for robust feature matching to enable stable 2D-3D transformation for ALACS. • System evaluation and validation of the ALACS are performed indoors and outdoors in comparison with a conventional 3D sensing technique, i.e., Intel RealSense D435i RGB-Depth camera. Recent advancements in deep learning-based approaches have led to remarkable progress in fruit detection, enabling robust fruit identification in complex environments. However, much less progress has been made on fruit 3D localization, which is equally crucial for robotic harvesting. Complex fruit shape/orientation, fruit clustering, varying lighting conditions, and occlusions by leaves and branches have greatly restricted existing sensors from achieving accurate fruit localization in the natural orchard environment. In this paper, we report on the design of a novel localization technique, called Active Laser-Camera Scanning (ALACS), to achieve accurate and robust fruit 3D localization. The ALACS hardware setup comprises a red line laser, an RGB color camera, a linear motion slide, and an external RGB-D camera. Leveraging the principles of dynamic-targeting laser-triangulation, ALACS enables precise transformation of the projected 2D laser line from the surface of apples to the 3D positions. To facilitate laser pattern acquisitions, a Laser Line Extraction (LLE) method is proposed for robust and high-precision feature extraction on apples. Comprehensive evaluations of LLE demonstrated its ability to extract precise patterns under variable lighting and occlusion conditions. The ALACS system achieved average apple localization accuracies of 6.9 - 11.2 mm at distances ranging from 1.0 m to 1.6 m, compared to 21.5 mm by a commercial RealSense RGB-D camera, in an indoor experiment. Orchard evaluations demonstrated that ALACS has achieved a 95% fruit detachment rate versus a 71% rate by the RealSense camera. By overcoming the challenges of apple 3D localization, this research contributes to the advancement of robotic fruit harvesting technology.

Fruit Tree Location Method Based on 3D LiDAR and Optimized DBSCAN Algorithm

Navigation of an Autonomous Spraying Robot for Orchard Operations Using LiDAR for Tree Trunk Detection

Mobile LiDAR Scanning System Combined with Canopy Morphology Extracting Methods for Tree Crown Parameters Evaluation in Orchards

Research on orchard navigation method based on fusion of 3D SLAM and point cloud positioning

Precision Variable-Rate Spraying Robot by Using Single 3D LIDAR in Orchards

A Fruit-Tree Mapping System for Semi-Structured Orchards Based on Multi-Sensor-Fusion SLAM

Research on the Relative Position Detection Method between Orchard Robots and Fruit Tree Rows

Canopy Volume Measurement of Fruit Trees Using Robotic Platform Loaded LiDAR Data

Accurate Fruit Localisation for Robotic Harvesting using High Resolution LiDAR-Camera Fusion

High-Precision Fruit Localization Using Active Laser-Camera Scanning: Robust Laser Line Extraction for 2D-3D Transformation

A spraying path planning algorithm based on colour-depth fusion segmentation in peach orchards

RESEARCH ON SLAM AND PATH PLANNING METHOD FOR INSPECTION ROBOT IN ORCHARD ENVIRONMENT

Research on 2D Laser Automatic Navigation Control for Standardized Orchard

Real-Time Localization and Colorful Three-Dimensional Mapping of Orchards Based on Multi-Sensor Fusion Using Extended Kalman Filter

Crop Row Detection Algorithm Based on 3-D LiDAR: Suitable for Crop Row Detection in Different Periods

Integrated Positioning System of Kiwifruit Orchard Mobile Robot Based on UWB/LiDAR/ODOM

Three-Dimensional Morphological Measurement Method for a Fruit Tree Canopy Based on Kinect Sensor Self-Calibration

Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting

Design and development of orchard autonomous navigation spray system