Abstract:Highlights • A novel Active Laser-Camera Scanning system,2 consisting of a red line laser, an RGB-D camera, and a linear motion slide, is designed, and developed for accurate fruit 3D localization. • A Laser Line Extraction (LLE) algorithm is proposed and implemented for robust feature matching to enable stable 2D-3D transformation for ALACS. • System evaluation and validation of the ALACS are performed indoors and outdoors in comparison with a conventional 3D sensing technique, i.e., Intel RealSense D435i RGB-Depth camera. Recent advancements in deep learning-based approaches have led to remarkable progress in fruit detection, enabling robust fruit identification in complex environments. However, much less progress has been made on fruit 3D localization, which is equally crucial for robotic harvesting. Complex fruit shape/orientation, fruit clustering, varying lighting conditions, and occlusions by leaves and branches have greatly restricted existing sensors from achieving accurate fruit localization in the natural orchard environment. In this paper, we report on the design of a novel localization technique, called Active Laser-Camera Scanning (ALACS), to achieve accurate and robust fruit 3D localization. The ALACS hardware setup comprises a red line laser, an RGB color camera, a linear motion slide, and an external RGB-D camera. Leveraging the principles of dynamic-targeting laser-triangulation, ALACS enables precise transformation of the projected 2D laser line from the surface of apples to the 3D positions. To facilitate laser pattern acquisitions, a Laser Line Extraction (LLE) method is proposed for robust and high-precision feature extraction on apples. Comprehensive evaluations of LLE demonstrated its ability to extract precise patterns under variable lighting and occlusion conditions. The ALACS system achieved average apple localization accuracies of 6.9 - 11.2 mm at distances ranging from 1.0 m to 1.6 m, compared to 21.5 mm by a commercial RealSense RGB-D camera, in an indoor experiment. Orchard evaluations demonstrated that ALACS has achieved a 95% fruit detachment rate versus a 71% rate by the RealSense camera. By overcoming the challenges of apple 3D localization, this research contributes to the advancement of robotic fruit harvesting technology.

Apple Fruit Recognition Algorithm Based on Laser Scanning 3-D Image

Research on Trunk and Branch Recognition Method of Apple Harvesting Robot

Recognition and Localization Method of Overlapping Apples for Apple Harvesting Robot

Recognition of Apple Fruit in Natural Environment

Automatic recognition of on-tree apples based on contour curvature

Recognition Method for Apple Fruit Based on SUSAN and PCNN

Automatic Apple Recognition Based on the Fusion of Color and 3D Feature for Robotic Fruit Picking

Recognition method of apple target based on significant contour

High-Precision Fruit Localization Using Active Laser-Camera Scanning: Robust Laser Line Extraction for 2D-3D Transformation

Segmentation Algorithm for Apple Recognition Using Image Features and Artificial Neural Network

Recognition and Localization of Apples Based on Super-red Image and Contour Curvature

Location Of Apples In Trees Using Stereoscopic Vision

Fruit recognition based on pulse coupled neural network and genetic Elman algorithm application in apple harvesting robot

Recognition and Reconstruction of Single Apple Occluded by Branches

The Recognition of Overlapping Apple Fruits Based on Boundary Curvature Estimation

Estimation Method of Apple Growing Attitude Based on Computer Vision

Research on Apple Recognition Algorithm in Complex Orchard Environment Based on Deep Learning

Fast Location and Recognition of Green Apple Based on RGB-D Image

A Method of Apple Image Segmentation Based on Color-Texture Fusion Feature and Machine Learning

Construction of laser vision system for apple harvesting robot

Research on High Performance Apple Recognition Model Construction Strategy Based on Deep Learning