Abstract:Our research focuses on winter jujube trees and is conducted in a greenhouse environment in a structured orchard to effectively control various growth conditions. The development of a robotic system for winter jujube harvesting is crucial for achieving mechanized harvesting. Harvesting winter jujubes efficiently requires accurate detection and location. To address this issue, we proposed a winter jujube detection and localization method based on the MobileVit-Large selective kernel-GSConv-YOLO (MLG-YOLO) model. First, a winter jujube dataset is constructed to comprise various scenarios of lighting conditions and leaf obstructions to train the model. Subsequently, the MLG-YOLO model based on YOLOv8n is proposed, with improvements including the incorporation of MobileViT to reconstruct the backbone and keep the model more lightweight. The neck is enhanced with LSKblock to capture broader contextual information, and the lightweight convolutional technology GSConv is introduced to further improve the detection accuracy. Finally, a 3-dimensional localization method combining MLG-YOLO with RGB-D cameras is proposed. Through ablation studies, comparative experiments, 3-dimensional localization error tests, and full-scale tree detection tests in laboratory environments and structured orchard environments, the effectiveness of the MLG-YOLO model in detecting and locating winter jujubes is confirmed. With MLG-YOLO, the mAP increases by 3.50%, while the number of parameters is reduced by 61.03% in comparison with the baseline YOLOv8n model. Compared with mainstream object detection models, MLG-YOLO excels in both detection accuracy and model size, with a mAP of 92.70%, a precision of 86.80%, a recall of 84.50%, and a model size of only 2.52 MB. The average detection accuracy in the laboratory environmental testing of winter jujube reached 100%, and the structured orchard environmental accuracy reached 92.82%. The absolute positioning errors in the X, Y, and Z directions are 4.20, 4.70, and 3.90 mm, respectively. This method enables accurate detection and localization of winter jujubes, providing technical support for winter jujube harvesting robots.

Automatic Branch Detection of Jujube Trees Based on 3D Reconstruction for Dormant Pruning Using the Deep Learning-Based Method

Automatic Reconstruction and Modeling of Dormant Jujube Trees Using Three-View Image Constraints for Intelligent Pruning Applications

Research on Trunk and Branch Recognition Method of Apple Harvesting Robot

Skeleton extraction and pruning point identification of jujube tree for dormant pruning using space colonization algorithm

Detection of Subtle Bruises on Winter Jujube Using Hyperspectral Imaging with Pixel-Wise Deep Learning Method

Accuracy Evaluation and Branch Detection Method of 3D Modeling Using Backpack 3D Lidar SLAM and UAV-SfM for Peach Trees during the Pruning Period in Winter

YOLOv7-Branch: A Jujube Leaf Branch Detection Model for Agricultural Robot

3D Branch Point Cloud Completion for Robotic Pruning in Apple Orchards

Design and Performance Test of a Jujube Pruning Manipulator

A real-time branch detection and reconstruction mechanism for harvesting robot via convolutional neural network and image segmentation

MLG-YOLO: A Model for Real-Time Accurate Detection and Localization of Winter Jujube in Complex Structured Orchard Environments

A novel jujube tree trunk and branch salient object detection method for catch-and-shake robotic visual perception

Drone Stereo Vision for Radiata Pine Branch Detection and Distance Measurement: Integrating SGBM and Segmentation Models

Obscured tree branches segmentation and 3D reconstruction using deep learning and geometrical constraints

Three-dimensional reconstruction of guava fruits and branches using instance segmentation and geometry analysis

Detection and Reconstruction of Passion Fruit Branches via CNN and Bidirectional Sector Search

Drone Stereo Vision for Radiata Pine Branch Detection and Distance Measurement: Utilizing Deep Learning and YOLO Integration

Apple Tree Branch Information Extraction from Terrestrial Laser Scanning and Backpack-LiDAR

Semantic segmentation for partially occluded apple trees based on deep learning

An autonomous robot for pruning modern, planar fruit trees

Uncovering implementable dormant pruning decisions from three different stakeholder perspectives