Abstract:The fast and precise detection of dense litchi fruits and the determination of their maturity is of great practical significance for yield estimation in litchi orchards and robot harvesting. Factors such as complex growth environment, dense distribution, and random occlusion by leaves, branches, and other litchi fruits easily cause the predicted output based on computer vision deviate from the actual value. This study proposed a fast and precise litchi fruit detection method and application software based on an improved You Only Look Once version 5 (YOLOv5) model, which can be used for the detection and yield estimation of litchi in orchards. First, a dataset of litchi with different maturity levels was established. Second, the YOLOv5s model was chosen as a base version of the improved model. ShuffleNet v2 was used as the improved backbone network, and then the backbone network was fine-tuned to simplify the model structure. In the feature fusion stage, the CBAM module was introduced to further refine litchi's effective feature information. Considering the characteristics of the small size of dense litchi fruits, the 1,280 × 1,280 was used as the improved model input size while we optimized the network structure. To evaluate the performance of the proposed method, we performed ablation experiments and compared it with other models on the test set. The results showed that the improved model's mean average precision (mAP) presented a 3.5% improvement and 62.77% compression in model size compared with the original model. The improved model size is 5.1 MB, and the frame per second (FPS) is 78.13 frames/s at a confidence of 0.5. The model performs well in precision and robustness in different scenarios. In addition, we developed an Android application for litchi counting and yield estimation based on the improved model. It is known from the experiment that the correlation coefficient R 2 between the application test and the actual results was 0.9879. In summary, our improved method achieves high precision, lightweight, and fast detection performance at large scales. The method can provide technical means for portable yield estimation and visual recognition of litchi harvesting robots.

Computer Vision-Based Localisation of Picking Points for Automatic Litchi Harvesting Applications Towards Natural Scenarios

Feature Extraction And Recognition Based On Machine Vision Application In Lotus Picking Robot

Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNet

Lychee Fruit Detection Based on Monocular Machine Vision in Orchard Environment

Fast and precise detection of litchi fruits for yield estimation based on the improved YOLOv5 model

Research on the vision system of lychee picking robot based on stereo vision

Accurate Cutting-point Estimation for Robotic Lychee Harvesting through Geometry-aware Learning

Recognition and Localization Methods for Vision-Based Fruit Picking Robots: A Review

A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard

Detection and localization of citrus fruit based on improved You Only Look Once v5s and binocular vision in the orchard

A Performance Analysis of a Litchi Picking Robot System for Actively Removing Obstructions, Using an Artificial Intelligence Algorithm

Autonomous navigation for a wolfberry picking robot using visual cues and fuzzy control

Accurate Fruit Localisation for Robotic Harvesting using High Resolution LiDAR-Camera Fusion

Prototype Network for Predicting Occluded Picking Position Based on Lychee Phenotypic Features

Retrieval of Leaf Chlorophyll Contents (LCCs) in Litchi Based on Fractional Order Derivatives and VCPA-GA-ML Algorithms

Detecting ripe fruits under natural occlusion and illumination conditions

Robotic Harvesting of the Occluded Fruits with a Precise Shape and Position Reconstruction Approach

Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the Field

A detection approach for late-autumn shoots of litchi based on unmanned aerial vehicle (UAV) remote sensing

Clustered tomato detection and picking point location using machine learning-aided image analysis for automatic robotic harvesting

Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting