Abstract:Intelligent detection and localization of mature citrus fruits is a critical challenge in developing an automatic harvesting robot. Variable illumination conditions and different occlusion states are some of the essential issues that must be addressed for the accurate detection and localization of citrus in the orchard environment. In this paper, a novel method for the detection and localization of mature citrus using improved You Only Look Once (YOLO) v5s with binocular vision is proposed. First, a new loss function (polarity binary cross-entropy with logit loss) for YOLO v5s is designed to calculate the loss value of class probability and objectness score, so that a large penalty for false and missing detection is applied during the training process. Second, to recover the missing depth information caused by randomly overlapping background participants, Cr-Cb chromatic mapping, the Otsu thresholding algorithm, and morphological processing are successively used to extract the complete shape of the citrus, and the kriging method is applied to obtain the best linear unbiased estimator for the missing depth value. Finally, the citrus spatial position and posture information are obtained according to the camera imaging model and the geometric features of the citrus. The experimental results show that the recall rates of citrus detection under non-uniform illumination conditions, weak illumination, and well illumination are 99.55%, 98.47%, and 98.48%, respectively, approximately 2–9% higher than those of the original YOLO v5s network. The average error of the distance between the citrus fruit and the camera is 3.98 mm, and the average errors of the citrus diameters in the 3D direction are less than 2.75 mm. The average detection time per frame is 78.96 ms. The results indicate that our method can detect and localize citrus fruits in the complex environment of orchards with high accuracy and speed. Our dataset and codes are available at https://github.com/AshesBen/citrus-detection-localization.

Pomelo Tree Detection Method Based on Attention Mechanism and Cross-Layer Feature Fusion

Fruits hidden by green: an improved YOLOV8n for detection of young citrus in lush citrus trees

Lightweight and efficient deep learning models for fruit detection in orchards

CitrusYOLO: A Algorithm for Citrus Detection under Orchard Environment Based on YOLOv4

Real-time Fruit Detection Method Based on RGB-D Image Fusion

Real-time and Accurate Detection of Citrus in Complex Scenes Based on HPL-YOLOv4.

Fruit Target Detection Based on BCo-YOLOv5 Model

A Multiscale Lightweight and Efficient Model Based on YOLOv7: Applied to Citrus Orchard

A Method of Robot Picking Citrus Based on 3D Detection

DeepFruits: A Fruit Detection System Using Deep Neural Networks

Detecting Citrus in Orchard Environment by Using Improved YOLOv4

YOLOC-tiny: a generalized lightweight real-time detection model for multiripeness fruits of large non-green-ripe citrus in unstructured environments

Dragon fruit detection in natural orchard environment by integrating lightweight network and attention mechanism

BFP Net: Balanced Feature Pyramid Network for Small Apple Detection in Complex Orchard Environment

LDC-PP-YOLOE: a lightweight model for detecting and counting citrus fruit

Multi-stage Citrus Detection Based on Improved Yolov4

A Lightweight and High-Precision Passion Fruit YOLO Detection Model for Deployment in Embedded Devices

Recognition and Detection of Persimmon in a Natural Environment Based on an Improved YOLOv5 Model

Research on Apple Recognition Algorithm in Complex Orchard Environment Based on Deep Learning

Detection and localization of citrus fruit based on improved You Only Look Once v5s and binocular vision in the orchard

Detection in Complex Scenes Using Rgb and Depth Multimodal Feature Fusion.