Abstract:urate peach detection is a prerequisite for automated agronomic management, e.g., peach mechanical harvesting. However, due to uneven illumination and ubiquitous occlusion, it is challenging to detect the peaches, especially when the peaches are bagged in orchards. To this end, an accurate multi-class peach detection method was proposed by means of improving YOLOv5s and using multi-modal visual data for mechanical harvesting in this paper. RGB-D dataset with multi-class annotations of naked and bagging peach was proposed, including 4127 multimodal images of corresponding pixel-aligned color, depth, and infrared images acquired with consumer-level RGBD camera. Subsequently, an improved lightweight YOLOv5s (small depth) model was put forward by introducing a direction-aware and position-sensitive attention mechanism, which could capture long-range dependencies along one spatial direction and preserve precise positional information along the other spatial direction, helping the networks accurately detect peach targets. Meanwhile, the depthwise separable convolution was employed to reduce the model computation by decomposing the convolution operation into convolution in the depth direction and convolution in the width and height directions, which helped to speed up the training and inference of the network while maintaining accuracy. The comparison experimental results demonstrated that the improved YOLOv5s using multimodal visual data recorded the detection mAP of 98.6% and 88.9% on the naked and bagging peach with 5.05 M model parameters in complex illumination and severe occlusion environment, increasing by 5.3% and 16.5% than only using RGB images, as well as by 2.8% and 6.2% when compared to YOLOv5s. As compared with other networks in detecting bagging peaches, the improved YOLOv5s performed best in terms of mAP, which was 16.3%, 8.1% and 4.5% higher than YOLOX-Nano, PP-YOLO-Tiny, and EfficientDet-D0, respectively. In addition, the proposed improved YOLOv5s model offered better results in different degrees than other methods in detecting Fuji apple and Hayward kiwifruit, verified the effectiveness on different fruit detection tasks. Further investigation revealed the contribution of each imaging modality, as well as the proposed improvement in YOLOv5s, to favorable detection results of both naked and bagging peaches in natural orchards. Additionally, on the popular mobile hardware plat‐form, it was found out that the improved YOLOv5s model could implement 19 times detection per second with the considered five-channel multi-modal images, offering real-time peach detection. These promising results demonstrated the potential of the improved YOLOv5s and multi-modal visual data with multi-class annotations to achieve visual intelligence of automated fruit harvesting systems.

Application of deep learning diagnosis for multiple traits sorting in peach fruit

Quality detection and grading of peach fruit based on image processing method and neural networks in agricultural industry

Apple Ripeness Identification Using Deep Learning

Evaluation of peach quality indices using an electronic nose by MLR, QPST and BP network

A general purpose multi-fruit system for assessing the quality of fruits with the application of recurrent neural network

Fruit Detection and Three-Stage Maturity Grading Using CNN

Evaluation of Peach Quality Attribute Using an Electronic Nose

Application of Electronic Nose and Statistical Analysis to Predict Quality Indices of Peach

Deep Learning-Based Fruit Recognition and Quality Assessment: A Convolutional Neural Network Approach

Facilitated machine learning for image-based fruit quality assessment

Computer Vision System for Mango Fruit Defect Detection Using Deep Convolutional Neural Network

Appearance quality classification method of Huangguan pear under complex background based on instance segmentation and semantic segmentation

Prediction of Fruit Maturity, Quality, and Its Life Using Deep Learning Algorithms

Deep neural network based fruit identification and grading system for precision agriculture

An Application of Deep Learning for Sweet Cherry Phenotyping using YOLO Object Detection

Multi-Class on-Tree Peach Detection Using Improved YOLOv5s and Multi-Modal Images

Hyperspectral Imaging Combined with Multivariate Analysis and Band Math for Detection of Common Defects on Peaches (prunus Persica)

AI in Business AI Driven Water Content Measurement for Detecting Internal Rot in Fruits: Enhancing Quality Control in Grocery Stores

CucumberAI: Cucumber Fruit Morphology Identification System Based on Artificial Intelligence

Peaches Detection Using a Deep Learning Technique—A Contribution to Yield Estimation, Resources Management, and Circular Economy