Abstract:In automated fruit detection, RGB-Depth (RGB-D) images aid the detection model with additional depth information to enhance detection accuracy. However, outdoor depth images are usually of low quality, which limits the quality of depth data. In this study, an approach/technique for real-time apple fruit detection in a high-density orchard environment by using multi-modal data is presented. Non-targeted background removal using the depth fusion (NBR-DF) method was developed to reduce the high noise condition of depth images. The noise occurred due to the uncontrolled lighting condition and holes with incomplete depth information in the depth images. NBR-DF technique follows three primary steps: pre-processing of depth images (point cloud generation), target object extraction, and background removal. The NBR-DF method serves as a pipeline to pre-process multi-modal data to enhance features of depth images by filling holes to eliminate noise generated by depth holes. Further, the NBR-DF implemented with the YOLOv5 enhances the detection accuracy in dense orchard conditions by using multi-modal information as input. An attention-based depth fusion module that adaptively fuses the multi-modal features was developed. The integration of the depth-attention matrix involved pooling operations and sigmoid normalization, both of which are efficient methods for summarizing and normalizing depth information. The fusion module improves the identification of multiscale objects and strengthens the network's resistance to noise. The network then detects the fruit position using multiscale information from the RGB-D images in highly complex orchard environments. The detection results were compared and validated with other methods using different input modals and fusion strategies. The results showed that the detection accuracy using the NBR-DF approach achieved an average precision rate of 0.964 in real time. The performance comparison with other state-of-the-art methods and the model generalization study also establish that the present advanced depth-fusion attention mechanism and effective preprocessing steps in NBR-DF-YOLOv5 significantly surpass those in performance. In conclusion, the developed NBR-DF technique showed the potential to improve real-time apple fruit detection using multi-modal information.

Improving real-time apple fruit detection: Multi-modal data and depth fusion with non-targeted background removal

Real-time Fruit Detection Method Based on RGB-D Image Fusion

A Seamless Deep Learning Approach for Apple Detection, Depth Estimation, and Tracking Using YOLO Models Enhanced by Multi-Head Attention Mechanism

A Real-Time Kiwifruit Detection Based on Improved YOLOv7.

An Improved Apple Object Detection Method Based on Lightweight YOLOv4 in Complex Backgrounds

Research on Apple Recognition Algorithm in Complex Orchard Environment Based on Deep Learning

Single-Shot Convolution Neural Networks for Real-Time Fruit Detection Within the Tree

Lightweight and efficient deep learning models for fruit detection in orchards

DeepFruits: A Fruit Detection System Using Deep Neural Networks

Immature Green Apple Detection and Sizing in Commercial Orchards Using YOLOv8 and Shape Fitting Techniques

Lightweight Apple Detection in Complex Orchards Using YOLOV5-PRE

Apple stem/calyx real-time recognition using YOLO-v5 algorithm for fruit automatic loading system

Deep Learning-Based Apple Detection with Attention Module and Improved Loss Function in YOLO

Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking Methods

Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting

A Two-Stage Deep-Learning Model for Detection and Occlusion-Based Classification of Kashmiri Orchard Apples for Robotic Harvesting

MLP-based multimodal tomato detection in complex scenarios: Insights from task-specific analysis of feature fusion architectures

A Real-Time Apple Targets Detection Method for Picking Robot Based on Improved YOLOv5

Real-Time Apple Detection System Using Embedded Systems With Hardware Accelerators: An Edge AI Application

Fast Location and Recognition of Green Apple Based on RGB-D Image