Abstract:Fruit detection is a fundamental task for automatic yield estimation. The goal is to detect all the fruits in images. The-state of the art of fruit detection algorithm, Faster R-CNN, shows a lack of detection advantage on small fruits. One of the reasons is only that single-level features and a classifier are used for localization of proposal candidates. In this article, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CNN network for small fruit detection. We utilize features from three different levels to learn three classifiers for objectness classification in the stage of proposal localization. Probabilities from classifiers are combined by a simple convolutional layer to generate final objectness classification for proposal candidates. During training, in order to train a model with strong generalization capability, we propose to use correlation coefficients to measure the diversity of multiple classifiers. A novel loss function with classifier correlation is introduced to train the region proposal network. We evaluate the proposed model on two data sets of small fruits. Extensive experiments show that the proposed model outperforms the state-of-the-art detectors for fruit detection. Note to Practitioners—This article was motivated by the problem of the detection of small fruits for yield estimation but it also applies to the detection of other small objects. Existing approaches to fruit detection experience difficulty in detecting small fruits and the overall detection accuracy suffers as a result. This article suggests a novel approach to detect fruits of all sizes by using a multiple classifier fusion strategy in the stage of fruit localization. In the learning of the classifiers, multilevel features of fruits are extracted. In addition, we achieve classifier diversity through the analysis of the correlation of classification probabilities. The diversity of classifiers also helps improve the performance of fruit localization. Using experiments on real fruit tree images, we show that the suggested approach is feasible on orange detection of various sites in different seasons. As a future work, we will integrate the suggested algorithm into a fruit yield estimation system and test it online.

Faster R-CNN for Multi-Class Fruit Detection Using a Robotic Vision System

Detection of Strawberry Flowers in Outdoor Field by Deep Neural Network

High-Quality Coarse-to-Fine Fruit Detector for Harvesting Robot in Open Environment

DeepFruits: A Fruit Detection System Using Deep Neural Networks

Multi-class Fruit-on-plant Detection for Apple in SNAP System Using Faster R-CNN

&lt;i&gt;Optimized Faster R-CNN for Fruit Detection of Strawberry Harvesting Robot&lt;/i&gt;

Multi-Task Cascaded Convolutional Networks Based Intelligent Fruit Detection for Designing Automated Robot.

Real-time Fruit Detection Method Based on RGB-D Image Fusion

Faster R–CNN–based Apple Detection in Dense-Foliage Fruiting-Wall Trees Using RGB and Depth Features for Robotic Harvesting

Faster R-CNN with Classifier Fusion for Automatic Detection of Small Fruits

Fruit Detection and Pose Estimation for Grape Cluster-Harvesting Robot Using Binocular Imagery Based on Deep Neural Networks

Integrated detection of citrus fruits and branches using a convolutional neural network

A Method of Robot Picking Citrus Based on 3D Detection

Single-Shot Convolution Neural Networks for Real-Time Fruit Detection Within the Tree

Kiwifruit Detection in Field Images Using Faster R-CNN with ZFNet

Fruit Recognition, Task Plan, and Control for Apple Harvesting Robots

FF3D: A Rapid and Accurate 3D Fruit Detector for Robotic Harvesting

Fruit Quantity and Quality Estimation using a Robotic Vision System

Real-Time Fruit Recognition and Grasping Estimation for Autonomous Apple Harvesting

Improved multi-classes kiwifruit detection in orchard to avoid collisions during robotic picking

Robotic Harvesting of the Occluded Fruits with a Precise Shape and Position Reconstruction Approach