Abstract:In orchard fruit picking systems for pears, the challenge is to identify the full shape of the soft fruit to avoid injuries while using robotic or automatic picking systems. Advancements in computer vision have brought the potential to train for different shapes and sizes of fruit using deep learning algorithms. In this research, a fruit recognition method for robotic systems was developed to identify pears in a complex orchard environment using a 3D stereo camera combined with Mask Region-Convolutional Neural Networks (Mask R-CNN) deep learning technology to obtain targets. This experiment used 9054 RGBA original images (3018 original images and 6036 augmented images) to create a dataset divided into a training, validation, and testing sets. Furthermore, we collected the dataset under different lighting conditions at different times which were high-light (9–10 am) and low-light (6–7 pm) conditions at JST, Tokyo Time, August 2021 (summertime) to prepare training, validation, and test datasets at a ratio of 6:3:1. All the images were taken by a 3D stereo camera which included PERFORMANCE, QUALITY, and ULTRA models. We used the PERFORMANCE model to capture images to make the datasets; the camera on the left generated depth images and the camera on the right generated the original images. In this research, we also compared the performance of different types with the R-CNN model (Mask R-CNN and Faster R-CNN); the mean Average Precisions (mAP) of Mask R-CNN and Faster R-CNN were compared in the same datasets with the same ratio. Each epoch in Mask R-CNN was set at 500 steps with total 80 epochs. And Faster R-CNN was set at 40,000 steps for training. For the recognition of pears, the Mask R-CNN, had the mAPs of 95.22% for validation set and 99.45% was observed for the testing set. On the other hand, mAPs were observed 87.9% in the validation set and 87.52% in the testing set using Faster R-CNN. The different models using the same dataset had differences in performance in gathering clustered pears and individual pear situations. Mask R-CNN outperformed Faster R-CNN when the pears are densely clustered at the complex orchard. Therefore, the 3D stereo camera-based dataset combined with the Mask R-CNN vision algorithm had high accuracy in detecting the individual pears from gathered pears in a complex orchard environment.

&lt;i&gt;Optimized Faster R-CNN for Fruit Detection of Strawberry Harvesting Robot&lt;/i&gt;

Detection of Strawberry Flowers in Outdoor Field by Deep Neural Network

Accurate Strawberry Plant Detection System Based on Low-altitude Remote Sensing and Deep Learning Technologies

Dynamic Recognition of Oscillating Fruit for Harvesting Robot

Faster R-CNN for Multi-Class Fruit Detection Using a Robotic Vision System

Strawberry R-CNN: Recognition and counting model of strawberry based on improved faster R-CNN

Fruit Detection and Pose Estimation for Grape Cluster-Harvesting Robot Using Binocular Imagery Based on Deep Neural Networks

DeepFruits: A Fruit Detection System Using Deep Neural Networks

Real-time Fruit Detection Method Based on RGB-D Image Fusion

CES-YOLOv8: Strawberry Maturity Detection Based on the Improved YOLOv8

Robotic Harvesting of the Occluded Fruits with a Precise Shape and Position Reconstruction Approach

A Small Target Strawberry Recognition Method Based on Improved YOLOv8n Model

An Accurate Detection and Segmentation Model of Obscured Green Fruits

A Method of Robot Picking Citrus Based on 3D Detection

A fine recognition method of strawberry ripeness combining Mask R-CNN and region segmentation

Fruit fast tracking and recognition of apple picking robot based on improved YOLOv5

A Novel Greenhouse-Based System for the Detection and Plumpness Assessment of Strawberry Using an Improved Deep Learning Technique

Real-Time Detection and Instance Segmentation of Strawberry in Unstructured Environment

Detecting ripe fruits under natural occlusion and illumination conditions

Pear Recognition in an Orchard from 3D Stereo Camera Datasets to Develop a Fruit Picking Mechanism Using Mask R-CNN

&amp;lt;i&amp;gt;Optimized Faster R-CNN for Fruit Detection of Strawberry Harvesting Robot&amp;lt;/i&amp;gt;