Abstract:In orchard fruit picking systems for pears, the challenge is to identify the full shape of the soft fruit to avoid injuries while using robotic or automatic picking systems. Advancements in computer vision have brought the potential to train for different shapes and sizes of fruit using deep learning algorithms. In this research, a fruit recognition method for robotic systems was developed to identify pears in a complex orchard environment using a 3D stereo camera combined with Mask Region-Convolutional Neural Networks (Mask R-CNN) deep learning technology to obtain targets. This experiment used 9054 RGBA original images (3018 original images and 6036 augmented images) to create a dataset divided into a training, validation, and testing sets. Furthermore, we collected the dataset under different lighting conditions at different times which were high-light (9–10 am) and low-light (6–7 pm) conditions at JST, Tokyo Time, August 2021 (summertime) to prepare training, validation, and test datasets at a ratio of 6:3:1. All the images were taken by a 3D stereo camera which included PERFORMANCE, QUALITY, and ULTRA models. We used the PERFORMANCE model to capture images to make the datasets; the camera on the left generated depth images and the camera on the right generated the original images. In this research, we also compared the performance of different types with the R-CNN model (Mask R-CNN and Faster R-CNN); the mean Average Precisions (mAP) of Mask R-CNN and Faster R-CNN were compared in the same datasets with the same ratio. Each epoch in Mask R-CNN was set at 500 steps with total 80 epochs. And Faster R-CNN was set at 40,000 steps for training. For the recognition of pears, the Mask R-CNN, had the mAPs of 95.22% for validation set and 99.45% was observed for the testing set. On the other hand, mAPs were observed 87.9% in the validation set and 87.52% in the testing set using Faster R-CNN. The different models using the same dataset had differences in performance in gathering clustered pears and individual pear situations. Mask R-CNN outperformed Faster R-CNN when the pears are densely clustered at the complex orchard. Therefore, the 3D stereo camera-based dataset combined with the Mask R-CNN vision algorithm had high accuracy in detecting the individual pears from gathered pears in a complex orchard environment.

Peach Tree Detection for Weeding Robot Based on Faster-RCNN

Detection of Strawberry Flowers in Outdoor Field by Deep Neural Network

Accurate Strawberry Plant Detection System Based on Low-altitude Remote Sensing and Deep Learning Technologies

Image Detection of Peach Diseases and Pests

Peaches Detection Using a Deep Learning Technique—A Contribution to Yield Estimation, Resources Management, and Circular Economy

Fast detection method of green peach for application of picking robot

A Faster R-CNN-Based Model for the Identification of Weed Seedling

CRLNet: A Multimodal Peach Detection Network Based on Cooperative Asymptotic Enhancement and the Fusion of Granularity Refinement

Application of an improved U-Net with image-to-image translation and transfer learning in peach orchard segmentation

DeepFruits: A Fruit Detection System Using Deep Neural Networks

Beet seedling and weed recognition based on convolutional neural network and multi-modality images

Real-time Fruit Detection Method Based on RGB-D Image Fusion

Pear Recognition in an Orchard from 3D Stereo Camera Datasets to Develop a Fruit Picking Mechanism Using Mask R-CNN

Detection and Segmentation of Mature Green Tomatoes Based on Mask R-CNN with Automatic Image Acquisition Approach

Detection and Reconstruction of Passion Fruit Branches via CNN and Bidirectional Sector Search

Deep Learning-Based Segmentation of Peach Diseases Using Convolutional Neural Network

WeedNet-R: a sugar beet field weed detection algorithm based on enhanced RetinaNet and context semantic fusion

Fruit Detection and Pose Estimation for Grape Cluster-Harvesting Robot Using Binocular Imagery Based on Deep Neural Networks

Accurate segmentation of green fruit based on optimized mask RCNN application in complex orchard

Deep learning based weed detection and target spraying robot system at seedling stage of cotton field

Improving U-net network for semantic segmentation of corns and weeds during corn seedling stage in field