Abstract:In orchard fruit picking systems for pears, the challenge is to identify the full shape of the soft fruit to avoid injuries while using robotic or automatic picking systems. Advancements in computer vision have brought the potential to train for different shapes and sizes of fruit using deep learning algorithms. In this research, a fruit recognition method for robotic systems was developed to identify pears in a complex orchard environment using a 3D stereo camera combined with Mask Region-Convolutional Neural Networks (Mask R-CNN) deep learning technology to obtain targets. This experiment used 9054 RGBA original images (3018 original images and 6036 augmented images) to create a dataset divided into a training, validation, and testing sets. Furthermore, we collected the dataset under different lighting conditions at different times which were high-light (9–10 am) and low-light (6–7 pm) conditions at JST, Tokyo Time, August 2021 (summertime) to prepare training, validation, and test datasets at a ratio of 6:3:1. All the images were taken by a 3D stereo camera which included PERFORMANCE, QUALITY, and ULTRA models. We used the PERFORMANCE model to capture images to make the datasets; the camera on the left generated depth images and the camera on the right generated the original images. In this research, we also compared the performance of different types with the R-CNN model (Mask R-CNN and Faster R-CNN); the mean Average Precisions (mAP) of Mask R-CNN and Faster R-CNN were compared in the same datasets with the same ratio. Each epoch in Mask R-CNN was set at 500 steps with total 80 epochs. And Faster R-CNN was set at 40,000 steps for training. For the recognition of pears, the Mask R-CNN, had the mAPs of 95.22% for validation set and 99.45% was observed for the testing set. On the other hand, mAPs were observed 87.9% in the validation set and 87.52% in the testing set using Faster R-CNN. The different models using the same dataset had differences in performance in gathering clustered pears and individual pear situations. Mask R-CNN outperformed Faster R-CNN when the pears are densely clustered at the complex orchard. Therefore, the 3D stereo camera-based dataset combined with the Mask R-CNN vision algorithm had high accuracy in detecting the individual pears from gathered pears in a complex orchard environment.

Identification and Location Algorithm of Fragrant Pear Fruit and Its Application in Automatic Picking System

Feature Extraction And Recognition Based On Machine Vision Application In Lotus Picking Robot

A Novel Adaptive Zone-fusion Network for Precise Waxberry Semantic Segmentation to Improve Automated-Harvesting in a Complex Orchard Environment

Apple Ripeness Identification Using Deep Learning

Appearance quality classification method of Huangguan pear under complex background based on instance segmentation and semantic segmentation

Pear Recognition in an Orchard from 3D Stereo Camera Datasets to Develop a Fruit Picking Mechanism Using Mask R-CNN

Integrated Fruit Ripeness Assessment System Based on an Artificial Olfactory Sensor and Deep Learning

DeepFruits: A Fruit Detection System Using Deep Neural Networks

A Dragon Fruit Picking Detection Method Based on YOLOv7 and PSP-Ellipse

Fruit Detection from Digital Images Using CenterNet

YOLO-P: An efficient method for pear fast detection in complex orchard picking environment

Mature pomegranate fruit detection and location combining improved F-PointNet with 3D point cloud clustering in orchard

A Method of Robot Picking Citrus Based on 3D Detection

Prototype Network for Predicting Occluded Picking Position Based on Lychee Phenotypic Features

Research on High Performance Apple Recognition Model Construction Strategy Based on Deep Learning

The Identification, Separation, and Clamp Function of an Intelligent Flexible Blueberry Picking Robot

Adaptive Active Positioning of Camellia oleifera Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm

A general purpose multi-fruit system for assessing the quality of fruits with the application of recurrent neural network

Research on Apple Recognition Algorithm in Complex Orchard Environment Based on Deep Learning

Peaches Detection Using a Deep Learning Technique—A Contribution to Yield Estimation, Resources Management, and Circular Economy

Fruit Detection and Pose Estimation for Grape Cluster-Harvesting Robot Using Binocular Imagery Based on Deep Neural Networks