Abstract:In orchard fruit picking systems for pears, the challenge is to identify the full shape of the soft fruit to avoid injuries while using robotic or automatic picking systems. Advancements in computer vision have brought the potential to train for different shapes and sizes of fruit using deep learning algorithms. In this research, a fruit recognition method for robotic systems was developed to identify pears in a complex orchard environment using a 3D stereo camera combined with Mask Region-Convolutional Neural Networks (Mask R-CNN) deep learning technology to obtain targets. This experiment used 9054 RGBA original images (3018 original images and 6036 augmented images) to create a dataset divided into a training, validation, and testing sets. Furthermore, we collected the dataset under different lighting conditions at different times which were high-light (9–10 am) and low-light (6–7 pm) conditions at JST, Tokyo Time, August 2021 (summertime) to prepare training, validation, and test datasets at a ratio of 6:3:1. All the images were taken by a 3D stereo camera which included PERFORMANCE, QUALITY, and ULTRA models. We used the PERFORMANCE model to capture images to make the datasets; the camera on the left generated depth images and the camera on the right generated the original images. In this research, we also compared the performance of different types with the R-CNN model (Mask R-CNN and Faster R-CNN); the mean Average Precisions (mAP) of Mask R-CNN and Faster R-CNN were compared in the same datasets with the same ratio. Each epoch in Mask R-CNN was set at 500 steps with total 80 epochs. And Faster R-CNN was set at 40,000 steps for training. For the recognition of pears, the Mask R-CNN, had the mAPs of 95.22% for validation set and 99.45% was observed for the testing set. On the other hand, mAPs were observed 87.9% in the validation set and 87.52% in the testing set using Faster R-CNN. The different models using the same dataset had differences in performance in gathering clustered pears and individual pear situations. Mask R-CNN outperformed Faster R-CNN when the pears are densely clustered at the complex orchard. Therefore, the 3D stereo camera-based dataset combined with the Mask R-CNN vision algorithm had high accuracy in detecting the individual pears from gathered pears in a complex orchard environment.

In-Field Recognition and Navigation Path Extraction for Pineapple Harvesting Robots.

Recognition and Localization Method of Overlapping Apples for Apple Harvesting Robot

Research on Trunk and Branch Recognition Method of Apple Harvesting Robot

Computer Vision-Based Localisation of Picking Points for Automatic Litchi Harvesting Applications Towards Natural Scenarios

Pineapple (Ananas comosus) fruit detection and localization in natural environment based on binocular stereo vision and improved YOLOv3 model

Auto Recognition of Navigation Path for Harvest Robot Based on Machine Vision.

A Visual Recognition and Path Planning Method for Intelligent Fruit-Picking Robots

Adaptive Multi-ROI Agricultural Robot Navigation Line Extraction Based on Image Semantic Segmentation

Research on Spatial Positioning System of Fruits to be Picked in Field Based on Binocular Vision and SSD Model

Autonomous navigation for a wolfberry picking robot using visual cues and fuzzy control

Development of Real-time Recognition and Localization Methods for Fruits and Vegetables in Field

Recognition and Localization Methods for Vision-Based Fruit Picking Robots: A Review

Intelligent robotics harvesting system process for fruits grasping prediction

Geometry-Aware Fruit Grasping Estimation for Robotic Harvesting in Orchards

Real-time pineapple detection for agricultural robot via lightweight YOLOv7-tiny model

Pear Recognition in an Orchard from 3D Stereo Camera Datasets to Develop a Fruit Picking Mechanism Using Mask R-CNN

Fruit Detection and Pose Estimation for Grape Cluster-Harvesting Robot Using Binocular Imagery Based on Deep Neural Networks

Design and Implementation of an Artificial Intelligence of Things-Based Autonomous Mobile Robot System for Pitaya Harvesting

Fruit growing direction recognition and nesting grasping strategies for tomato harvesting robots

Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots

Fruit position recognition and ripeness determination methods for a fruit automatic harvesting robot