Abstract:Accurate segmentation and detection of cucumber in non-structural environment is of great significance for automatic robotic harvesting. However, the high similarity of cucumber color to that of branches and leaves and the complexity of the growing environment make it difficult for cucumbers to be correctly detected by harvesting robots. This paper presented an improved automatic method for segmentation and recognition of matured cucumbers that combines data processing, single-stage target recognition network (YOLO-v3 and SSD), U-Net semantic segmentation network and Transfer Learning. The algorithm adopted a multi-network model fusion strategy; an improved U-Net model was implemented for fine pixel-wise cucumber segmentation, and then YOLO-v3 or SSD model was applied for cucumber recognition based on the segmented images. In order to deeply analyze the segmentation process and recognition principle on cucumber images, several main feature layers in YOLO-v3 and U-Net network were respectively visualized to demonstrate the integrated feature maps learned and model interpretability. In parallel, to accelerate convergence and improve model detection robustness, Transfer Learning method was introduced to optimize network structure. All three network models combined with Transfer Learning showed good performance in cucumber image processing. For the two target recognition models, i.e., YOLO-v3 and SSD, the F1 scores were above 0.95, and the average accuracy (AP) was as high as 99%, but overall, the recognition performance of YOLO-v3 was still higher than that of the SSD model. Meanwhile, the mIOU value and average pixel accuracy of U-Net semantic segmentation model also reached 94.24% and 97.46%, respectively. Finally, YOLO-v3 was combined with the U-Net segmented image to further implement cucumber detection. Compared to the individual YOLO-v3 model, the hybrid U-Net + YOLO-v3 model obtained better prediction results with higher AP, F1 score and precision values, especially the AP value, which improved by 6%. The detection model combined with U-Net and YOLO-V3 network can enhance the shape, texture and other feature information of green cucumbers in complex agricultural environment, and improve the accurate positioning and grasping of the harvesting robot.

Transfer Learning Based Fruits Image Segmentation for Fruit-Picking Robots

Few-Shot Fruit Segmentation via Transfer Learning

Fruit Image Classification Based on MobileNetV2 with Transfer Learning Technique

DLNet: Accurate segmentation of green fruit in obscured environments

A novel green apple segmentation algorithm based on ensemble U-Net under complex orchard environment

Integrated detection of citrus fruits and branches using a convolutional neural network

Accurate segmentation of green fruit based on optimized mask RCNN application in complex orchard

Fruit Detection from Digital Images Using CenterNet

An Accurate Detection and Segmentation Model of Obscured Green Fruits

YOLOF-Snake: An Efficient Segmentation Model for Green Object Fruit

DeepFruits: A Fruit Detection System Using Deep Neural Networks

Real-time, Highly Accurate Robotic Grasp Detection Utilizing Transfer Learning for Robots Manipulating Fragile Fruits with Widely Variable Sizes and Shapes

Fruit Detection and Pose Estimation for Grape Cluster-Harvesting Robot Using Binocular Imagery Based on Deep Neural Networks

Multi-network Fusion Algorithm with Transfer Learning for Green Cucumber Segmentation and Recognition under Complex Natural Environment

Tomato Recognition and Localization Method Based on Improved YOLOv5n-seg Model and Binocular Stereo Vision

Fruit Detection, Segmentation and 3D Visualisation of Environments in Apple Orchards

LFPNet: Lightweight Network on Real Point Sets for Fruit Classification and Segmentation

Semantic Segmentation of Fruits on Multi-sensor Fused Data in Natural Orchards

SE-COTR: A Novel Fruit Segmentation Model for Green Apples Application in Complex Orchard

Detection and Segmentation of Mature Green Tomatoes Based on Mask R-CNN with Automatic Image Acquisition Approach

An Improved Yolov3 Based on Dual Path Network for Cherry Tomatoes Detection