Abstract:Recognition and localization of fruits are key components to achieve automated fruit picking. However, current neural-network-based fruit recognition algorithms have disadvantages such as high complexity. Traditional stereo matching algorithms also have low accuracy. To solve these problems, this study targeting greenhouse tomatoes proposed an algorithm framework based on YOLO-TomatoSeg, a lightweight tomato instance segmentation model improved from YOLOv5n-seg, and an accurate tomato localization approach using RAFT-Stereo disparity estimation and least squares point cloud fitting. First, binocular tomato images were captured using a binocular camera system. The left image was processed by YOLO-TomatoSeg to segment tomato instances and generate masks. Concurrently, RAFT-Stereo estimated image disparity for computing the original depth point cloud. Then, the point cloud was clipped by tomato masks to isolate tomato point clouds, which were further preprocessed. Finally, a least squares sphere fitting method estimated the 3D centroid co-ordinates and radii of tomatoes by fitting the tomato point clouds to spherical models. The experimental results showed that, in the tomato instance segmentation stage, the YOLO-TomatoSeg model replaced the Backbone network of YOLOv5n-seg with the building blocks of ShuffleNetV2 and incorporated an SE attention module, which reduced model complexity while improving model segmentation accuracy. Ultimately, the YOLO-TomatoSeg model achieved an AP of 99.01% with a size of only 2.52 MB, significantly outperforming mainstream instance segmentation models such as Mask R-CNN (98.30% AP) and YOLACT (96.49% AP). The model size was reduced by 68.3% compared to the original YOLOv5n-seg model. In the tomato localization stage, at the range of 280 mm to 480 mm, the average error of the tomato centroid localization was affected by occlusion and sunlight conditions. The maximum average localization error was ±5.0 mm, meeting the localization accuracy requirements of the tomato-picking robots. This study developed a lightweight tomato instance segmentation model and achieved accurate localization of tomato, which can facilitate research, development, and application of fruit-picking robots.

Mature Tomato Fruit Detection Algorithm Based on Improved HSV and Watershed Algorithm

Computer Vision-Based Localisation of Picking Points for Automatic Litchi Harvesting Applications Towards Natural Scenarios

Detection and Segmentation of Mature Green Tomatoes Based on Mask R-CNN with Automatic Image Acquisition Approach

Efficient tomato harvesting robot based on image processing and deep learning

An FCM-based Method to Recognize and Extract Ripe Tomato for Harvesting Robotic System

Clustered tomato detection and picking point location using machine learning-aided image analysis for automatic robotic harvesting

A Lightweight Cherry Tomato Maturity Real-Time Detection Algorithm Based on Improved YOLOV5n

A Lightweight YOLOv8 Tomato Detection Algorithm Combining Feature Enhancement and Attention

Mature tomato recognition and location algorithm based on binocular vision and deep learning

Detecting ripe fruits under natural occlusion and illumination conditions

Benchmark of Deep Learning and a Proposed HSV Colour Space Models for the Detection and Classification of Greenhouse Tomato

Intelligent Tomato Picking Robot System Based on Multimodal Depth Feature Analysis Method

Design a Robot System for Tomato Picking Based on YOLO v5

Tomato Recognition Method Based on the YOLOv8-Tomato Model in Complex Greenhouse Environments

Fruit maturity and location identification of beef tomato using R-CNN and binocular imaging technology

Tomato Maturity Detection and Counting Model Based on MHSA-YOLOv8

Development of Monitoring Robot System for Tomato Fruits in Hydroponic Greenhouses

Fruit growing direction recognition and nesting grasping strategies for tomato harvesting robots

Tomato Recognition and Localization Method Based on Improved YOLOv5n-seg Model and Binocular Stereo Vision

YOLO-Tomato: A Robust Algorithm for Tomato Detection Based on YOLOv3

Enhanced precision in greenhouse tomato recognition and localization