Abstract:Real-time detection of apples in natural environment is a necessary condition for robots to pick apples automatically, and it is also a key technique for orchard yield prediction and fine management. To make the harvesting robots detect apples quickly and accurately in complex environment, a Des-YOLO v4 algorithm and a detection method of apples are proposed. Compared with the current mainstream detection algorithms, YOLO v4 has better detection performance. However, the complex network structure of YOLO v4 will reduce the picking efficiency of the robot. Therefore, a Des-YOLO structure is proposed, which reduces network parameters and improves the detection speed of the algorithm. In the training phase, the imbalance of positive and negative samples will cause false detection of apples. To solve the above problem, a class loss function based on AP-Loss (Average Precision Loss) is proposed to improve the accuracy of apple recognition. Traditional YOLO algorithm uses NMS (Nonmaximum Suppression) method to filter the prediction boxes, but NMS cannot detect the adjacent apples when they overlap each other. Therefore, Soft-NMS is used instead of NMS to solve the problem of missing detection, so as to improve the generalization of the algorithm. The proposed algorithm is tested on the self-made apple image data set. The results show that Des-YOLO v4 network has ideal features with a mAP (mean Average Precision) of apple detection of 97.13%, a recall rate of 90%, and a detection speed of 51 f/s. Compared with traditional network models such as YOLO v4 and Faster R-CNN, the Des-YOLO v4 can meet the accuracy and speed requirements of apple detection at the same time. Finally, the self-designed apple-harvesting robot is used to carry out the harvesting experiment. The experiment shows that the harvesting time is 8.7 seconds and the successful harvesting rate of the robot is 92.9%. Therefore, the proposed apple detection method has the advantages of higher recognition accuracy and faster recognition speed. It can provide new solutions for apple-harvesting robots and new ideas for smart agriculture.

Design and Research on Vision System of Apple Harvesting Robot

Research on Trunk and Branch Recognition Method of Apple Harvesting Robot

Recognition and Localization Method of Overlapping Apples for Apple Harvesting Robot

Research on the Strategy of Advancing Harvest Efficiency of Fruit Harvest Robot in the Oscillation Conditions

Dynamic Recognition of Oscillating Fruit for Harvesting Robot

Algorithm Design and Integration for a Robotic Apple Harvesting System

Fast positioning method of apple harvesting robot for oscillating fruit

Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots

System Design and Control of an Apple Harvesting Robot

An automated apple harvesting robot—From system design to field evaluation

Intelligent robotics harvesting system process for fruits grasping prediction

Design of an Intelligent Vision Algorithm for Recognition and Classification of Apples in an Orchard Scene

Recognition and Localization Methods for Vision-Based Fruit Picking Robots: A Review

Vision system of apple picking robot based on twin support vector machine

Developments in Automated Harvesting Equipment for the Apple in the Orchard: Review

Research on Robot Image Recognition Based on Deep Learning

Vision System for Aerial Robot

An Apple Detection Method Based on Des-YOLO v4 Algorithm for Harvesting Robots in Complex Environment

Design and application of visual system in the Agaricus bisporus picking robot

Real-Time Fruit Recognition and Grasping Estimation for Autonomous Apple Harvesting

Precise Apple Detection and Localization in Orchards using YOLOv5 for Robotic Harvesting Systems