YOLO5-spear: A robust and real-time spear tips locator by improving image augmentation and lightweight network for selective harvesting robot of white asparagus

Ping Zhang,Xuemei Liu,Jin Yuan,Chengliang Liu
DOI: https://doi.org/10.1016/j.biosystemseng.2022.04.006
IF: 5.002
2022-01-01
Biosystems Engineering
Abstract:The characteristics of underground cultivation have posed a challenge to the development of white asparagus selective harvesting robot. Because the ridge surface is mixed with soil particles and has a complex background including variable soil moisture and illumination, detecting and locating spear tips rapidly and accurately is a key difficulty. To address this problem, image augmentation is applied firstly to extract spear tip patches from the harvesting area image in actual scenarios to form a multi-scale combined image, and then processed with proposed resampling-based image transformation, such as illumination, rotation, mirror, motion blur, and shadow. Secondly, a model referred to as YOLO5-Spear is proposed to detect spear tips by replacing C3 and Conv of YOLO5 with LC3 (Light C3) and DWConv (Depthwise-separable Convolution) and by adding the SE (Squeeze-and -Excitation) module to improve both the detection speed and accuracy. Finally, the model is deployed on embedded devices as a spear tips locator for a selective harvesting robot. The results showed that YOLO5-Spear achieved 97.8% at AP0.5, 2.4% higher than YOLO5. Moreover, its parameters, computation, model size, and detection time were reduced by 51.3%, 33.7%, 50.3%, and 18.2%, respectively. Further, the average inference time on Jetson Nano decreased to 63 ms, which meets the requirement for real-time performance of robotic harvesting. Compared with YOLO4-scaled, YOLO5-Spear increased accuracy by a maximum of 31.4%, was nearly 5 times faster, and reduced the model size by 94.9%. Localisation accuracy in different scenarios offers directions to optimise robot design and planting patterns to reduce the complexity. (c) 2022 IAgrE. Published by Elsevier Ltd. All rights reserved.
What problem does this paper attempt to address?