Abstract:Detecting and estimating size of apples during the early stages of growth is crucial for predicting yield, pest management, and making informed decisions related to crop-load management, harvest and post-harvest logistics, and marketing. Traditional fruit size measurement methods are laborious and time-consuming. This study employs the state-of-the-art YOLOv8 object detection and instance segmentation algorithm in conjunction with geometric shape fitting techniques on 3D point cloud data to accurately determine the size of immature green apples (or fruitlet) in a commercial orchard environment. The methodology utilized two RGB-D sensors: Intel RealSense D435i and Microsoft Azure Kinect DK. Notably, the YOLOv8 instance segmentation models exhibited proficiency in immature green apple detection, with the YOLOv8m-seg model achieving the highest AP@0.5 and AP@0.75 scores of 0.94 and 0.91, respectively. Using the ellipsoid fitting technique on images from the Azure Kinect, we achieved an RMSE of 2.35 mm, MAE of 1.66 mm, MAPE of 6.15 mm, and an R-squared value of 0.9 in estimating the size of apple fruitlets. Challenges such as partial occlusion caused some error in accurately delineating and sizing green apples using the YOLOv8-based segmentation technique, particularly in fruit clusters. In a comparison with 102 outdoor samples, the size estimation technique performed better on the images acquired with Microsoft Azure Kinect than the same with Intel Realsense D435i. This superiority is evident from the metrics: the RMSE values (2.35 mm for Azure Kinect vs. 9.65 mm for Realsense D435i), MAE values (1.66 mm for Azure Kinect vs. 7.8 mm for Realsense D435i), and the R-squared values (0.9 for Azure Kinect vs. 0.77 for Realsense D435i). This study demonstrated the feasibility of accurately sizing immature green fruit in early growth stages using the combined 3D sensing and shape-fitting technique, which shows promise for improved precision agricultural operations such as optimal crop-load management in orchards.

Automatic segmentation of trees in dynamic outdoor environments

Semantic segmentation for partially occluded apple trees based on deep learning

Application of Image Segmentation Algorithm Based on Entropy Clustering in Apple Harvesting Robot

Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards

Semantic Segmentation of Fruits on Multi-sensor Fused Data in Natural Orchards

Citrus Tree Segmentation from UAV Images Based on Monocular Machine Vision in a Natural Orchard Environment

Optical flow-based branch segmentation for complex orchard environments

Immature Green Apple Detection and Sizing in Commercial Orchards Using YOLOv8 and Shape Fitting Techniques

Autonomous Apple Fruitlet Sizing and Growth Rate Tracking using Computer Vision

Automatic Segmentation of Foreground Objects from Multiple Images Based on Consistency Analysis

A spraying path planning algorithm based on colour-depth fusion segmentation in peach orchards

Fruit Detection, Segmentation and 3D Visualisation of Environments in Apple Orchards

Fast Location and Recognition of Green Apple Based on RGB-D Image

A novel algorithm for segmenting fruit from unorganized point clouds

Automatic Leaf Extraction from Outdoor Images

Road-Side Individual Tree Segmentation from Urban MLS Point Clouds Using Metric Learning

Environmentally adaptive segmentation algorithm for outdoor image segmentation

PD-SegNet: Semantic Segmentation of Small Agricultural Targets in Complex Environments

Orchard Mapping with Deep Learning Semantic Segmentation

Autonomous Navigation in Rows of Trees and High Crops with Deep Semantic Segmentation