Abstract:Ground vehicles equipped with monocular vision systems are a valuable source of high resolution image data for precision agriculture applications in orchards. This paper presents an image processing framework for fruit detection and counting using orchard image data. A general purpose image segmentation approach is used, including two feature learning algorithms; multi-scale Multi-Layered Perceptrons (MLP) and Convolutional Neural Networks (CNN). These networks were extended by including contextual information about how the image data was captured (metadata), which correlates with some of the appearance variations and/or class distributions observed in the data. The pixel-wise fruit segmentation output is processed using the Watershed Segmentation (WS) and Circular Hough Transform (CHT) algorithms to detect and count individual fruits. Experiments were conducted in a commercial apple orchard near Melbourne, Australia. The results show an improvement in fruit segmentation performance with the inclusion of metadata on the previously benchmarked MLP network. We extend this work with CNNs, bringing agrovision closer to the state-of-the-art in computer vision, where although metadata had negligible influence, the best pixel-wise F1-score of $0.791$ was achieved. The WS algorithm produced the best apple detection and counting results, with a detection F1-score of $0.858$. As a final step, image fruit counts were accumulated over multiple rows at the orchard and compared against the post-harvest fruit counts that were obtained from a grading and counting machine. The count estimates using CNN and WS resulted in the best performance for this dataset, with a squared correlation coefficient of $r^2=0.826$.

Application of Image Segmentation Algorithm Based on Entropy Clustering in Apple Harvesting Robot

Recognition and Localization Method of Overlapping Apples for Apple Harvesting Robot

Night Vision Image De-Noising Of Apple Harvesting Robots Based On The Wavelet Fuzzy Threshold

Research on Trunk and Branch Recognition Method of Apple Harvesting Robot

An Apple Image Segmentation Method Based on a Color Index Obtained by a Genetic Algorithm

Multi-Feature Patch-Based Segmentation Technique in the Gray-Centered RGB Color Space for Improved Apple Target Recognition

Image Segmentation Based on Rough Entropy and K-Means Clustering Algorithm

Target Recognition and Trajectory Planning of Apple Harvesting Robot considering Color Multimedia Image Segmentation Algorithm

A Method of Segmenting Apples Based on Gray-Centered RGB Color Space

A Method of Apple Image Segmentation Based on Color-Texture Fusion Feature and Machine Learning

Recognition and Localization of Occluded Apples Using K-means Clustering Algorithm and Convex Hull Theory: a Comparison

Shadow Detection and Removal in Apple Image Segmentation under Natural Light Conditions Using an Ultrametric Contour Map

Research on Branches Recognition of Apple Harvesting Robot

Segmentation of Foreground Apple Targets by Fusing Visual Attention Mechanism and Growth Rules of Seed Points

Detection Of Young Green Apples In Orchard Environment Using Adaptive Ratio Chromatic Aberration And Hog-Svm

Segmentation Method of Apple Image Based on GaussAdaptive Curve Fitting

Fast Location and Recognition of Green Apple Based on RGB-D Image

Segmentation Algorithm for Apple Recognition Using Image Features and Artificial Neural Network

SEGMENTATION OF APPLE POINT CLOUDS BASED ON ROI IN RGB IMAGES

Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards

Segmentation of Agriculture Images Based on Improved K-means Clustering Algorithm