Abstract:Robotic pruning of dormant grapevines is an area of active research in order to promote vine balance and grape quality, but so far robotic efforts have largely focused on planar, simplified vines not representative of commercial vineyards. This paper aims to advance the robotic perception capabilities necessary for pruning in denser and more complex vine structures by extending plant skeletonization techniques. The proposed pipeline generates skeletal grapevine models that have lower reprojection error and higher connectivity than baseline algorithms. We also show how 3D and skeletal information enables prediction accuracy of pruning weight for dense vines surpassing prior work, where pruning weight is an important vine metric influencing pruning site selection.
What problem does this paper attempt to address?
This paper attempts to address the problem of achieving precise robotic pruning in complex grapevine structures. Specifically, existing research mainly focuses on planar, simplified grapevines, which do not fully represent the actual conditions in commercial vineyards. This paper aims to enhance the robot's perception capabilities in denser and more complex grapevine structures by extending plant skeletonization techniques. To this end, the authors propose a new skeletonization pipeline that can generate skeleton models with lower reprojection error and higher connectivity, and use 3D and skeleton information to predict pruning weight, surpassing previous research results.
### Main Issues
1. **Modeling Complex Grapevine Structures**: Existing robotic pruning methods mainly target planar, simplified grapevines and cannot handle the dense, cluttered grapevine structures common in commercial vineyards.
2. **Pruning Weight Prediction**: Pruning weight is an important indicator of grapevine health and vigor, influencing the choice of pruning points. Existing methods have low prediction accuracy when dealing with dense, occluded grapevines.
### Solutions
1. **Skeletonization Pipeline**: A new skeletonization pipeline is proposed that can generate high-quality grapevine skeleton models with high reprojection scores and connectivity.
2. **Handling Cyclic Structures**: Improved graph and refinement skeletonization strategies can handle cycles in the structure graph, resulting in more accurate skeleton models.
3. **Pruning Weight Prediction**: Using 3D data and skeleton information, the accuracy of pruning weight prediction in dense grapevines is improved.
### Specific Contributions
1. **High-Precision Skeletonization Pipeline**: The generated skeleton models have high reprojection scores and can connect skeleton segments.
2. **Ability to Handle Cyclic Structures**: Improved graph and refinement strategies can handle cycles in the structure graph, generating more accurate skeleton models.
3. **Vigor Estimation of Dense Grapevine Growth**: Using 3D data and skeleton information, the accuracy of pruning weight prediction in dense grapevines is improved.
### Dataset
The authors provide images and annotations as a public dataset, including stereo images of 144 Concord grapevines and their pruning weights. These data are used to train and evaluate the model.
### Methods
1. **Stereo Image Registration**: Register stereo images to generate a unified point cloud and create image segmentation masks.
2. **Skeleton Model Generation**: Generate skeleton models from the point cloud, including steps such as local connection graph, minimum spanning tree path, loop detection, and segment generation.
3. **Pruning Weight Prediction**: Use a linear regression model, combining five variables (number of vine voxels, number of trunk voxels, internode distance, skeleton length, number of vine pixels) to predict pruning weight.
### Experimental Results
1. **Skeletonization Results**: Compared to two baseline methods, the proposed method generates skeleton models with better performance in terms of reprojection scores and connectivity.
2. **Pruning Weight Prediction Results**: The proposed method outperforms existing methods in terms of accuracy and stability in pruning weight prediction.
### Conclusion
By improving the skeletonization method, this paper successfully achieves more precise robotic pruning in complex grapevine structures. Future work can further explore higher-capacity 2D or 3D learning methods to improve the accuracy of pruning weight prediction. Additionally, developing new types of plant skeleton models based on growth sources and possible growth paths will help address more issues related to plant dynamics and operations.