BonnBeetClouds3D: A Dataset Towards Point Cloud-based Organ-level Phenotyping of Sugar Beet Plants under Field Conditions

Elias Marks,Jonas Bömer,Federico Magistri,Anurag Sah,Jens Behley,Cyrill Stachniss

DOI: https://doi.org/10.48550/arXiv.2312.14706

2023-12-22

Abstract:Agricultural production is facing severe challenges in the next decades induced by climate change and the need for sustainability, reducing its impact on the environment. Advancements in field management through non-chemical weeding by robots in combination with monitoring of crops by autonomous unmanned aerial vehicles (UAVs) and breeding of novel and more resilient crop varieties are helpful to address these challenges. The analysis of plant traits, called phenotyping, is an essential activity in plant breeding, it however involves a great amount of manual labor. With this paper, we address the problem of automatic fine-grained organ-level geometric analysis needed for precision phenotyping. As the availability of real-world data in this domain is relatively scarce, we propose a novel dataset that was acquired using UAVs capturing high-resolution images of a real breeding trial containing 48 plant varieties and therefore covering great morphological and appearance diversity. This enables the development of approaches for autonomous phenotyping that generalize well to different varieties. Based on overlapping high-resolution images from multiple viewing angles, we compute photogrammetric dense point clouds and provide detailed and accurate point-wise labels for plants, leaves, and salient points as the tip and the base. Additionally, we include measurements of phenotypic traits performed by experts from the German Federal Plant Variety Office on the real plants, allowing the evaluation of new approaches not only on segmentation and keypoint detection but also directly on the downstream tasks. The provided labeled point clouds enable fine-grained plant analysis and support further progress in the development of automatic phenotyping approaches, but also enable further research in surface reconstruction, point cloud completion, and semantic interpretation of point clouds.

Computer Vision and Pattern Recognition,Robotics

What problem does this paper attempt to address?

The paper attempts to address the problem of achieving fine geometric analysis at the organ level of sugar beet plants under field conditions using point cloud data to support precise phenotypic analysis. Specifically, the paper aims to:

1. **Automated Fine-Grained Organ-Level Geometric Analysis**: Traditional plant phenotypic analysis typically relies on manual measurements or visual scoring, which are not only time-consuming and labor-intensive but also subjective. The paper proposes an automated method that utilizes point cloud data generated from high-resolution images to achieve precise geometric analysis of plant organs (such as leaves).

2. **Data Scarcity Issue**: In the agricultural field, especially in plant phenotypic analysis, real-world data available for fine-grained perception tasks is relatively scarce. The paper addresses this gap by collecting a large amount of data from real experimental fields, providing valuable resources for the development of automated phenotypic analysis methods.

3. **Multi-Variety Coverage**: The dataset provided in the paper covers 48 different varieties of sugar beet plants, which allows the developed algorithms to better generalize to different plant varieties, enhancing the robustness and applicability of the models.

4. **Multi-Task Evaluation**: In addition to providing task labels for segmentation and keypoint detection, the dataset also includes actual phenotypic feature values (such as leaf length, width, etc.) measured by experts. This enables researchers to directly evaluate the performance of downstream tasks.

In summary, the main goal of the paper is to promote the development of automated plant phenotypic analysis technology by providing a high-quality point cloud dataset, thereby improving the sustainability and efficiency of agricultural production.