3D Reconstruction of Non-Rigid Plants and Sensor Data Fusion for Agriculture Phenotyping

Gustavo Scalabrini Sampaio,Leandro A. Silva,Maurício Marengoni

DOI: https://doi.org/10.3390/s21124115

IF: 3.9

2021-06-15

Sensors

Abstract:Technology has been promoting a great transformation in farming. The introduction of robotics; the use of sensors in the field; and the advances in computer vision; allow new systems to be developed to assist processes, such as phenotyping, of crop’s life cycle monitoring. This work presents, which we believe to be the first time, a system capable of generating 3D models of non-rigid corn plants, which can be used as a tool in the phenotyping process. The system is composed by two modules: an terrestrial acquisition module and a processing module. The terrestrial acquisition module is composed by a robot, equipped with an RGB-D camera and three sets of temperature, humidity, and luminosity sensors, that collects data in the field. The processing module conducts the non-rigid 3D plants reconstruction and merges the sensor data into these models. The work presented here also shows a novel technique for background removal in depth images, as well as efficient techniques for processing these images and the sensor data. Experiments have shown that from the models generated and the data collected, plant structural measurements can be performed accurately and the plant’s environment can be mapped, allowing the plant’s health to be evaluated and providing greater crop efficiency.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to generate accurate 3D models of non - rigid plants (such as maize) in an uncontrolled environment in agricultural phenotyping analysis and fuse sensor data into these models. Specifically, the paper proposes a system that is able to:

1. **Collect field data**: Automatically collect data in the field through a robot equipped with an RGB - D camera and multiple temperature, humidity and light sensors.
2. **Generate 3D models**: Use the collected data, especially depth images, to perform non - rigid 3D reconstruction and generate 3D models of plants.
3. **Fuse sensor data**: Fuse the collected environmental data (such as temperature, humidity, light) with the 3D model to provide more comprehensive plant growth information.

The main innovation points of the paper are:
- **Non - rigid 3D reconstruction**: This is the first study on 3D reconstruction of non - rigid plants in an uncontrolled environment, which can more realistically reflect the morphological changes of plants under natural conditions.
- **Background removal technology**: A background removal technology based on the logarithmic function is proposed, which can more effectively remove the background from depth images and improve the accuracy of 3D models.
- **Data fusion**: Fuse multiple sensor data with 3D models to provide more abundant information for phenotyping analysis.

Through these technologies, the paper aims to provide more accurate and comprehensive data support for agricultural phenotyping analysis, thereby improving the production efficiency and management precision of crops.