Research on automatic 3D reconstruction of plant phenotype based on Multi-View images
Danni Yang,Huijun Yang,Dongfeng Liu,Xianlin Wang
DOI: https://doi.org/10.1016/j.compag.2024.108866
IF: 8.3
2024-05-01
Computers and Electronics in Agriculture
Abstract:Three-dimensional reconstruction plays a crucial role in quantifying crop phenotypes and exploring crop physiological structures. This paper presents a phenotyping platform designed for the 3D reconstruction of complex plants, utilizing multi-view images and introducing a joint evaluation criterion for both the reconstruction algorithm and the platform. Initially, a device composed of Raspberry Pi camera, SSH protocol, USB-TTL, motorized turntable, and shadow booth is built for automated image acquisition and transmission. Then, a dataset containing carex cabbage and kale is created and trained based on U 2 - n e t to achieve precise image segmentation. After that, an improved structure from motion algorithm, named IVOP & AKAZE-SFM, and multi-view stereo algorithm are utilized for the fine-scale reconstruction of plants. Next, by combing color filtering and Euclidean clustering, a denoising algorithm is proposed to obtain clean point clouds of plants. Finally, a method for calibrating the size of the plant point cloud based on priori condition is proposed to solve the problem of point cloud deformation in reconstruction. The evaluation of image segmentation model resulted in a precision of 0.91, a recall of 0.972, an IOU of 0.943, and a max F β of 0.099. The proposed IVOP&AKAZE-SFM is assessed against mainstream algorithms, the results show that our algorithm has the minimum average track length, minimum average reprojection error and generated the most points. The correlation coefficient ( R 2 ) between the extracted traits and measured phenotype, such as plant height and plant width, are 0.999 and 1.000, while the root means squared errors (RMSE) are 0.298 cm and 0.338 cm. Consequently, the platform offers a cost-effective, automated, and integrated solution for fine-scale plant 3D reconstruction.
agriculture, multidisciplinary,computer science, interdisciplinary applications