Leaf Angle Estimation using Mask R-CNN and LETR Vision Transformer

Venkat Margapuri,Prapti Thapaliya,Trevor Rife
2024-08-02
Abstract:Modern day studies show a high degree of correlation between high yielding crop varieties and plants with upright leaf angles. It is observed that plants with upright leaf angles intercept more light than those without upright leaf angles, leading to a higher rate of photosynthesis. Plant scientists and breeders benefit from tools that can directly measure plant parameters in the field i.e. on-site phenotyping. The estimation of leaf angles by manual means in a field setting is tedious and cumbersome. We mitigate the tedium using a combination of the Mask R-CNN instance segmentation neural network, and Line Segment Transformer (LETR), a vision transformer. The proposed Computer Vision (CV) pipeline is applied on two image datasets, Summer 2015-Ames ULA and Summer 2015- Ames MLA, with a combined total of 1,827 plant images collected in the field using FieldBook, an Android application aimed at on-site phenotyping. The leaf angles estimated by the proposed pipeline on the image datasets are compared to two independent manual measurements using ImageJ, a Java-based image processing program developed at the National Institutes of Health and the Laboratory for Optical and Computational Instrumentation. The results, when compared for similarity using the Cosine Similarity measure, exhibit 0.98 similarity scores on both independent measurements of Summer 2015-Ames ULA and Summer 2015-Ames MLA image datasets, demonstrating the feasibility of the proposed pipeline for on-site measurement of leaf angles.
Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning
What problem does this paper attempt to address?
The paper attempts to address the problem of automatically estimating plant leaf angles in field environments. Traditionally, the measurement of leaf angles relies on manual methods, such as using mechanical inclinometers or image processing software (e.g., ImageJ), which are both time-consuming and prone to human error. To overcome these issues, this paper proposes a computer vision (CV) pipeline that combines the Mask R-CNN instance segmentation neural network and the Line Segment Transformer (LETR) vision transformer to achieve automated leaf angle estimation. Specifically, the study aims to: 1. **Improve measurement efficiency**: Reduce the time and labor costs of manual measurements through automation. 2. **Enhance measurement accuracy**: Minimize human error and improve the consistency and reliability of measurement results. 3. **Suitability for field environments**: Ensure that the method works effectively under actual field conditions, providing a practical tool for plant scientists and breeders. The study utilized two datasets (Summer 2015-Ames ULA and Summer 2015-Ames MLA), comprising a total of 1,827 field-collected plant images, and compared the estimated leaf angles with independent manual measurement results to validate the effectiveness of the proposed method. The results showed that the method achieved cosine similarities of 0.98 and 0.99 on the two datasets, respectively, indicating high accuracy and feasibility.