Abstract:Accurate estimation of plant skeletal structure (e.g., branching structure) from images is essential for smart agriculture and plant science. Unlike human skeletons with fixed topology, plant skeleton estimation presents a unique challenge, i.e., estimating arbitrary tree graphs from images. While recent graph generation methods successfully infer thin structures from images, it is challenging to constrain the output graph strictly to a tree structure. To this problem, we present TreeFormer, a plant skeleton estimator via tree-constrained graph generation. Our approach combines learning-based graph generation with traditional graph algorithms to impose the constraints during the training loop. Specifically, our method projects an unconstrained graph onto a minimum spanning tree (MST) during the training loop and incorporates this prior knowledge into the gradient descent optimization by suppressing unwanted feature values. Experiments show that our method accurately estimates target plant skeletal structures for multiple domains: Synthetic tree patterns, real botanical roots, and grapevine branches. Our implementations are available at <a class="link-external link-https" href="https://github.com/huntorochi/TreeFormer/" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

This paper attempts to solve the problem of accurately estimating the plant skeleton structure (such as the branching structure) from a single image, which is crucial for smart agriculture and plant science. Unlike the human skeleton with a fixed topological structure, plant skeleton estimation faces a unique challenge, that is, estimating an arbitrary tree - like graph from an image. Although recent graph generation methods can infer elongated structures from images, it is difficult to strictly constrain the output graph to conform to a tree - like structure. ### Specific Problems and Solutions 1. **Problem Description**: - Estimating the plant skeleton structure from a single image. - The plant skeleton structure does not have a fixed topological structure like the human skeleton, so it is necessary to estimate an arbitrary tree - like graph. - Although existing graph generation methods can extract unconstrained graph structures from images, they often violate the required constraints when generating tree - like graphs. 2. **Solutions**: - Proposed TreeFormer, a method for estimating plant skeletons through tree - constrained graph generation. - This method combines learning - based graph generation with traditional graph algorithms (such as the minimum spanning tree MST) and imposes these constraints during the training process. - In each training cycle, the unconstrained graph is projected onto the minimum spanning tree and converted into an MST - based tree - like graph in a differentiable way through the Selective Feature Suppression (SFS) layer. ### Method Overview - **SFS Layer**: The SFS layer effectively simulates the behavior of non - differentiable graph algorithms by modifying intermediate features, thereby introducing constraints in the neural network. - **TreeFormer**: Based on the SFS layer and the state - of - the - art Transformer - based graph generator, TreeFormer is developed to infer tree - like structures from a single plant image. ### Experimental Results - The authors evaluated the effectiveness of TreeFormer on multiple datasets, including synthetic tree - like patterns, real - life plant root and grapevine branch images. - The results show that TreeFormer outperforms existing benchmark methods in terms of accuracy, especially in predicting shape and topological structure. ### Main Contributions 1. Proposed a novel method that closely combines learning - based graph generation with traditional graph algorithms, using the newly proposed SFS layer to modify intermediate features in the network, effectively simulating the behavior of non - differentiable graph algorithms. 2. Developed TreeFormer, which is the first end - to - end method that can directly infer the skeleton structure from a single plant image, which is of great significance in the fields of agriculture and plant science. Through this method, the paper solves the challenges faced by existing graph generation methods in dealing with complex plant structures and provides a more accurate method for plant skeleton estimation.

TreeFormer: Single-view Plant Skeleton Estimation via Tree-constrained Graph Generation

STD-Trees: Spatio-temporal Deformable Trees for Multirotors Kinodynamic Planning

Synthesizing trees by plantons

Automatic Reconstruction of Tree Skeletal Structures from Point Clouds

Treegraph: tree architecture from terrestrial laser scanning point clouds

Skeletonization of Plant Point Cloud Data Using Stochastic Optimization Framework

CherryPicker: Semantic Skeletonization and Topological Reconstruction of Cherry Trees

Sketch-based Tree Modeling Using Markov Random Field

ExploreTree: Interactive Tree Modeling in Semantic Trait Space with Online Intent Learning

Simple Reconstruction of Tree Branches from a Single Range Image.

L1-Tree: A Novel Algorithm for Constructing 3D Tree Models and Estimating Branch Architectural Traits Using Terrestrial Laser Scanning Data

Fast Tree Skeleton Extraction Using Voxel Thinning Based on Tree Point Cloud

L 1 -Tree: A novel algorithm for constructing 3D tree models and estimating branch architectural traits using terrestrial laser scanning data

Statistical Modeling of the 3D Geometry and Topology of Botanical Trees

Image-based Tree Modeling.

Sketch-Based Branch Modeling from Images

Tree Modeling with Real Tree-Parts Examples.

3D Tree Skeletonization from Multiple Images Based on Pyrlk Optical Flow

Towards Robotic Tree Manipulation: Leveraging Graph Representations

Method for modeling plant root morphology based on ball splines skeleton

TreeFormer: a Semi-Supervised Transformer-based Framework for Tree Counting from a Single High Resolution Image