Abstract:We introduce Tree D-fusion, featuring the first collection of 600,000 environmentally aware, 3D simulation-ready tree models generated through Diffusion priors. Each reconstructed 3D tree model corresponds to an image from Google's Auto Arborist Dataset, comprising street view images and associated genus labels of trees across North America. Our method distills the scores of two tree-adapted diffusion models by utilizing text prompts to specify a tree genus, thus facilitating shape reconstruction. This process involves reconstructing a 3D tree envelope filled with point markers, which are subsequently utilized to estimate the tree's branching structure using the space colonization algorithm conditioned on a specified genus.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: Reconstruct realistic 3D tree models from a single image for simulation and ecological research. Specifically, the paper proposes a method named Tree - D Fusion. This method uses diffusion models as prior knowledge and combines tree - species - specific information to generate high - quality 3D tree models from a single image in Google Street View. These models are not only visually realistic but also able to simulate the growth process of trees, including the details of branch structures and leaves, thus providing support for large - scale ecological research. ### Main Challenges 1. **Complex Tree Shapes**: Trees have complex branching structures and self - occlusion phenomena, which make it very difficult to reconstruct a complete 3D model from a single image. 2. **Lack of Data**: Existing 3D tree model data sets are scarce and are usually unstructured point clouds or images, which cannot meet the needs of large - scale simulation. 3. **Species Diversity**: The morphological characteristics of different tree species vary greatly, and tree - species - specific information needs to be considered to generate more realistic models. ### Solutions - **Diffusion Models**: Use diffusion models as prior knowledge and specify tree species through text prompts to generate tree - species - specific 3D models. - **Spatial Colonization Algorithm**: Utilize the spatial colonization algorithm to generate detailed branching structures according to tree - species - specific parameters (such as branching angles, internode lengths, etc.). - **Large - Scale Data Set**: Construct a data set containing 600,000 3D tree models. These models are generated based on images in Google Street View and cover 23 cities in North America and more than 300 tree species. ### Application Scenarios - **Ecological Research**: It can be used to estimate the timber volume and carbon sequestration amount of trees and simulate the shadow coverage under different conditions. - **Urban Planning**: Support urban greening design and management, such as predicting the urban heat island effect. - **Augmented Reality**: Integrate into maps and AR applications to provide realistic 3D tree visualizations. Through these methods, Tree - D Fusion aims to provide a large - scale, high - precision 3D tree model data set to promote research and applications in the fields of forestry and computer vision.

Tree-D Fusion: Simulation-Ready Tree Dataset from Single Images with Diffusion Priors

TreeNet3D : A large scale tree benchmark for 3D tree modeling, carbon storage estimation and tree segmentation

SVDTree: Semantic Voxel Diffusion for Single Image Tree Reconstruction

Tree Counting by Bridging 3D Point Clouds with Imagery

LT3SD: Latent Trees for 3D Scene Diffusion

Tree Modeling with Real Tree-Parts Examples.

Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior

LiDAR-Forest Dataset: LiDAR Point Cloud Simulation Dataset for Forestry Application

Automatic Reconstruction of Tree Skeletal Structures from Point Clouds

Image-based Tree Modeling.

TreeFormer: a Semi-Supervised Transformer-based Framework for Tree Counting from a Single High Resolution Image

From Google Maps to a Fine-Grained Catalog of Street trees

Tree Detection and Diameter Estimation Based on Deep Learning

PCTreeS: 3D Point Cloud Tree Species Classification Using Airborne LiDAR Images

AdaTreeFormer: Few Shot Domain Adaptation for Tree Counting from a Single High-Resolution Image

Generating Images with 3D Annotations Using Diffusion Models

OctFusion: Octree-based Diffusion Models for 3D Shape Generation

The Shape Space of 3D Botanical Tree Models.

Using Synthetic Tree Data in Deep Learning-Based Tree Segmentation Using LiDAR Point Clouds

Transformer for Tree Counting in Aerial Images

TreeFormer: Single-view Plant Skeleton Estimation via Tree-constrained Graph Generation