Abstract:Over the last few decades, image-based building surface reconstruction has garnered substantial research interest and has been applied across various fields, such as heritage preservation, architectural planning, etc. Compared to the traditional photogrammetric and NeRF-based solutions, recently, Gaussian fields-based methods have exhibited significant potential in generating surface meshes due to their time-efficient training and detailed 3D information preservation. However, most gaussian fields-based methods are trained with all image pixels, encompassing building and nonbuilding areas, which results in a significant noise for building meshes and degeneration in time efficiency. This paper proposes a novel framework, Masked Gaussian Fields (MGFs), designed to generate accurate surface reconstruction for building in a time-efficient way. The framework first applies EfficientSAM and COLMAP to generate multi-level masks of building and the corresponding masked point clouds. Subsequently, the masked gaussian fields are trained by integrating two innovative losses: a multi-level perceptual masked loss focused on constructing building regions and a boundary loss aimed at enhancing the details of the boundaries between different masks. Finally, we improve the tetrahedral surface mesh extraction method based on the masked gaussian spheres. Comprehensive experiments on UAV images demonstrate that, compared to the traditional method and several NeRF-based and Gaussian-based SOTA solutions, our approach significantly improves both the accuracy and efficiency of building surface reconstruction. Notably, as a byproduct, there is an additional gain in the novel view synthesis of building.

Spatial Gaussian Markov Random Fields: Modelling, Applications and Efficient Computations

Gaussian Markov random field spatial models in GAMLSS

Nonstationary Spatial Gaussian Markov Random Fields

On Joint Estimation of Gaussian Graphical Models for Spatial and Temporal Data

Spatial Contextual Classification of Remote Sensing Images Using a Gaussian Process

Efficient Inference of Spatially-varying Gaussian Markov Random Fields with Applications in Gene Regulatory Networks

A multivariate Gaussian random field prior against spatial confounding

Markov Random Field in Visual Information Processing

Nearest-Neighbor Mixture Models for Non-Gaussian Spatial Processes

Highly Multivariate High-dimensionality Spatial Stochastic Processes-A Mixed Conditional Approach

Markov chain random fields, spatial Bayesian networks, and optimal neighborhoods for simulation of categorical fields

Maximum a Posteriori Estimation for Markov Chains Based on Gaussian Markov Random Fields

A Truly Spatial Random Forests Algorithm for Geoscience Data Analysis and Modelling

Simultaneous Inference for Non-Stationary Random Fields, with Application to Gridded Data Analysis

Flexible Basis Representations for Modeling Large Non-Gaussian Spatial Data

A Variational Approach for Modeling High-dimensional Spatial Generalized Linear Mixed Models

Gaussian predictive process models for large spatial data sets

Deep Gaussian Markov Random Fields for Graph-Structured Dynamical Systems

Classification of High Spatial Resolution Imagery Using Improved Gaussian Markov Random-Field-Based Texture Features

Image Segmentation Based on Spatially Coherent Gaussian Mixture Model

MGFs: Masked Gaussian Fields for Meshing Building based on Multi-View Images