Abstract:Recently, 3D Gaussian splatting has gained attention for its capability to generate high-fidelity rendering results. At the same time, most applications such as games, animation, and AR/VR use mesh-based representations to represent and render 3D scenes. We propose a novel approach that integrates mesh representation with 3D Gaussian splats to perform high-quality rendering of reconstructed real-world scenes. In particular, we introduce a distance-based Gaussian splatting technique to align the Gaussian splats with the mesh surface and remove redundant Gaussian splats that do not contribute to the rendering. We consider the distance between each Gaussian splat and the mesh surface to distinguish between tightly-bound and loosely-bound Gaussian splats. The tightly-bound splats are flattened and aligned well with the mesh geometry. The loosely-bound Gaussian splats are used to account for the artifacts in reconstructed 3D meshes in terms of rendering. We present a training strategy of binding Gaussian splats to the mesh geometry, and take into account both types of splats. In this context, we introduce several regularization techniques aimed at precisely aligning tightly-bound Gaussian splats with the mesh surface during the training process. We validate the effectiveness of our method on large and unbounded scene from mip-NeRF 360 and Deep Blending datasets. Our method surpasses recent mesh-based neural rendering techniques by achieving a 2dB higher PSNR, and outperforms mesh-based Gaussian splatting methods by 1.3 dB PSNR, particularly on the outdoor mip-NeRF 360 dataset, demonstrating better rendering quality. We provide analyses for each type of Gaussian splat and achieve a reduction in the number of Gaussian splats by 30% compared to the original 3D Gaussian splatting.

TetSphere Splatting: Representing High-Quality Geometry with Lagrangian Volumetric Meshes

Tetrahedron Splatting for 3D Generation

MVG-Splatting: Multi-View Guided Gaussian Splatting with Adaptive Quantile-Based Geometric Consistency Densification

2DGH: 2D Gaussian-Hermite Splatting for High-quality Rendering and Better Geometry Reconstruction

A New Split Algorithm for 3D Gaussian Splatting

HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction

Volumetric Data Modeling and Analysis Based on Seven-Directional Box Spline

3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis

Quadratic Gaussian Splatting for Efficient and Detailed Surface Reconstruction

SplatFace: Gaussian Splat Face Reconstruction Leveraging an Optimizable Surface

GeoSplatting: Towards Geometry Guided Gaussian Splatting for Physically-based Inverse Rendering

Large-Scale 3D Terrain Reconstruction Using 3D Gaussian Splatting for Visualization and Simulation

SAGS: Structure-Aware 3D Gaussian Splatting

GS-Octree: Octree-based 3D Gaussian Splatting for Robust Object-level 3D Reconstruction Under Strong Lighting

BillBoard Splatting (BBSplat): Learnable Textured Primitives for Novel View Synthesis

Shape Segmentation by Hierarchical Splat Clustering.

GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering

AdaSplats: Adaptive Splatting of Point Clouds for Accurate 3D Modeling and Real-time High-Fidelity LiDAR Simulation

MeshGS: Adaptive Mesh-Aligned Gaussian Splatting for High-Quality Rendering

FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes

Mini-Splatting2: Building 360 Scenes within Minutes via Aggressive Gaussian Densification