Abstract:Recently, 3D Gaussian splatting has gained attention for its capability to generate high-fidelity rendering results. At the same time, most applications such as games, animation, and AR/VR use mesh-based representations to represent and render 3D scenes. We propose a novel approach that integrates mesh representation with 3D Gaussian splats to perform high-quality rendering of reconstructed real-world scenes. In particular, we introduce a distance-based Gaussian splatting technique to align the Gaussian splats with the mesh surface and remove redundant Gaussian splats that do not contribute to the rendering. We consider the distance between each Gaussian splat and the mesh surface to distinguish between tightly-bound and loosely-bound Gaussian splats. The tightly-bound splats are flattened and aligned well with the mesh geometry. The loosely-bound Gaussian splats are used to account for the artifacts in reconstructed 3D meshes in terms of rendering. We present a training strategy of binding Gaussian splats to the mesh geometry, and take into account both types of splats. In this context, we introduce several regularization techniques aimed at precisely aligning tightly-bound Gaussian splats with the mesh surface during the training process. We validate the effectiveness of our method on large and unbounded scene from mip-NeRF 360 and Deep Blending datasets. Our method surpasses recent mesh-based neural rendering techniques by achieving a 2dB higher PSNR, and outperforms mesh-based Gaussian splatting methods by 1.3 dB PSNR, particularly on the outdoor mip-NeRF 360 dataset, demonstrating better rendering quality. We provide analyses for each type of Gaussian splat and achieve a reduction in the number of Gaussian splats by 30% compared to the original 3D Gaussian splatting.

Hardware accelerated approach for accurate surface splatting

A Uniform Hardware-Accelerated Adaptive EWA Splatting Algorithm

A Hard Accelerated Algorithm for EWA Splatting Based on Modern GPU

A Novel Pointcloud-Based Isosurface Extraction

Hardware-Accelerated Real-time Rendering of Water Surface

Real-time High-Quality Surface Rendering for Large Scale Particle-Based Fluids

Affine Arithmetic-Based B-Spline Surface Intersection with GPU Acceleration

Real-time Ray Casting of Algebraic B-spline Surfaces.

Real-time Rendering of Algebraic B-spline Surfaces Via Bézier Point Insertion

Technical Section: Real-time B-spline Free-Form Deformation via GPU acceleration

Rapid Evaluation of B-spline Surfaces Using GPU

Real-time B-spline Free-Form Deformation via GPU acceleration.

2D Gaussian Splatting for Geometrically Accurate Radiance Fields

A practical GPU accelerated surface reconstruction.

HDGS: Textured 2D Gaussian Splatting for Enhanced Scene Rendering

RaDe-GS: Rasterizing Depth in Gaussian Splatting

Fast DRR splat rendering using common consumer graphics hardware

PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction

SPLAT WITH LOCAL SURFACE RECONSTRUCTION

AdaSplats: Adaptive Splatting of Point Clouds for Accurate 3D Modeling and Real-time High-Fidelity LiDAR Simulation

MeshGS: Adaptive Mesh-Aligned Gaussian Splatting for High-Quality Rendering