Abstract:Using parts of existing models to rebuild new models, commonly termed as example-based modeling, is a classical methodology in the realm of computer graphics. Previous works mostly focus on shape composition, making them very hard to use for realistic composition of 3D objects captured from real-world scenes. This leads to combining multiple NeRFs into a single 3D scene to achieve seamless appearance blending. However, the current SeamlessNeRF method struggles to achieve interactive editing and harmonious stitching for real-world scenes due to its gradient-based strategy and grid-based representation. To this end, we present an example-based modeling method that combines multiple Gaussian fields in a point-based representation using sample-guided synthesis. Specifically, as for composition, we create a GUI to segment and transform multiple fields in real time, easily obtaining a semantically meaningful composition of models represented by 3D Gaussian Splatting (3DGS). For texture blending, due to the discrete and irregular nature of 3DGS, straightforwardly applying gradient propagation as SeamlssNeRF is not supported. Thus, a novel sampling-based cloning method is proposed to harmonize the blending while preserving the original rich texture and content. Our workflow consists of three steps: 1) real-time segmentation and transformation of a Gaussian model using a well-tailored GUI, 2) KNN analysis to identify boundary points in the intersecting area between the source and target models, and 3) two-phase optimization of the target model using sampling-based cloning and gradient constraints. Extensive experimental results validate that our approach significantly outperforms previous works in terms of realistic synthesis, demonstrating its practicality. More demos are available at <a class="link-external link-https" href="https://ingra14m.github.io/gs_stitching_website" rel="external noopener nofollow">this https URL</a>.

Texture Mixing by Interpolating Deep Statistics Via Gaussian Models

Synthesizing and Mixing Stationary Gaussian Texture Models.

Optimal transport mixing of gaussian

Static and Dynamic Texture Mixing Using Optimal Transport.

Texture Mixing and Texture Movie Synthesis Using Statistical Learning

GramGAN: Deep 3D Texture Synthesis From 2D Exemplars

Texture Synthesis with Spatial Generative Adversarial Networks

MSMix:An Interpolation-Based Text Data Augmentation Method Manifold Swap Mixup

Texture Representation via Analysis and Synthesis with Generative Adversarial Networks

Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editing

Combining Inconsistent Textures Using Convolutional Neural Networks.

Compositional Neural Textures

The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications

Towards Realistic Example-based Modeling via 3D Gaussian Stitching

Conditional Generative ConvNets for Exemplar-Based Texture Synthesis

Wasserstein Barycenter and Its Application to Texture Mixing

Arbitrary-Scale Texture Generation From Coarse-Grained Control

Neural-g: A Deep Learning Framework for Mixing Density Estimation

Stationary Dynamic Texture Synthesis Using Convolutional Neural Networks

Texture Synthesis And Texture Mixing Using Random Lines

NeRF-Texture: Synthesizing Neural Radiance Field Textures