Abstract:In the rapidly emerging era of untact ("contact-free") technologies, the requirement for three-dimensional (3D) virtual environments utilized in virtual reality (VR)/augmented reality (AR) and the metaverse has seen significant growth, owing to their extensive application across various domains. Current research focuses on the automatic transfer of the style of rendering images within a 3D virtual environment using artificial intelligence, which aims to minimize human intervention. However, the prevalent studies on rendering-based 3D environment-style transfers have certain inherent limitations. First, the training of a style transfer network dedicated to 3D virtual environments demands considerable style image data. These data must align with viewpoints that closely resemble those of the virtual environment. Second, there was noticeable inconsistency within the 3D structures. Predominant studies often neglect 3D scene geometry information instead of relying solely on 2D input image features. Finally, style adaptation fails to accommodate the unique characteristics inherent in each object. To address these issues, we propose a novel approach: a neural rendering-based 3D scene-style conversion technique. This methodology employs semantic nearest-neighbor feature matching, thereby facilitating the transfer of style within a 3D scene while considering the distinctive characteristics of each object, even when employing a single style image. The neural radiance field enables the network to comprehend the geometric information of a 3D scene in relation to its viewpoint. Subsequently, it transfers style features by employing the unique features of a single style image via semantic nearest-neighbor feature matching. In an empirical context, our proposed semantic 3D scene style transfer method was applied to 3D scene style transfers for both interior and exterior environments. This application utilizes the replica, 3DFront, and Tanks and Temples datasets for testing. The results illustrate that the proposed methodology surpasses existing style transfer techniques in terms of maintaining 3D viewpoint consistency, style uniformity, and semantic coherence.

Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images

TeSTNeRF: Text-Driven 3D Style Transfer Via Cross-Modal Learning.

Correlation-based and Content-Enhanced Network for Video Style Transfer

MM-NeRF: Multimodal-Guided 3D Multi-Style Transfer of Neural Radiance Field

G3DST: Generalizing 3D Style Transfer with Neural Radiance Fields across Scenes and Styles

Artistic Style Transfer with Internal-external Learning and Contrastive Learning

StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning

3D Face Style Transfer with a Hybrid Solution of NeRF and Mesh Rasterization

Evaluate and Improve the Quality of Neural Style Transfer.

Arbitrary 3D stylization of radiance fields

UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene

NeRF-Art: Text-Driven Neural Radiance Fields Stylization

StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields

StyleDyRF: Zero-shot 4D Style Transfer for Dynamic Neural Radiance Fields

Neural Rendering-Based 3D Scene Style Transfer Method via Semantic Understanding Using a Single Style Image

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis

TSNeRF: Text-driven Stylized Neural Radiance Fields Via Semantic Contrastive Learning

Towards Multi-View Consistent Style Transfer with One-Step Diffusion via Vision Conditioning

CLIP3Dstyler: Language Guided 3D Arbitrary Neural Style Transfer

Stylizing Sparse-View 3D Scenes with Hierarchical Neural Representation