Abstract:In the rapidly emerging era of untact ("contact-free") technologies, the requirement for three-dimensional (3D) virtual environments utilized in virtual reality (VR)/augmented reality (AR) and the metaverse has seen significant growth, owing to their extensive application across various domains. Current research focuses on the automatic transfer of the style of rendering images within a 3D virtual environment using artificial intelligence, which aims to minimize human intervention. However, the prevalent studies on rendering-based 3D environment-style transfers have certain inherent limitations. First, the training of a style transfer network dedicated to 3D virtual environments demands considerable style image data. These data must align with viewpoints that closely resemble those of the virtual environment. Second, there was noticeable inconsistency within the 3D structures. Predominant studies often neglect 3D scene geometry information instead of relying solely on 2D input image features. Finally, style adaptation fails to accommodate the unique characteristics inherent in each object. To address these issues, we propose a novel approach: a neural rendering-based 3D scene-style conversion technique. This methodology employs semantic nearest-neighbor feature matching, thereby facilitating the transfer of style within a 3D scene while considering the distinctive characteristics of each object, even when employing a single style image. The neural radiance field enables the network to comprehend the geometric information of a 3D scene in relation to its viewpoint. Subsequently, it transfers style features by employing the unique features of a single style image via semantic nearest-neighbor feature matching. In an empirical context, our proposed semantic 3D scene style transfer method was applied to 3D scene style transfers for both interior and exterior environments. This application utilizes the replica, 3DFront, and Tanks and Temples datasets for testing. The results illustrate that the proposed methodology surpasses existing style transfer techniques in terms of maintaining 3D viewpoint consistency, style uniformity, and semantic coherence.

A Mobile Augmented Reality Application For Simulating Claude Monet’s Impressionistic Art Style

Artistic Augmentation of Photographs with Droplets

Neural Rendering-Based 3D Scene Style Transfer Method via Semantic Understanding Using a Single Style Image

Machine Learning with Synthetic Data – a New Way to Learn and Classify the Pictorial Augmented Reality Markers in Real-Time

Automatic Data Generation for Deep Learning Model Training of Image Classification Used for Augmented Reality on Pre-school Books

Real-time Virtual-Try-On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers

Novel 3D-Aware Composition Images Synthesis for Object Display with Diffusion Model.

Experimental Analysis of Style Transfer and Target Detection in Interactive Art on Smartphones

Stylized Neural Painting

Scaling Painting Style Transfer

PTGCF: Printing Texture Guided Color Fusion for Impressionism Oil Painting Style Rendering

Scene Depth Estimation from Traditional Oriental Landscape Paintings

A Study on Augmented Reality Application in Situational Simulation Learning

GPU-Accelerated Mobile Multi-view Style Transfer

Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translation

ProcessPainter: Learn Painting Process from Sequence Data

The Application of Computer Augmented Reality Technology in Ancient Murals

An AR system for artistic creativity education

Research on Multichannel 3D Oil Painting Style Rendering Model Based on 3DS Max

Bridging the Gap: Studio-like Avatar Creation from a Monocular Phone Capture