Abstract:3D scene stylization aims at generating stylized images of the scene from arbitrary novel views following a given set of style examples, while ensuring consistency when rendered from different views. Directly applying methods for image or video stylization to 3D scenes cannot achieve such consistency. Thanks to recently proposed neural radiance fields (NeRF), we are able to represent a 3D scene in a consistent way. Consistent 3D scene stylization can be effectively achieved by stylizing the corresponding NeRF. However, there is a significant domain gap between style examples which are 2D images and NeRF which is an implicit volumetric representation. To address this problem, we propose a novel mutual learning framework for 3D scene stylization that combines a 2D image stylization network and NeRF to fuse the stylization ability of 2D stylization network with the 3D consistency of NeRF. We first pre-train a standard NeRF of the 3D scene to be stylized and replace its color prediction module with a style network to obtain a stylized NeRF. It is followed by distilling the prior knowledge of spatial consistency from NeRF to the 2D stylization network through an introduced consistency loss. We also introduce a mimic loss to supervise the mutual learning of the NeRF style module and fine-tune the 2D stylization decoder. In order to further make our model handle ambiguities of 2D stylization results, we introduce learnable latent codes that obey the probability distributions conditioned on the style. They are attached to training samples as conditional inputs to better learn the style module in our novel stylized NeRF. Experimental results demonstrate that our method is superior to existing approaches in both visual quality and long-range consistency.

StyleDyRF: Zero-shot 4D Style Transfer for Dynamic Neural Radiance Fields

StyleDyRF: Zero-shot 4D Style Transfer for Dynamic Neural Radiance Fields

StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields

TeSTNeRF: Text-Driven 3D Style Transfer Via Cross-Modal Learning.

Learning Structure-Aware Transformations for Arbitrary Image Style Transfer

Correlation-based and Content-Enhanced Network for Video Style Transfer

Style Permutation for Diversified Arbitrary Style Transfer

Optimal Transport of Deep Feature for Image Style Transfer

4DStyleGaussian: Zero-shot 4D Style Transfer with Gaussian Splatting

UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene

G3DST: Generalizing 3D Style Transfer with Neural Radiance Fields across Scenes and Styles

3D Face Style Transfer with a Hybrid Solution of NeRF and Mesh Rasterization

Arbitrary 3D stylization of radiance fields

Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images

StyleRF-VolVis: Style Transfer of Neural Radiance Fields for Expressive Volume Visualization

ConRF: Zero-shot Stylization of 3D Scenes with Conditioned Radiation Fields

FPRF: Feed-Forward Photorealistic Style Transfer of Large-Scale 3D Neural Radiance Fields

MM-NeRF: Multimodal-Guided 3D Multi-Style Transfer of Neural Radiance Field

Towards 4D Human Video Stylization

Locally Stylized Neural Radiance Fields

StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning