Abstract:3D meshes are widely used in computer vision and graphics for their efficiency in animation and minimal memory use, playing a crucial role in movies, games, AR, and VR. However, creating temporally consistent and realistic textures for mesh sequences remains labor-intensive for professional artists. On the other hand, while video diffusion models excel at text-driven video generation, they often lack 3D geometry awareness and struggle with achieving multi-view consistent texturing for 3D meshes. In this work, we present Tex4D, a zero-shot approach that integrates inherent 3D geometry knowledge from mesh sequences with the expressiveness of video diffusion models to produce multi-view and temporally consistent 4D textures. Given an untextured mesh sequence and a text prompt as inputs, our method enhances multi-view consistency by synchronizing the diffusion process across different views through latent aggregation in the UV space. To ensure temporal consistency, we leverage prior knowledge from a conditional video generation model for texture synthesis. However, straightforwardly combining the video diffusion model and the UV texture aggregation leads to blurry results. We analyze the underlying causes and propose a simple yet effective modification to the DDIM sampling process to address this issue. Additionally, we introduce a reference latent texture to strengthen the correlation between frames during the denoising process. To the best of our knowledge, Tex4D is the first method specifically designed for 4D scene texturing. Extensive experiments demonstrate its superiority in producing multi-view and multi-frame consistent videos based on untextured mesh sequences.

Texture Generation on 3D Meshes with Point-UV Diffusion

Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models

UV-free Texture Generation with Denoising and Geodesic Heat Diffusions

TEXGen: a Generative Diffusion Model for Mesh Textures

Generating Textures of Irregular Objects from Models and Photo Sequences

TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and Resampling

Consistent Mesh Diffusion

TexOct: Generating Textures of 3D Models with Octree-based Diffusion

TexPainter: Generative Mesh Texturing with Multi-view Consistency

TUVF: Learning Generalizable Texture UV Radiance Fields

MVTexGen: Synthesising 3D Textures Using Multi-View Diffusion

TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models

Text-Guided Texturing by Synchronized Multi-View Diffusion

UVMap-ID: A Controllable and Personalized UV Map Generative Model

GenesisTex: Adapting Image Denoising Diffusion to Texture Space

GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation

Text-guided High-definition Consistency Texture Model

VCD-Texture: Variance Alignment based 3D-2D Co-Denoising for Text-Guided Texturing

Tex4D: Zero-shot 4D Scene Texturing with Video Diffusion Models

Text2Tex: Text-driven Texture Synthesis via Diffusion Models