Abstract:Existing photorealistic relightable hand models require extensive identity-specific observations in different views, poses, and illuminations, and face challenges in generalizing to natural illuminations and novel identities. To bridge this gap, we present URHand, the first universal relightable hand model that generalizes across viewpoints, poses, illuminations, and identities. Our model allows few-shot personalization using images captured with a mobile phone, and is ready to be photorealistically rendered under novel illuminations. To simplify the personalization process while retaining photorealism, we build a powerful universal relightable prior based on neural relighting from multi-view images of hands captured in a light stage with hundreds of identities. The key challenge is scaling the cross-identity training while maintaining personalized fidelity and sharp details without compromising generalization under natural illuminations. To this end, we propose a spatially varying linear lighting model as the neural renderer that takes physics-inspired shading as input feature. By removing non-linear activations and bias, our specifically designed lighting model explicitly keeps the linearity of light transport. This enables single-stage training from light-stage data while generalizing to real-time rendering under arbitrary continuous illuminations across diverse identities. In addition, we introduce the joint learning of a physically based model and our neural relighting model, which further improves fidelity and generalization. Extensive experiments show that our approach achieves superior performance over existing methods in terms of both quality and generalizability. We also demonstrate quick personalization of URHand from a short phone scan of an unseen identity.

ARM: Appearance Reconstruction Model for Relightable 3D Generation

High-Fidelity 3D Model Generation with Relightable Appearance from Single Freehand Sketches and Text Guidance

RGM: Reconstructing High-fidelity 3D Car Assets with Relightable 3D-GS Generative Model from a Single Image

Robust Geometry and Reflectance Disentanglement for 3D Face Reconstruction from Sparse-view Images

Relightable Detailed Human Reconstruction from Sparse Flashlight Images

RelitLRM: Generative Relightable Radiance for Large Reconstruction Models

URHand: Universal Relightable Hands

GTR: Improving Large 3D Reconstruction Models through Geometry and Texture Refinement

Self-supervised Re-renderable Facial Albedo Reconstruction from Single Image

ARCH++: Animation-Ready Clothed Human Reconstruction Revisited

High-Fidelity Facial Albedo Estimation via Texture Quantization

Relightify: Relightable 3D Faces from a Single Image via Diffusion Models

REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment

Real-Time Globally Consistent Dense 3D Reconstruction with Online Texturing.

Deep Appearance Models for Face Rendering

R2Human: Real-Time 3D Human Appearance Rendering from a Single Image

UV-IDM: Identity-Conditioned Latent Diffusion Model for Face UV-Texture Generation

MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shading

DeepShaRM: Multi-View Shape and Reflectance Map Recovery Under Unknown Lighting

MeshLRM: Large Reconstruction Model for High-Quality Mesh