Abstract:Efficiently digitizing high-fidelity animatable human avatars from videos is a challenging and active research topic. Recent volume rendering-based neural representations open a new way for human digitization with their friendly usability and photo-realistic reconstruction quality. However, they are inefficient for long optimization times and slow inference speed; their implicit nature results in entangled geometry, materials, and dynamics of humans, which are hard to edit afterward. Such drawbacks prevent their direct applicability to downstream applications, especially the prominent rasterization-based graphic ones. We present EMA, a method that Efficiently learns Meshy neural fields to reconstruct animatable human Avatars. It jointly optimizes explicit triangular canonical mesh, spatial-varying material, and motion dynamics, via inverse rendering in an end-to-end fashion. Each above component is derived from separate neural fields, relaxing the requirement of a template, or rigging. The mesh representation is highly compatible with the efficient rasterization-based renderer, thus our method only takes about an hour of training and can render in real-time. Moreover, only minutes of optimization is enough for plausible reconstruction results. The disentanglement of meshes enables direct downstream applications. Extensive experiments illustrate the very competitive performance and significant speed boost against previous methods. We also showcase applications including novel pose synthesis, material editing, and relighting. The project page: <a class="link-external link-https" href="https://xk-huang.github.io/ema/" rel="external noopener nofollow">this https URL</a>.

Mesh Neural Cellular Automata

Mesh Neural Cellular Automata

Multi-Texture Synthesis through Signal Responsive Neural Cellular Automata

Surface‐aware Mesh Texture Synthesis with Pre‐trained 2D CNNs

Exploring Multiple Neighborhood Neural Cellular Automata (MNNCA) for Enhanced Texture Learning

Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

NoiseNCA: Noisy Seed Improves Spatio-Temporal Continuity of Neural Cellular Automata

TM-NET: Deep Generative Networks for Textured Meshes

NeuMesh: Learning Disentangled Neural Mesh-Based Implicit Field for Geometry and Texture Editing.

Emergent Dynamics in Neural Cellular Automata

Mesh2Tex: Generating Mesh Textures from Image Queries

NeRF-Texture: Synthesizing Neural Radiance Field Textures

MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers

Neural Progressive Meshes

Neural Cellular Automata Manifold

Efficient Meshy Neural Fields for Animatable Human Avatars

Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scale

Generating Texture for 3D Human Avatar from a Single Image using Sampling and Refinement Networks

3DTextureTransformer: Geometry Aware Texture Generation for Arbitrary Mesh Topology

Learning Dynamic Textures for Neural Rendering of Human Actors

Make-A-Texture: Fast Shape-Aware Texture Generation in 3 Seconds