Abstract:We propose a method to deform static multi‐view volumetric models, such as NeRF, in real‐time using blendshape or physics‐driven animation. This allows us to create dynamic scenes from static captures in an interpretable, artistically controllable way. Top: a face tracker controls an avatar. Bottom: object shattering controlled by physics‐based simulation. The recent increase in popularity of volumetric representations for scene reconstruction and novel view synthesis has put renewed focus on animating volumetric content at high visual quality and in real‐time. While implicit deformation methods based on learned functions can produce impressive results, they are 'black boxes' to artists and content creators, they require large amounts of training data to generalize meaningfully, and they do not produce realistic extrapolations outside of this data. In this work, we solve these issues by introducing a volume deformation method which is real‐time even for complex deformations, easy to edit with off‐the‐shelf software and can extrapolate convincingly. To demonstrate the versatility of our method, we apply it in two scenarios: physics‐based object deformation and telepresence where avatars are controlled using blendshapes. We also perform thorough experiments showing that our method compares favourably to both volumetric approaches combined with implicit deformation and methods based on mesh deformation.

Velocity Skinning for Real-time Stylized Skeletal Animation

EasySkinning: Target-oriented Skinning by Mesh Contraction and Curve Editing

Example-based Dynamic Skinning in Real Time

Real-time Skeletonization for Sketch-Based Modeling

Animating Scanned Human Models.

Automatic skinning and animation of skeletal models

Data-Driven Motion Exaggeration

Fast Complementary Dynamics via Skinning Eigenmodes

A Deep Emulator for Secondary Motion of 3D Characters

Automatic Skinning and Weight Retargeting of Articulated Characters Using Extended Position-Based Dynamics

VolTeMorph: Real‐time, Controllable and Generalizable Animation of Volumetric Representations

Sketch-based skeleton-driven 2D animation and motion capture

Efficient and Realistic Character Animation Through Analytical Physics-Based Skin Deformation

A Design for 3D Animation Based on Real-Time Data Collections

Real-time Deformation with Coupled Cages and Skeletons

Physically Based Real-Time Human Figure Animation

Fast and Physical Plausible Elastic-Deformation Driven by Skeleton

Automatic Skinning using the Mixed Finite Element Method

Skeleton driven animation based on implicit skinning.

Design and Implementation of Skeletal Animation for Marine Life System Based on VTF