Abstract:Understanding the dynamics of generic 3D scenes is fundamentally challenging in computer vision, essential in enhancing applications related to scene reconstruction, motion tracking, and avatar creation. In this work, we address the task as the problem of inferring dense, long-range motion of 3D points. By observing a set of point trajectories, we aim to learn an implicit motion field parameterized by a neural network to predict the movement of novel points within the same domain, without relying on any data-driven or scene-specific priors. To achieve this, our approach builds upon the recently introduced dynamic point field model that learns smooth deformation fields between the canonical frame and individual observation frames. However, temporal consistency between consecutive frames is neglected, and the number of required parameters increases linearly with the sequence length due to per-frame modeling. To address these shortcomings, we exploit the intrinsic regularization provided by SIREN, and modify the input layer to produce a spatiotemporally smooth motion field. Additionally, we analyze the motion field Jacobian matrix, and discover that the motion degrees of freedom (DOFs) in an infinitesimal area around a point and the network hidden variables have different behaviors to affect the model's representational power. This enables us to improve the model representation capability while retaining the model compactness. Furthermore, to reduce the risk of overfitting, we introduce a regularization term based on the assumption of piece-wise motion smoothness. Our experiments assess the model's performance in predicting unseen point trajectories and its application in temporal mesh alignment with guidance. The results demonstrate its superiority and effectiveness. The code and data for the project are publicly available: \url{<a class="link-external link-https" href="https://yz-cnsdqz.github.io/eigenmotion/DOMA/" rel="external noopener nofollow">this https URL</a>}

Modeling Trajectories for 3D Motion Analysis

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

Human motion segmentation using collaborative representations of 3D skeletal sequences.

A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding

3D Human Motion Synthesis Based on Nonlinear Manifold Learning

On Double-Level Kernel Self-Similarities for 3D Motion Trajectory Description and Recognition

3D Skeleton-based Human Motion Prediction with Manifold-Aware GAN

A Study of Relative Motion Point Trajectories for Action Recognition

Describing Local Reference Frames for 3-D Motion Trajectory Recognition

On Multiscale Self-Similarities Description for Effective Three-Dimensional/Six-Dimensional Motion Trajectory Recognition

TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos

MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds

Decoupling Human and Camera Motion from Videos in the Wild

A New Representation of Skeleton Sequences for 3D Action Recognition

Trajectory Space: A Dual Representation for Nonrigid Structure from Motion

Trajectory-Based Modeling Of Human Actions With Motion Reference Points

Deep Trajectory for Recognition of Human Behaviours

3D Kinematics Estimation from Video with a Biomechanical Model and Synthetic Training Data

Degrees of Freedom Matter: Inferring Dynamics from Point Trajectories

Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs