Abstract:Understanding the dynamics of generic 3D scenes is fundamentally challenging in computer vision, essential in enhancing applications related to scene reconstruction, motion tracking, and avatar creation. In this work, we address the task as the problem of inferring dense, long-range motion of 3D points. By observing a set of point trajectories, we aim to learn an implicit motion field parameterized by a neural network to predict the movement of novel points within the same domain, without relying on any data-driven or scene-specific priors. To achieve this, our approach builds upon the recently introduced dynamic point field model that learns smooth deformation fields between the canonical frame and individual observation frames. However, temporal consistency between consecutive frames is neglected, and the number of required parameters increases linearly with the sequence length due to per-frame modeling. To address these shortcomings, we exploit the intrinsic regularization provided by SIREN, and modify the input layer to produce a spatiotemporally smooth motion field. Additionally, we analyze the motion field Jacobian matrix, and discover that the motion degrees of freedom (DOFs) in an infinitesimal area around a point and the network hidden variables have different behaviors to affect the model's representational power. This enables us to improve the model representation capability while retaining the model compactness. Furthermore, to reduce the risk of overfitting, we introduce a regularization term based on the assumption of piece-wise motion smoothness. Our experiments assess the model's performance in predicting unseen point trajectories and its application in temporal mesh alignment with guidance. The results demonstrate its superiority and effectiveness. The code and data for the project are publicly available: \url{<a class="link-external link-https" href="https://yz-cnsdqz.github.io/eigenmotion/DOMA/" rel="external noopener nofollow">this https URL</a>}

Evolving Distributions Under Local Motion

Limit Distribution for Smooth Total Variation and $χ^2$-Divergence in High Dimensions

Distributionally Robust Optimization with Data Geometry

Dynamics of Data-driven Ambiguity Sets for Hyperbolic Conservation Laws with Uncertain Inputs

Degrees of Freedom Matter: Inferring Dynamics from Point Trajectories

Tracking Evolving labels using Cone based Oracles

Theoretically Guaranteed Distribution Adaptable Learning

Semi-Nonparametric Estimation of Distribution Divergence in Non-Euclidean Spaces

Geometric planted matchings beyond the Gaussian model

Self-similar Motion for Modeling Anomalous Diffusion and Nonextensive Statistical Distributions

Large deviations for hyperbolic $k$-nearest neighbor balls

Efficient Evader Detection in Mobile Sensor Networks

On Low Discrepancy Samplings in Product Spaces of Motion Groups

In-and-Out: Algorithmic Diffusion for Sampling Convex Bodies

Geometric constraints improve inference of sparsely observed stochastic dynamics

Strategic Distribution Shift of Interacting Agents via Coupled Gradient Flows

Recovering Best Statistical Guarantees via the Empirical Divergence-based Distributionally Robust Optimization

Locally Stationary Distributions: A Framework for Analyzing Slow-Mixing Markov Chains

Smooth Distance Approximation

Probabilistic variational methods for vision based complex motion analysis

Distribution learning via neural differential equations: a nonparametric statistical perspective