Abstract:Natural scenes contain a wide range of textured motion phenomena which are characterized by the movement of a large amount of particle and wave elements, such as falling snow, wavy water, and dancing grass. In this paper, we present a generative model for representing these motion patterns and study a Markov chain Monte Carlo algorithm for inferring the generative representation from observed video sequences. Our generative model consists of three components. The first is a photometric model which represents an image as a linear superposition of image bases selected from a generic and overcomplete dictionary. The dictionary contains Gabor and LoG bases for point/particle elements and Fourier bases for wave elements. These bases compete to explain the input images and transfer them to a token (base) representation with an O(10(2))-fold dimension reduction. The second component is a geometric model which groups spatially adjacent tokens (bases) and their motion trajectories into a number of moving elements--called "motons." A moton is a deformable template in time-space representing a moving element, such as a falling snowflake or a flying bird. The third component is a dynamic model which characterizes the motion of particles, waves, and their interactions. For example, the motion of particle objects floating in a river, such as leaves and balls, should be coupled with the motion of waves. The trajectories of these moving elements are represented by coupled Markov chains. The dynamic model also includes probabilistic representations for the birth/death (source/sink) of the motons. We adopt a stochastic gradient algorithm for learning and inference. Given an input video sequence, the algorithm iterates two steps: 1) computing the motons and their trajectories by a number of reversible Markov chain jumps, and 2) learning the parameters that govern the geometric deformations and motion dynamics. Novel video sequences are synthesized from the learned models and, by editing the model parameters, we demonstrate the controllability of the generative model.

A Multi-scale Generative Model for Animate Shapes and Parts.

Make-A-Shape: a Ten-Million-scale 3D Shape Model

Diverse Part Synthesis for 3D Shape Creation

Multi-chart Generative Surface Modeling

Multi-Resolution Generative Modeling of Human Motion from Limited Data

Title Multiscale Generative Model of Human Faces Permalink

Modeling Textured Motion : Particle, Wave and Sketch.

Analysis and synthesis of textured motion: particles and waves.

Analysis and Synthesis of Textured Motion: Particle, Wave and Cartoon Sketch

Progressive Multimodal Shape Generation via Contextual Part Reasoning

Modeling Complex Motion by Tracking and Editing Hidden Markov Graphs.

Learning to Generate 3D Shapes from a Single Example

Active basis for modeling, learning and recognizing deformable templates

Discriminative and Generative Models for Anatomical Shape Analysison Point Clouds with Deep Neural Networks

Learning Part Generation and Assembly for Structure-aware Shape Synthesis

B\'ezierGAN: Automatic Generation of Smooth Curves from Interpretable Low-Dimensional Parameters

Self-Organizing a Latent Hierarchy of Sketch Patterns for Controllable Sketch Synthesis.

A two-level generative model for cloth representation and shape from shading.

Adaptive Multi-Scale Generative Models for Complex Data Synthesis

Generative modeling of biological shapes and images using a probabilistic -shape sampler

Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis