Abstract:Understanding the dynamics of a system is important in many scientific and engineering domains. This problem can be approached by learning state transition rules from observations using machine learning techniques. Such observed time-series data often consist of sequences of many continuous variables with noise and ambiguity, but we often need rules of dynamics that can be modeled with a few essential variables. In this work, we propose a method for extracting a small number of essential hidden variables from high-dimensional time-series data and for learning state transition rules between these hidden variables. The proposed method is based on the Restricted Boltzmann Machine (RBM), which treats observable data in the visible layer and latent features in the hidden layer. However, real-world data, such as video and audio, include both discrete and continuous variables, and these variables have temporal relationships. Therefore, we propose Recurrent Temporal GaussianBernoulli Restricted Boltzmann Machine (RTGB-RBM), which combines Gaussian-Bernoulli Restricted Boltzmann Machine (GB-RBM) to handle continuous visible variables, and Recurrent Temporal Restricted Boltzmann Machine (RT-RBM) to capture time dependence between discrete hidden variables. We also propose a rule-based method that extracts essential information as hidden variables and represents state transition rules in interpretable form. We conduct experiments on Bouncing Ball and Moving MNIST datasets to evaluate our proposed method. Experimental results show that our method can learn the dynamics of those physical systems as state transition rules between hidden variables and can predict unobserved future states from observed state transitions.

Subspace Restricted Boltzmann Machine

Cardinality restricted boltzmann machines

Sparse Group Restricted Boltzmann Machines

An Overview of Restricted Boltzmann Machines

Gaussian Cardinality Restricted Boltzmann Machines.

Training Restricted Boltzmann Machines with Binary Synapses Using the Bayesian Learning Rule

Boltzmann Machines with Clusters of Stochastic Binary Units

Generative and Discriminative Infinite Restricted Boltzmann Machine Training

A New Sparse Restricted Boltzmann Machine

Restricted Boltzmann Machine with Adaptive Local Hidden Units.

Properties and Bayesian fitting of restricted Boltzmann machines

Privacy-Preserving Restricted Boltzmann Machine

Learning Discriminative Representation with Signed Laplacian Restricted Boltzmann Machine

Learning State Transition Rules from Hidden Layers of Restricted Boltzmann Machines

An Overview on Restricted Boltzmann Machines.

Restricted Boltzmann Machines: Introduction and Review

Matrix Product Operator Restricted Boltzmann Machines

Sequence Classification Restricted Boltzmann Machines With Gated Units

Adversarial Training Methods for Boltzmann Machines

Structural Restricted Boltzmann Machine for image denoising and classification