Abstract:As one of the essential deep learning models, a restricted Boltzmann machine (RBM) is a commonly used generative training model. By adaptively growing the size of the hidden units, infinite RBM (IRBM) is obtained, which possesses an excellent property of automatically choosing the hidden layer size depending on a specific task. An IRBM presents a competitive generative capability with the traditional RBM. First, a generative model called Gaussian IRBM (GIRBM) is proposed to deal with practical scenarios from the perspective of data discretization. Subsequently, a discriminative IRBM (DIRBM) and a discriminative GIRBM (DGIRBM) are established to solve classification problems by attaching extra-label units next to the input layer. They are motivated by the fact that a discriminative variant of an RBM can complete an individual framework for classification with better performance than some standard classifiers. Remarkably, the proposed models retain both generative and discriminative properties synchronously, that is, they can reconstruct data effectively and be established in considerable self-contained classifiers. The experimental results on image classification (both large and small), text identification, and facial recognition (both clean and noisy) reflect that a DIRBM and a DGIRBM are superior to some state-of-the-art RBM models in terms of the reconstruction error and the classification accuracy. Intuitively, they require models to avoid utilizing more hidden units than needed when confronted with various sizes of data, prioritizing smaller networks. In addition, the proposed models behave more robustly than other classic classifiers when dealing with noisy facial recognition.

Mining Invariance in Restricted Boltzmann Machine Via Information Geometry

Generative and Discriminative Infinite Restricted Boltzmann Machine Training

Training Restricted Boltzmann Machines with Binary Synapses Using the Bayesian Learning Rule

An Overview on Restricted Boltzmann Machines.

Privacy-Preserving Restricted Boltzmann Machine

Understanding Deep Learning by Revisiting Boltzmann Machines: an Information Geometry Approach

An Overview of Restricted Boltzmann Machines

Thermodynamics of the Ising Model Encoded in Restricted Boltzmann Machines

Restricted Boltzmann Machine with Adaptive Local Hidden Units.

Weight Uncertainty in Boltzmann Machine.

Generalized Boltzmann Machine with Deep Neural Structure.

Properties and Bayesian fitting of restricted Boltzmann machines

Infinite Fuzzy Restricted Boltzmann Machines for the Enhancement of Fuzzy Neural Networks

Restricted Boltzmann Machines: Introduction and Review

Inferring effective couplings with Restricted Boltzmann Machines

Understanding Boltzmann Machine and Deep Learning via A Confident Information First Principle

Matrix Variate RBM Model with Gaussian Distributions.

A New Variant of Restricted Boltzmann Machine with Horizontal Connections

A Novel Restricted Boltzmann Machine Training Algorithm with Fast Gibbs Sampling Policy

Towards the representational power of restricted Boltzmann machines

A Note on Restricted Boltzmann Machines and Variational Autoencoders