Abstract:As one of the essential deep learning models, a restricted Boltzmann machine (RBM) is a commonly used generative training model. By adaptively growing the size of the hidden units, infinite RBM (IRBM) is obtained, which possesses an excellent property of automatically choosing the hidden layer size depending on a specific task. An IRBM presents a competitive generative capability with the traditional RBM. First, a generative model called Gaussian IRBM (GIRBM) is proposed to deal with practical scenarios from the perspective of data discretization. Subsequently, a discriminative IRBM (DIRBM) and a discriminative GIRBM (DGIRBM) are established to solve classification problems by attaching extra-label units next to the input layer. They are motivated by the fact that a discriminative variant of an RBM can complete an individual framework for classification with better performance than some standard classifiers. Remarkably, the proposed models retain both generative and discriminative properties synchronously, that is, they can reconstruct data effectively and be established in considerable self-contained classifiers. The experimental results on image classification (both large and small), text identification, and facial recognition (both clean and noisy) reflect that a DIRBM and a DGIRBM are superior to some state-of-the-art RBM models in terms of the reconstruction error and the classification accuracy. Intuitively, they require models to avoid utilizing more hidden units than needed when confronted with various sizes of data, prioritizing smaller networks. In addition, the proposed models behave more robustly than other classic classifiers when dealing with noisy facial recognition.

Towards the representational power of restricted Boltzmann machines

On the Representational Power of Restricted Boltzmann Machines for Symmetric Functions and Boolean Functions.

Training Restricted Boltzmann Machines with Binary Synapses Using the Bayesian Learning Rule

Cardinality restricted boltzmann machines

On The Magnitude Of Parameters Of Rbms Being Universal Approximators

Generative and Discriminative Infinite Restricted Boltzmann Machine Training

An Overview of Restricted Boltzmann Machines

Efficient Representation Of Topologically Ordered States With Restricted Boltzmann Machines

Properties and Bayesian fitting of restricted Boltzmann machines

Approximation Properties of Gaussian-binary Restricted Boltzmann Machines and Gaussian-binary Deep Belief Networks

Refinements of Approximation Results of Conditional Restricted Boltzmann Machines

An Overview on Restricted Boltzmann Machines.

Boltzmann machines as two-dimensional tensor networks

Geometry and Expressive Power of Conditional Restricted Boltzmann Machines

Tensor Ring Restricted Boltzmann Machines

Privacy-Preserving Restricted Boltzmann Machine

Restricted Boltzmann Machines: Introduction and Review

Temperature based Restricted Boltzmann Machines

Generalized Boltzmann Machine with Deep Neural Structure.

Scalar field Restricted Boltzmann Machine as an ultraviolet regulator

An Efficient Algorithmic Way to Construct Boltzmann Machine Representations for Arbitrary Stabilizer Code