Abstract:Restricted Boltzmann machines (RBMs) and their variants are usually trained by contrastive divergence (CD) learning, but the training procedure is an unsupervised learning approach, without any guidances of the background knowledge. To enhance the expression ability of traditional RBMs, in this paper, we propose pairwise constraints (PCs) RBM with Gaussian visible units (pcGRBM) model, in which the learning procedure is guided by PCs and the process of encoding is conducted under these guidances. The PCs are encoded in hidden layer features of pcGRBM. Then, some pairwise hidden features of pcGRBM flock together and another part of them are separated by the guidances. In order to deal with real-valued data, the binary visible units are replaced by linear units with Gaussian noise in the pcGRBM model. In the learning process of pcGRBM, the PCs are iterated transitions between visible and hidden units during CD learning procedure. Then, the proposed model is inferred by approximative gradient descent method and the corresponding learning algorithm is designed. In order to compare the availability of pcGRBM and traditional RBMs with Gaussian visible units, the features of the pcGRBM and RBMs hidden layer are used as input "data" for K -means, spectral clustering (SP) and affinity propagation (AP) algorithms, respectively. We also use tenfold cross-validation strategy to train and test pcGRBM model to obtain more meaningful results with PCs which are derived from incremental sampling procedures. A thorough experimental evaluation is performed with 12 image datasets of Microsoft Research Asia Multimedia. The experimental results show that the clustering performance of K -means, SP, and AP algorithms based on pcGRBM model are significantly better than traditional RBMs. In addition, the pcGRBM model for clustering tasks shows better performance than some semi-supervised clustering algorithms.

A Gaussian RBM with Binary Auxiliary Units

Generative and Discriminative Infinite Restricted Boltzmann Machine Training

Gaussian Cardinality Restricted Boltzmann Machines.

Training Restricted Boltzmann Machines with Binary Synapses Using the Bayesian Learning Rule

Matrix Variate RBM Model with Gaussian Distributions.

Restricted Boltzmann Machines With Gaussian Visible Units Guided by Pairwise Constraints

Restricted Boltzmann Machine with Adaptive Local Hidden Units.

A Precise Method for RBMs Training Using Phased Curricula

A Novel Restricted Boltzmann Machine Training Algorithm with Fast Gibbs Sampling Policy

From Class-Specific to Class-Mixture: Cascaded Feature Representations Via Restricted Boltzmann Machine Learning.

A GA-based RBF classifier with class-dependent features

Gaussian-Bernoulli Based Convolutional Restricted Boltzmann Machine for Images Feature Extraction.

Matrix Variate RBM and Its Applications.

Approximation Properties of Gaussian-binary Restricted Boltzmann Machines and Gaussian-binary Deep Belief Networks

Monogenic Binary Pattern (MBP): A Novel Feature Extraction and Representation Model for Face Recognition

Generalized Boltzmann Machine with Deep Neural Structure.

A Ga-Based Novel Rbf Classifier With Class-Dependent Features

Boltzmann Machines with Clusters of Stochastic Binary Units

Bayesian Regularized Gaussian Mixture Regression with Application to Soft Sensor Modeling for Multi-Mode Industrial Processes

Annealing Gaussian into ReLU: a New Sampling Strategy for Leaky-ReLU RBM