Abstract:The restricted Boltzmann machine (RBM) is one of the widely used basic models in the field of deep learning. Although many indexes are available for evaluating the advantages of RBM training algorithms, the classification accuracy is the most convincing index that can most effectively reflect its advantages. RBM training algorithms are sampling algorithms essentially based on Gibbs sampling. Studies focused on algorithmic improvements have mainly faced challenges in improving the classification accuracy of the RBM training algorithms. To address the above problem, in this paper, we propose a fast Gibbs sampling (FGS) algorithm to learn the RBM by adding accelerated weights and adjustment coefficient. An important link based on Gibbs sampling theory was established between the update of the network weights and mixing rate of Gibbs sampling chain. The proposed FGS method was used to accelerate the mixing rate of Gibbs sampling chain by adding accelerated weights and adjustment coefficients. To further validate the FGS method, numerous experiments were performed to facilitate comparisons with the classical RBM algorithm. The experiments involved learning the RBM based on standard data. The results showed that the proposed FGS method outperformed the CD, PCD, PT5, PT10, and DGS algorithms, particularly with respect to the handwriting database. The findings of our study suggest the potential applications of FGS to real-world problems and demonstrate that the proposed method can build an improved RBM for classification.

Research on RBM Accelerating Learning Algorithm with Weight Momentum

Accelerated Learning for Restricted Boltzmann Machine with a Novel Momentum Algorithm

Research on RBM Training Algorithm with Dynamic Gibbs Sampling

Research on RBM Networks Training Based on Improved Parallel Tempering Algorithm

A Novel Restricted Boltzmann Machine Training Algorithm with Fast Gibbs Sampling Policy

Hyperparameters Adaptation for Restricted Boltzmann Machines Based on Free Energy

Training Restricted Boltzmann Machine Using Gradient Fixing Based Algorithm

A Precise Method for RBMs Training Using Phased Curricula

MomentumRNN: Integrating Momentum into Recurrent Neural Networks

Stochastic Momentum Method with Double Acceleration for Regularized Empirical Risk Minimization

Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum

A New Adaptive Momentum Algorithm for Split-Complex Recurrent Neural Networks

Improving Deep Neural Networks' Training for Image Classification with Nonlinear Conjugate Gradient-Style Adaptive Momentum.

A Back-Propagation Algorithm with Adaptive Momentum Factor

Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization

When and Why Momentum Accelerates SGD:An Empirical Study

Method to Improve the Performance of Restricted Boltzmann Machines.

When and why momentum accelerates sgd: An empirical study

Random Batch Method with Momentum Correction

Convergence of Gradient Method with Momentum for Back-Propagation Neural Networks

An Asymptotic Analysis of Random Partition Based Minibatch Momentum Methods for Linear Regression Models