Abstract: The Variational Monte Carlo (VMC) is a promising approach for computing the ground state energy of many-body quantum problems and attracts more and more interests due to the development of machine learning. The recent paradigms in VMC construct neural networks as trial wave functions, sample quantum configurations using Markov chain Monte Carlo (MCMC) and train neural networks with stochastic gradient descent (SGD) method. However, the theoretical convergence of VMC is still unknown when SGD interacts with MCMC sampling given a well-designed trial wave function. Since MCMC reduces the difficulty of estimating gradients, it has inevitable bias in practice. Moreover, the local energy may be unbounded, which makes it harder to analyze the error of MCMC sampling. Therefore, we assume that the local energy is sub-exponential and use the Bernstein inequality for non-stationary Markov chains to derive error bounds of the MCMC estimator. Consequently, VMC is proven to have a first order convergence rate $O(\log K/\sqrt{n K})$ with $K$ iterations and a sample size $n$. It partially explains how MCMC influences the behavior of SGD. Furthermore, we verify the so-called correlated negative curvature condition and relate it to the zero-variance phenomena in solving eigenvalue functions. It is shown that VMC escapes from saddle points and reaches $(\epsilon,\epsilon^{1/4})$ -approximate second order stationary points or $\epsilon^{1/2}$-variance points in at least $O(\epsilon^{-11/2}\log^{2}(1/\epsilon) )$ steps with high probability. Our analysis enriches the understanding of how VMC converges efficiently and can be applied to general variational methods in physics and statistics.

Convergence Properties of A General Algorithm for Calculating Variational Bayesian Estimates for A Normal Mixture Model

Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values

Asymptotic Convergence Properties of the EM Algorithm with Respect to the Overlap in the Mixture

Statistical Convergence of the EM Algorithm on Gaussian Mixture Models

A Deterministic Global Optimization Method for Variational Inference

Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection

Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians

Convergence rates for estimating multivariate scale mixtures of uniform densities

Provable Convergence of Variational Monte Carlo Methods

Toward Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixture Models

Comparative Analysis on Convergence Rates of the Em Algorithm and Its Two Modifications for Gaussian Mixtures

On Convergence Properties of the EM Algorithm for Gaussian Mixtures.

Clustering of non-Gaussian data by variational Bayes for normal inverse Gaussian mixture models

Variational Bayes Approximations for Clustering via Mixtures of Normal Inverse Gaussian Distributions

On the Correct Convergence of the EM Algorithm for Gaussian Mixtures

Monte Carlo Approximation of Bayes Factors via Mixing with Surrogate Distributions

On Efficient and Scalable Computation of the Nonparametric Maximum Likelihood Estimator in Mixture Models

Consistency of variational Bayesian inference for non-linear inverse problems of partial differential equations

Convergence of Expectation-Maximization Algorithm With Mixed-Integer Optimization

Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations