Abstract:In this paper, we propose new Metropolis-Hastings and simulated annealing algorithms on finite state space via modifying the energy landscape. The core idea of landscape modification rests on introducing a parameter $c$, in which the landscape is modified once the algorithm is above this threshold parameter to encourage exploration, while the original landscape is utilized when the algorithm is below the threshold for exploitation purpose. We illustrate the power and benefits of landscape modification by investigating its effect on the classical Curie-Weiss model with Glauber dynamics and external magnetic field in the subcritical regime. This leads to a landscape-modified mean-field equation, and with appropriate choice of $c$ the free energy landscape can be transformed from a double-well into a single-well, while the location of the global minimum is preserved on the modified landscape. Consequently, running algorithms on the modified landscape can improve the convergence to the ground-state in the Curie-Weiss model. In the setting of simulated annealing, we demonstrate that landscape modification can yield improved or even subexponential mean tunneling time between global minima in the low-temperature regime by appropriate choice of $c$, and give convergence guarantee using an improved logarithmic cooling schedule with reduced critical height. We also discuss connections between landscape modification and other acceleration techniques such as Catoni's energy transformation algorithm, preconditioning, importance sampling and quantum annealing. The technique developed in this paper is not only limited to simulated annealing and is broadly applicable to any difference-based discrete optimization algorithm by a change of landscape.

State Space Emulation and Annealed Sequential Monte Carlo for High Dimensional Optimization

Improving sample efficiency of high dimensional Bayesian optimization with MCMC

A Bayesian Optimization through Sequential Monte Carlo and Statistical Physics-Inspired Techniques

Optimization by Simulated Annealing

An approximate Itô-SDE based simulated annealing algorithm for multivariate design optimization problems

Ensemble annealing of complex physical systems

A Simulated Annealing Algorithm for Multiobjective Optimizations of Electromagnetic Devices

Hybrid Simulated Annealing Algorithm Based on the Parallel Strategy

A parallel evolutionary multiple-try metropolis Markov chain Monte Carlo algorithm for sampling spatial partitions

Optimal schedules for annealing algorithms

Sequential Monte Carlo with Highly Informative Observations

Adaptive Partitioning Strategy for High-Dimensional Discrete Simulation-based Optimization Problems

Optimised Annealed Sequential Monte Carlo Samplers

Simulated Ising Annealing Algorithm with Gaussian Augmented Hamiltonian Monte Carlo

Improved Metropolis-Hastings algorithms via landscape modifcation with applications to simulated annealing and the Curie-Weiss model

An adaptive simulated annealing algorithm

Simulated Annealing with the State Generator Based on Cauchy and Gaussian Distributions

Bayesian Annealed Sequential Importance Sampling: An Unbiased Version of Transitional Markov Chain Monte Carlo

SEQUENTIAL MONTE CARLO METHODS AND THEIR APPLICATIONS

Improving quantum annealing by engineering the coupling to the environment