Abstract:In this work, we propose a first-order sampling method called the Metropolis-adjusted Preconditioned Langevin Algorithm for approximate sampling from a target distribution whose support is a proper convex subset of $\mathbb{R}^{d}$. Our proposed method is the result of applying a Metropolis-Hastings filter to the Markov chain formed by a single step of the preconditioned Langevin algorithm with a metric $\mathscr{G}$, and is motivated by the natural gradient descent algorithm for optimisation. We derive non-asymptotic upper bounds for the mixing time of this method for sampling from target distributions whose potentials are bounded relative to $\mathscr{G}$, and for exponential distributions restricted to the support. Our analysis suggests that if $\mathscr{G}$ satisfies stronger notions of self-concordance introduced in Kook and Vempala (2024), then these mixing time upper bounds have a strictly better dependence on the dimension than when is merely self-concordant. We also provide numerical experiments that demonstrates the practicality of our proposed method. Our method is a high-accuracy sampler due to the polylogarithmic dependence on the error tolerance in our mixing time upper bounds.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: sampling efficiently and accurately from a constrained space. Specifically, the author proposes a first - order sampling method named Metropolis - adjusted Preconditioned Langevin Algorithm (MAPLA) for approximately sampling from the target distribution Π, where the support set of Π is an appropriate convex subset of \( \mathbb{R}^d \).
### Problem Background
In many statistical estimation and inference tasks, it is necessary to draw samples from a distribution, such as estimating functionals, generating confidence intervals for point estimates, and conducting structured exploration of the state space. However, due to the complexity of the problem or the influence of modeling choices, it is usually infeasible to sample exactly from these distributions directly. Therefore, Markov chain Monte Carlo (MCMC) algorithms have been widely used to address this challenge in the past few decades, especially in high - dimensional settings.
For some practical problems, the support set of the target distribution may be a proper subset of \( \mathbb{R}^d \), the so - called **constrained distribution**. Such problems occur in areas such as Bayesian modeling, regularized regression, metabolic network modeling, and differential privacy.
### Main Contributions of the Paper
1. **Proposing a New Method**: The paper proposes a new MCMC method - Metropolis - adjusted Preconditioned Langevin Algorithm (MAPLA), which achieves efficient sampling in the constrained space by applying the Metropolis - Hastings filter on the basis of the preconditioned Langevin algorithm.
2. **Theoretical Analysis**: The author derives non - asymptotic upper bounds on the mixing time of MAPLA, especially when the potential function of the target distribution is bounded with respect to the metric G, or when the target distribution is an exponential distribution and is restricted to the support set. Moreover, if the metric G satisfies a stronger self - concordance condition (Kook and Vempala, 2024), these upper bounds perform better in terms of dimensional dependence.
3. **Numerical Experiments**: The paper also provides numerical experiments to demonstrate the practical feasibility of the proposed method.
### Working Principle of MAPLA
The core idea of MAPLA is to combine the natural gradient descent method and iterate through the following steps:
1. **Generate a Proposal**: Starting from the current iteration point X, use the preconditioned Langevin algorithm with metric G to generate a proposal Z.
2. **Check Whether the Proposal is within the Support Set**: If Z is not within the support set K, reject Z and keep the current iteration point unchanged.
3. **Calculate the Acceptance Probability**: If Z is within the support set K, calculate the Metropolis - Hastings acceptance probability.
4. **Decide to Accept or Reject**: Decide whether to accept the proposal Z according to the acceptance probability.
### Conclusion
In general, this paper aims to solve the problem of sampling efficiently and accurately from a constrained space, and by introducing a new MCMC method - MAPLA, it provides a theoretically rigorous and practically feasible solution.