Abstract:Numerous applications in biology, statistics, science, and engineering require generating samples from high-dimensional probability distributions. In recent years, the Hamiltonian Monte Carlo (HMC) method has emerged as a state-of-the-art Markov chain Monte Carlo technique, exploiting the shape of such high-dimensional target distributions to efficiently generate samples. Despite its impressive empirical success and increasing popularity, its wide-scale adoption remains limited due to the high computational cost of gradient calculation. Moreover, applying this method is impossible when the gradient of the posterior cannot be computed (for example, with black-box simulators). To overcome these challenges, we propose a novel two-stage Hamiltonian Monte Carlo algorithm with a surrogate model. In this multi-fidelity algorithm, the acceptance probability is computed in the first stage via a standard HMC proposal using an inexpensive differentiable surrogate model, and if the proposal is accepted, the posterior is evaluated in the second stage using the high-fidelity (HF) numerical solver. Splitting the standard HMC algorithm into these two stages allows for approximating the gradient of the posterior efficiently, while producing accurate posterior samples by using HF numerical solvers in the second stage. We demonstrate the effectiveness of this algorithm for a range of problems, including linear and nonlinear Bayesian inverse problems with in-silico data and experimental data. The proposed algorithm is shown to seamlessly integrate with various low-fidelity and HF models, priors, and datasets. Remarkably, our proposed method outperforms the traditional HMC algorithm in both computational and statistical efficiency by several orders of magnitude, all while retaining or improving the accuracy in computed posterior statistics.

Bayesian Estimation of Simultaneous Regression Quantiles Using Hamiltonian Monte Carlo

Distributed Bootstrap Simultaneous Inference for High-Dimensional Quantile Regression

Quantile Regression Neural Networks: A Bayesian Approach

Flexible Bayesian quantile regression for nonlinear mixed effects models based on the generalized asymmetric Laplace distribution

Quasi-Newton Hamiltonian Monte Carlo.

Bayesian estimation for longitudinal data in a joint model with HPCs

Quantum Metropolis-Hastings algorithm with the target distribution calculated by quantum Monte Carlo integration

Multi-fidelity Hamiltonian Monte Carlo

A novel Bayesian method for variable selection and estimation in binary quantile regression

Bayesian Quantile Estimation and Regression with Martingale Posteriors

A Bayesian Approach to Multiple-Output Quantile Regression

Quantum Dynamical Hamiltonian Monte Carlo

Antithetic Riemannian Manifold And Quantum-Inspired Hamiltonian Monte Carlo

Horseshoe Prior Bayesian Quantile Regression

Bayesian Quantile Regression for Ordinal Models

Hamiltonian sequential Monte Carlo with application to consumer choice behavior

Hamiltonian Monte Carlo using an adjoint-differentiated Laplace approximation: Bayesian inference for latent Gaussian models and beyond

Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases

Quantify uncertainty by estimating the probability density function of the output of interest using MLMC based Bayes method

Copula-based pairwise estimator for quantile regression with hierarchical missing data

Bayesian Quantile Regression with Subset Selection: A Posterior Summarization Perspective