Abstract:Markov Chain Monte Carlo (MCMC) is a robust statistical approach for estimating posterior distributions. However, the significant computational cost associated with MCMC presents a considerable challenge, complicating the selection of an appropriate algorithm tailored to the specific problem at hand. This study introduces a novel and comprehensive framework for evaluating the performance of MCMC algorithms, drawing inspiration from diagnostics used for multi-objective evolutionary algorithms. We employ visualizations to evaluate key algorithmic characteristics: Effectiveness (the ability to accurately find representative posterior modes, quantified by the Kullback-Leibler Divergence (KLD) and Wasserstein Distance (WD)), Efficiency (the speed of posterior characterization), Reliability (consistency across different random seeds), and Controllability (insensitivity to hyperparameter variation). Evaluating three prominent MCMC algorithms—Metropolis-Hastings (MH), Adaptive Metropolis (AM), and Differential Evolution Adaptive Metropolis (DREAM)—on high-dimensional and bimodal test problems, our analysis uncovers several insights. First, across algorithms, the number of function evaluations most controls performance on the high-dimensional problem, while the number of chains most controls performance on the bimodal problem. While this suggests similar controllability across algorithms, differences emerge on the other algorithmic characteristics. For high numbers of functions evaluations, AM performs best on the high-dimensional problem, while for low ( 15) chain counts, MH and AM perform best on the bimodal problem, as measured by KLD. However, outside these specific cases, DREAM consistently demonstrates superior efficiency and reliability, making it a robust choice for both high-dimensional and multimodal problems. These findings can inform MCMC algorithm selection for Bayesian inference applications, as well as hyperparameterization of the chosen algorithm. More importantly, the diagnostics represent a generalizable contribution to research on MCMC diagnostics that can be used to evaluate and inform the design of new algorithms.

A Metric Based on the Efficient Determination Criterion

EBT: a Statistic Test Identifying Moderate Size of Significant Features with Balanced Power and Precision for Genome-Wide Rate Comparisons

A Vine-Copula Based Adaptive Mcmc Sampler for Efficient Inference of Dynamical Systems

New Diagnostic Assessment of MCMC Algorithm Effectiveness, Efficiency, Reliability, and Controllability

Generalized Centroid Estimators in Bioinformatics

Convergence Analysis of Markov Chain Monte Carlo Estimators of Discrete Choice Models in Transportation

Automatic Forward Model Parameterization with Bayesian Inference of Conformational Populations

A new mixture-based fixed-effect model for a biometrical case-study related to immunogenecity with highly censored data

The Effective Sample Size in Bayesian Information Criterion for Level-Specific Fixed and Random Effects Selection in a Two-Level Nested Model

An MCMC method for the evaluation of the Fisher information matrix for non-linear mixed effect models

Bayesian variable selection using cost-adjusted BIC, with application to cost-effective measurement of quality of health care

Robust parametric inference for finite Markov chains

Fit for a Bayesian: An Evaluation of PPP and DIC for Structural Equation Modeling

Choosing the number of factors in factor analysis with incomplete data via a novel hierarchical Bayesian information criterion

Bayesian inference for multiple Gaussian graphical models with application to metabolic association networks

Penalized Bayesian forward continuation ratio model with application to high-dimensional data with discrete survival outcomes

Scalable Estimation of Multinomial Response Models with Random Consideration Sets

The Focused Information Criterion for Stochastic Model Selection Problems Using $M$-Estimators

Bayesian monotonic errors-in-variables models with applications to pathogen susceptibility testing

An Improved Evidential Markov Decision Making Model

Modeling citation concentration through a mixture of Leimkuhler curves