Abstract:Global-local shrinkage hierarchies are an important innovation in Bayesian estimation. We propose the use of log-scale distributions as a novel basis for generating familes of prior distributions for local shrinkage hyperparameters. By varying the scale parameter one may vary the degree to which the prior distribution promotes sparsity in the coefficient estimates. By examining the class of distributions over the logarithm of the local shrinkage parameter that have log-linear, or sub-log-linear tails, we show that many standard prior distributions for local shrinkage parameters can be unified in terms of the tail behaviour and concentration properties of their corresponding marginal distributions over the coefficients $\beta_j$. We derive upper bounds on the rate of concentration around $|\beta_j|=0$, and the tail decay as $|\beta_j| \to \infty$, achievable by this wide class of prior distributions. We then propose a new type of ultra-heavy tailed prior, called the log-$t$ prior with the property that, irrespective of the choice of associated scale parameter, the marginal distribution always diverges at $\beta_j = 0$, and always possesses super-Cauchy tails. We develop results demonstrating when prior distributions with (sub)-log-linear tails attain Kullback--Leibler super-efficiency and prove that the log-$t$ prior distribution is always super-efficient. We show that the log-$t$ prior is less sensitive to misspecification of the global shrinkage parameter than the horseshoe or lasso priors. By incorporating the scale parameter of the log-scale prior distributions into the Bayesian hierarchy we derive novel adaptive shrinkage procedures. Simulations show that the adaptive log-$t$ procedure appears to always perform well, irrespective of the level of sparsity or signal-to-noise ratio of the underlying model.

Catalytic Prior Distributions with Application to Generalized Linear Models

Catalytic Priors: Using Synthetic Data to Specify Prior Distributions in Bayesian Analysis

Bayesian inference on Cox regression models using catalytic prior distributions

Bayesian estimation and prediction for certain mixtures

Controlling the flexibility of non-Gaussian processes through shrinkage priors

The R2D2 Prior for Generalized Linear Mixed Models

Tail-adaptive Bayesian shrinkage

Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior

Translating predictive distributions into informative priors

A hierarchical prior for generalized linear models based on predictions for the mean response

Generalized cumulative shrinkage process priors with applications to sparse Bayesian factor analysis

Log-Scale Shrinkage Priors and Adaptive Bayesian Global-Local Shrinkage Estimation

Empirical Bayes inference in sparse high-dimensional generalized linear models

The Bridged Posterior: Optimization, Profile Likelihood and a New Approach to Generalized Bayes

Posterior propriety of an objective prior for generalized hierarchical normal linear models

Dirichlet process mixtures of block $g$ priors for model selection and prediction in linear models

Advances in Bayesian model selection consistency for high-dimensional generalized linear models

Potential Bias Models With Bayesian Shrinkage Priors for Dynamic Borrowing of Multiple Historical Control Data

Generating Independent Replicates Directly from the Posterior Distribution for a Class of Spatial Hierarchical Models

Stabilizing the Kumaraswamy Distribution