Abstract:A method is introduced for approximate marginal likelihood inference via adaptive Gaussian quadrature in mixed models with a single grouping factor. The core technical contribution is an algorithm for computing the exact gradient of the approximate log marginal likelihood. This leads to efficient maximum likelihood via quasi-Newton optimization that is demonstrated to be faster than existing approaches based on finite-differenced gradients or derivative-free optimization. The method is specialized to Bernoulli mixed models with multivariate, correlated Gaussian random effects; here computations are performed using an inverse log-Cholesky parameterization of the Gaussian density that involves no matrix decomposition during model fitting, while Wald confidence intervals are provided for variance parameters on the original scale. Simulations give evidence of these intervals attaining nominal coverage if enough quadrature points are used, for data comprised of a large number of very small groups exhibiting large between-group heterogeneity. In contrast, the Laplace approximation is shown to give especially poor coverage and high bias for data comprised of a large number of small groups. Adaptive quadrature mitigates this, and the methods in this paper improve the computational feasibility of this more accurate method. All results may be reproduced using code available at \url{<a class="link-external link-https" href="https://github.com/awstringer1/aghmm-paper-code" rel="external noopener nofollow">this https URL</a>}.

An adaptive mixture-population Monte Carlo method for likelihood-free inference

Unbiased MLMC-based Variational Bayes for Likelihood-Free Inference

A New Reliability Method Combining Adaptive Kriging and Active Variance Reduction Using Multiple Importance Sampling

Monte Carlo Approximation of Bayes Factors via Mixing with Surrogate Distributions

A smoothed semiparametric likelihood for estimation of nonparametric finite mixture models with a copula-based dependence structure

Adaptive nonparametric Bayesian inference using location-scale mixture priors

On the Behavior of the Expectation-Maximization Algorithm for Mixture Models

Efficient stochastic optimisation by unadjusted Langevin Monte Carlo. Application to maximum marginal likelihood and empirical Bayesian estimation

Model selection in finite mixture of regression models: a Bayesian approach with innovative weighted g priors and reversible jump Markov chain Monte Carlo implementation (vol 85, pg 2456, 2015)

On Efficient and Scalable Computation of the Nonparametric Maximum Likelihood Estimator in Mixture Models

Pseudo Independent Conditional Approximation for Training the Mixtures of Gaussian Processes

Adaptive sequential Monte Carlo by means of mixture of experts

Probabilistic Inference from Arbitrary Uncertainty using Mixtures of Factorized Generalized Gaussians

Parallel Generalized Elliptical Slice Sampling with Adaptive Regional Pseudo-Priors

Penalized Maximum Likelihood Estimator for Mixture of von Mises-Fisher Distributions

Variance Reduction in Black-box Variational Inference by Adaptive Importance Sampling

Likelihood approximations via Gaussian approximate inference

Approximate Marginal Likelihood Inference in Mixed Models for Grouped Data

Sample-efficient neural likelihood-free Bayesian inference of implicit HMMs

Estimating the Number of Components in Finite Mixture Models via Variational Approximation

Inverse Gaussian Process regression for likelihood-free inference