Abstract:In this paper, we provide a multiscale perspective on the problem of maximum marginal likelihood estimation. We consider and analyse a diffusion-based maximum marginal likelihood estimation scheme using ideas from multiscale dynamics. Our perspective is based on stochastic averaging; we make an explicit connection between ideas in applied probability and parameter inference in computational statistics. In particular, we consider a general class of coupled Langevin diffusions for joint inference of latent variables and parameters in statistical models, where the latent variables are sampled from a fast Langevin process (which acts as a sampler), and the parameters are updated using a slow Langevin process (which acts as an optimiser). We show that the resulting system of stochastic differential equations (SDEs) can be viewed as a two-time scale system. To demonstrate the utility of such a perspective, we show that the averaged parameter dynamics obtained in the limit of scale separation can be used to estimate the optimal parameter, within the strongly convex setting. We do this by using recent uniform-in-time non-asymptotic averaging bounds. Finally, we conclude by showing that the slow-fast algorithm we consider here, termed Slow-Fast Langevin Algorithm, performs on par with state-of-the-art methods on a variety of examples. We believe that the stochastic averaging approach we provide in this paper enables us to look at these algorithms from a fresh angle, as well as unlocking the path to develop and analyse new methods using well-established averaging principles.

Asymptotic properties of AD(1, n) model and its maximum likelihood estimator

On Conditional least squares estimation for the AD(1,n) model

Asymptotic Theory for Explosive Fractional Ornstein-Uhlenbeck Processes

On the Approximate Maximum Likelihood Estimation for Diffusion Processes

Asymptotic analysis of estimators of ergodic stochastic differential equations

Asymptotic Inference for A Nonstationary Double Ar(1) Model

Asymptotics for A Class of Dependent Random Variables

Asymptotic Theory of Least Distances Estimate in Multivariate Linear Models

Asymptotic Behavior of the Maximum Likelihood Estimator for General Markov Switching Models

Asymptotic Properties of Nonparametric M-Estimation for Mixing Functional Data

On Bayesian Asymptotics in Stochastic Differential Equations with Random Effects

Maximum likelihood estimation in continuous affine Volterra processes in the ergodic regime

Asymptotic Inference in Multiple-Threshold Double Autoregressive Models

Asymptotic Smiles for an Affine Jump-Diffusion Model

Affine Jump-Diffusions: Stochastic Stability and Limit Theorems

Asymptotic inference for a stochastic differential equation with uniformly distributed time delay

On asymptotic inference in stochastic differential equations with time‐varying covariates

A Multiscale Perspective on Maximum Marginal Likelihood Estimation

Maximum Likelihood Estimation for Conditionally Heteroscedastic Models when the Innovation Process is in the Domain of Attraction of a Stable Law

Asymptotic distributions related to mildly-explosive second order autoregressive models

NON‐STATIONARITY AND QUASI‐MAXIMUM LIKELIHOOD ESTIMATION ON A DOUBLE AUTOREGRESSIVE MODEL