Abstract:We develop a class of interacting particle systems for implementing a maximum marginal likelihood estimation (MMLE) procedure to estimate the parameters of a latent variable model. We achieve this by formulating a continuous-time interacting particle system which can be seen as a Langevin diffusion over an extended state space of parameters and latent variables. In particular, we prove that the parameter marginal of the stationary measure of this diffusion has the form of a Gibbs measure where number of particles acts as the inverse temperature parameter in classical settings for global optimisation. Using a particular rescaling, we then prove geometric ergodicity of this system and bound the discretisation error in a manner that is uniform in time and does not increase with the number of particles. The discretisation results in an algorithm, termed Interacting Particle Langevin Algorithm (IPLA) which can be used for MMLE. We further prove nonasymptotic bounds for the optimisation error of our estimator in terms of key parameters of the problem, and also extend this result to the case of stochastic gradients covering practical scenarios. We provide numerical experiments to illustrate the empirical behaviour of our algorithm in the context of logistic regression with verifiable assumptions. Our setting provides a straightforward way to implement a diffusion-based optimisation routine compared to more classical approaches such as the Expectation Maximisation (EM) algorithm, and allows for especially explicit nonasymptotic bounds.

Gradient-based simulated maximum likelihood estimation for Lévy-driven Ornstein–Uhlenbeck stochastic volatility models

Gradient-based Simulated Maximum Likelihood Estimation for Stochastic Volatility Models Using Characteristic Functions

Maximum Likelihood Estimation for Mixed Fractional Vasicek Processes

Estimation and Inference in Fractional Ornstein–Uhlenbeck Model with Discrete-Sampled Data∗

METHOD OF MOMENTS ESTIMATION FOR LÉVY-DRIVEN ORNSTEIN–UHLENBECK STOCHASTIC VOLATILITY MODELS

Power Utility Maximization Problem under Rough Stochastic Local Volatility Models with Continuous-Time Markov Chain Approximation

Maximum Likelihood Estimation for the Fractional Vasicek Model

Rate Of Convergence For The Maximum Likelihood Estimator In Fractional Ornstein-Uhlenbeck Processes

Mixed Sub-fractional Brownian Motion and Drift Estimation of Related Ornstein–Uhlenbeck Process

Sequential maximum likelihood estimation for reflected generalized Ornstein-Uhlenbeck processes

A Simulation and Empirical Study of the Maximum Likelihood Estimator for Stochastic Volatility Jump-Diffusion Models

Parameter Estimation using Empirical Likelihood combined with Market Information

Efficient stochastic optimisation by unadjusted Langevin Monte Carlo. Application to maximum marginal likelihood and empirical Bayesian estimation

Parameter Estimation for Geometric Lévy Processes with Constant Volatility

Simulation of Lévy-driven models and its application in finance

Interacting Particle Langevin Algorithm for Maximum Marginal Likelihood Estimation

Simulation of Levy-Driven Models and Its Application in Finance

Maximum likelihood estimation in continuous affine Volterra processes in the ergodic regime

Variational inference of the drift function for stochastic differential equations driven by Lévy processes

Maximum Likelihood Estimation by Monte Carlo Simulation: Toward Data-Driven Stochastic Modeling.

Estimation of Tempered Stable Lévy Models of Infinite Variation