Abstract:In this work we connect two notions: That of the nonparametric mode of a probability measure, defined by asymptotic small ball probabilities, and that of the Onsager-Machlup functional, a generalized density also defined via asymptotic small ball probabilities. We show that in a separable Hilbert space setting and under mild conditions on the likelihood, modes of a Bayesian posterior distribution based upon a Gaussian prior exist and agree with the minimizers of its Onsager-Machlup functional and thus also with weak posterior modes. We apply this result to inverse problems and derive conditions on the forward mapping under which this variational characterization of posterior modes holds. Our results show rigorously that in the limit case of infinite-dimensional data corrupted by additive Gaussian or Laplacian noise, nonparametric maximum a posteriori estimation is equivalent to Tikhonov-Phillips regularization. In comparison with the work of Dashti, Law, Stuart, and Voss (2013), the assumptions on the likelihood are relaxed so that they cover in particular the important case of white Gaussian process noise. We illustrate our results by applying them to a severely ill-posed linear problem with Laplacian noise, where we express the maximum a posteriori estimator analytically and study its rate of convergence in the small noise limit.

Toric invariant theory for maximum likelihood estimation in log-linear models

A Bridge between Invariant Theory and Maximum Likelihood Estimation

Maximum Likelihood Estimation of Toric Fano Varieties

Maximality of Laplacian Algebras, with Applications to Invariant Theory

On the maximum likelihood estimation in general log-linear models

Are minimizers of the Onsager-Machlup functional strong posterior modes?

Maximum information divergence from linear and toric models

Tropical toric maximum likelihood estimation

Toric geometry of entropic regularization

Likelihood Correspondence of Statistical Models

Maximum likelihood estimation in continuous affine Volterra processes in the ergodic regime

Geometry of rational quasi-independence models as toric fiber products

Maximum likelihood estimation for Gaussian processes under inequality constraints

Maximum Likelihood for Matrices with Rank Constraints

On variational inference and maximum likelihood estimation with the λ-exponential family

Minimal Invariant Tori in the Resonant Regions for Nearly Integrable Hamiltonian Systems

On the vanishing of eigenfunctions of the Laplacian on tori

A modern maximum-likelihood theory for high-dimensional logistic regression

Invariant measures from locally bounded orbits

Invariant manifolds of maps and vector fields with nilpotent parabolic tori

Gaussian graphical models with toric vanishing ideals