Abstract:The Fisher information matrix (FIM) is a key quantity in statistics as it is required for example for evaluating asymptotic precisions of parameter estimates, for computing test statistics or asymptotic distributions in statistical testing, for evaluating post model selection inference results or optimality criteria in experimental designs. However its exact computation is often not trivial. In particular in many latent variable models, it is intricated due to the presence of unobserved variables. Therefore the observed FIM is usually considered in this context to estimate the FIM. Several methods have been proposed to approximate the observed FIM when it can not be evaluated analytically. Among the most frequently used approaches are Monte-Carlo methods or iterative algorithms derived from the missing information principle. All these methods require to compute second derivatives of the complete data log-likelihood which leads to some disadvantages from a computational point of view. In this paper, we present a new approach to estimate the FIM in latent variable model. The advantage of our method is that only the first derivatives of the log-likelihood is needed, contrary to other approaches based on the observed FIM. Indeed we consider the empirical estimate of the covariance matrix of the score. We prove that this estimate of the Fisher information matrix is unbiased, consistent and asymptotically Gaussian. Moreover we highlight that none of both estimates is better than the other in terms of asymptotic covariance matrix. When the proposed estimate can not be directly analytically evaluated, we present a stochastic approximation estimation algorithm to compute it. This algorithm provides this estimate of the FIM as a by-product of the parameter estimates. We emphasize that the proposed algorithm only requires to compute the first derivatives of the complete data log-likelihood with respect to the parameters. We prove that the estimation algorithm is consistent and asymptotically Gaussian when the number of iterations goes to infinity. We evaluate the finite sample size properties of the proposed estimate and of the observed FIM through simulation studies in linear mixed effects models and mixture models. We also investigate the convergence properties of the estimation algorithm in non linear mixed effects models. We compare the performances of the proposed algorithm to those of other existing methods.

Rates of Fisher information convergence in the central limit theorem for nonlinear statistics

Rates of Fisher information convergence in the central limit theorem for nonlinear statistics

Fisher information bounds and applications to SDEs with small noise

Linear Convergence in Hilbert's Projective Metric for Computing Augustin Information and a Rényi Information Measure

Central Limit Theorems for Generalizedu-Statistics with Applications in Nonparametric Specification

Fisher information under local differential privacy

How to estimate Fisher information matrices from simulations

On the density of nonlinear statistics

Normal approximation for nonlinear statistics using a concentration inequality approach

On some information-theoretic aspects of non-linear statistical inverse problems

Finite-Sample Symmetric Mean Estimation with Fisher Information Rate

Estimating Fisher Information Matrix in Latent Variable Models based on the Score Function

Fisher information dissipation for time inhomogeneous stochastic differential equations

Rates of convergence in the central limit theorem for Banach valued dependent variables

$L^\infty$-optimal transport of anisotropic log-concave measures and exponential convergence in Fisher's infinitesimal model

On the monotonicity of the Fisher information for the Boltzmann equation

Fisher information and continuity estimates for nonlinear but 1-homogeneous diffusive PDEs (via the JKO scheme)

Central limit theorem for intrinsic Fréchet means in smooth compact Riemannian manifolds

Convergence Rates for the Generalized Fréchet Mean via the Quadruple Inequality

Any Kähler metric is a Fisher information metric

Rates of convergence for the continuum limit of nondominated sorting