Abstract:The Fisher information matrix (FIM) is a key quantity in statistics as it is required for example for evaluating asymptotic precisions of parameter estimates, for computing test statistics or asymptotic distributions in statistical testing, for evaluating post model selection inference results or optimality criteria in experimental designs. However its exact computation is often not trivial. In particular in many latent variable models, it is intricated due to the presence of unobserved variables. Therefore the observed FIM is usually considered in this context to estimate the FIM. Several methods have been proposed to approximate the observed FIM when it can not be evaluated analytically. Among the most frequently used approaches are Monte-Carlo methods or iterative algorithms derived from the missing information principle. All these methods require to compute second derivatives of the complete data log-likelihood which leads to some disadvantages from a computational point of view. In this paper, we present a new approach to estimate the FIM in latent variable model. The advantage of our method is that only the first derivatives of the log-likelihood is needed, contrary to other approaches based on the observed FIM. Indeed we consider the empirical estimate of the covariance matrix of the score. We prove that this estimate of the Fisher information matrix is unbiased, consistent and asymptotically Gaussian. Moreover we highlight that none of both estimates is better than the other in terms of asymptotic covariance matrix. When the proposed estimate can not be directly analytically evaluated, we present a stochastic approximation estimation algorithm to compute it. This algorithm provides this estimate of the FIM as a by-product of the parameter estimates. We emphasize that the proposed algorithm only requires to compute the first derivatives of the complete data log-likelihood with respect to the parameters. We prove that the estimation algorithm is consistent and asymptotically Gaussian when the number of iterations goes to infinity. We evaluate the finite sample size properties of the proposed estimate and of the observed FIM through simulation studies in linear mixed effects models and mixture models. We also investigate the convergence properties of the estimation algorithm in non linear mixed effects models. We compare the performances of the proposed algorithm to those of other existing methods.

Fisher information in order statistics and their concomitants for Cambanis bivariate distribution

Second-order concomitants of order statistics from bivariate Cambanis family: some information measures–estimation

Fisher and Bayes-Fisher information measures for finite mixture distributions

On the Analysis of the Fisher Information of a Perturbed Linear Model after Random Compression

Fisher-Based Inaccuracy Information Measure

Measures of Extropy Based on Concomitants of Generalized Order Statistics under a General Framework from Iterated Morgenstern Family

How to estimate Fisher information matrices from simulations

Fisher's Combined Probability Test for High-Dimensional Covariance Matrices

Extropy Properties of Ranked Set Sample for Cambanis Type Bivariate Distributions

Fisher information of correlated stochastic processes

Fisher Information and Spin Squeezing in the Lipkin-Meshkov-Glick Model

Information Measures and Concomitants of k-Record Values Based on Sarmanov Family of Bivariate Distributions

Semiparametric fiducial inference

Inference and other aspects for $ q- $Weibull distribution via generalized order statistics with applications to medical datasets

Fisher Information Among β-Ensembles

Parameter Estimation on Homogeneous Spaces

Estimating Fisher Information Matrix in Latent Variable Models based on the Score Function

The Fisher-Rao geometry of CES distributions

Extropy and statistical features of dual generalized order statistics' concomitants arising from the Sarmanov family

Quantum Fisher Information and its dynamical nature

Escort--Husimi distributions, Fisher information and nonextensivity