Abstract:Maximum Likelihood (ML) estimation requires precise knowledge of the underlying statistical model. In Quasi ML (QML), a presumed model is used as a substitute to the (unknown) true model. In the context of Independent Vector Analysis (IVA), we consider the Gaussian QML Estimate (QMLE) of the demixing matrices set and present an (approximate) analysis of its asymptotic separation performance. In Gaussian QML the sources are presumed to be Gaussian, with covariance matrices specified by some "educated guess". The resulting quasi-likelihood equations of the demixing matrices take a special form, recently termed an extended "Sequentially Drilled" Joint Congruence (SeDJoCo) transformation, which is reminiscent of (though essentially different from) classical joint diagonalization. We show that asymptotically this QMLE, i.e., the solution of the resulting extended SeDJoCo transformation, attains perfect separation (under some mild conditions) regardless of the sources' true distributions and/or covariance matrices. In addition, based on the "small-errors" assumption, we present a first-order perturbation analysis of the extended SeDJoCo solution. Using the resulting closed-form expressions for the errors in the solution matrices, we provide closed-form expressions for the resulting Interference-to-Source Ratios (ISRs) for IVA. Moreover, we prove that asymptotically the ISRs depend only on the sources' covariances, and not on their specific distributions. As an immediate consequence of this result, we provide an asymptotically attainable lower bound on the resulting ISRs. We also present empirical results, corroborating our analytical derivations, of three simulation experiments concerning two possible model errors - inaccurate covariance matrices and sources' distribution mismodeling.

MAXIMUM ENTROPY AND MINIMAL MUTUAL INFORMATION IN A NONLINEAR MODEL

Comparison of maximum entropy and minimal mutual information in a nonlinear setting

A Sparse MLE Approach for Joint Interference Mitigation and Data Recovery

Study on Separation of Mechanical Vibration Based on the Principle of the Minimum Mutual Information

Spurious Solution of the Maximum Likelihood Approach to Ica

Maximal Information Divergence from Statistical Models defined by Neural Networks

Optimum estimation via gradients of partition functions and information measures: a statistical-mechanical perspective

Mutual information analysis on non-stationary neuron importance for brain machine interfaces.

An Information Theoretic Approach to the Autoencoder

A novel semi-blind source separation framework towards maximum signal-to-interference ratio

Mutual information maximization for amortized likelihood inference from sampled trajectories: MINIMALIST

On Maximum Entropy Linear Feature Inversion

Supposed Maximum Mutual Information for Improving Generalization and Interpretation of Multi-Layered Neural Networks

Mixing and Non-Mixing Local Minima of the Entropy Contrast for Blind Source Separation

Independent Component Regression Based on Mutual Information Maximization

Mutual Information Based Approach for Nonnegative Independent Component Analysis

On Model Identifiability in Analytic Postnonlinear ICA

Performance Analysis of the Gaussian Quasi-Maximum Likelihood Approach for Independent Vector Analysis

Entropy and mutual information in models of deep neural networks

Modeling Images and Textures by Minimax Entropy.

On Variational Bounds of Mutual Information