Abstract:The minimum error entropy (MEE) criterion finds extensive utility across diverse applications, particularly in contexts characterized by non-Gaussian noise. However, its computational demands are notable, and are primarily attributable to the double summation operation involved in calculating the probability density function (PDF) of the error. To address this, our study introduces a novel approach, termed the fast minimum error entropy (FMEE) algorithm, aimed at mitigating computational complexity through the utilization of polynomial expansions of the error PDF. Initially, the PDF approximation of a random variable is derived via the Gram–Charlier expansion. Subsequently, we proceed to ascertain and streamline the entropy of the random variable. Following this, the error entropy inherent to the linear regression model is delineated and expressed as a function of the regression coefficient vector. Lastly, leveraging the gradient descent algorithm, we compute the regression coefficient vector corresponding to the minimum error entropy. Theoretical scrutiny reveals that the time complexity of FMEE stands at O(n), in stark contrast to the O(n2) complexity associated with MEE. Experimentally, our findings underscore the remarkable efficiency gains afforded by FMEE, with time consumption registering less than 1‰ of that observed with MEE. Encouragingly, this efficiency leap is achieved without compromising accuracy, as evidenced by negligible differentials observed between the accuracies of FMEE and MEE. Furthermore, comprehensive regression experiments on real-world electric datasets in northwest China demonstrate that our FMEE outperforms baseline methods by a clear margin.

Insights into the Robustness of Minimum Error Entropy Estimation

Minimum Total Error Entropy Method for Parameter Estimation

On Optimal Estimations with Minimum Error Entropy Criterion

Some Further Results on the Minimum Error Entropy Estimation.

A New Interpretation on the MMSE As a Robust MEE Criterion

An Extended Result on the Optimal Estimation under the Minimum Error Entropy Criterion

Restricted Minimum Error Entropy Criterion for Robust Classification

Fast Minimum Error Entropy for Linear Regression

Quantized Minimum Error Entropy with Fiducial Points for Robust Regression.

Convergence of a Fixed-Point Minimum Error Entropy Algorithm

On the Smoothed Minimum Error Entropy Criterion.

Generalized Minimum Error Entropy for Robust Learning

Robust Adaptive Filtering based on M-estimation-based Minimum Error Entropy Criterion

Robust Constrained Adaptive Filtering under Minimum Error Entropy Criterion

Understanding the Method of Interval Errors from the Information Theory Perspective

A Note on the W-S Lower Bound of the MEE Estimation

Consistency analysis of an empirical minimum error entropy algorithm

Sparsity Aware Minimum Error Entropy Algorithms

Convex Regularized Recursive Minimum Error Entropy Algorithm

An Analysis of Minimum Error Entropy Loss Functions in Wireless Communications

Fixed-Point Minimum Error Entropy With Fiducial Points