Amplifying Inter-Message Distance: On Information Divergence Measures in Big Data.

Rui She,Shanyun Liu,Pingyi Fan
DOI: https://doi.org/10.1109/ACCESS.2017.2768385
IF: 3.9
2017-01-01
IEEE Access
Abstract:Message identification (M-I) divergence is an important measure of the information distance between probability distributions, similar to Kullback Leibler (K-L) and Renyi divergence. In fact, M-I divergence with a variable parameter can make an effect on characterization of distinction between two distributions. Furthermore, by choosing an appropriate parameter of M-I divergence, it is possible to amplify the information distance between adjacent distributions while maintaining enough gap between two nonadjacent ones. Therefore, M-I divergence can play a vital role in distinguishing distributions more clearly. In this paper, we first define a parametric M-I divergence in the view of information theory and then present its major properties. In addition, we design a M-I divergence estimation algorithm by means of the ensemble estimator of the proposed weight kernel estimators, which can improve the convergence of mean squared error from O(Gamma(-j/d)) to O(F-1) (Gamma(-1)(j is an element of(0, d]). We also discuss the decision with M-I divergence for clustering or classification, and investigate its performance in a statistical sequence model of big data for the outlier detection problem.
What problem does this paper attempt to address?