Huffman Metric: A Novel Information Divergence Measure for Probability Distribution

Gangtao Xin,Pingyi Fan
DOI: https://doi.org/10.1109/iccc62479.2024.10681824
2024-01-01
Abstract:Information Divergence serves as a method for measuring the difference between two probability distributions, directly impacting the performance of various tasks in statistical inference and data processing. From a practical and pragmatic point of view, we propose a novel information divergence, termed the Huffman metric. It leverages the real impact of Huffman coding to quantify the difference between probability distributions. Additionally, it encapsulates insights from source coding in information theory, defining distinctions in information sources from the perspective of optimal codes. The Huffman metric is expected to have the potential to be applied in specific scenarios of machine learning.
What problem does this paper attempt to address?