Approximate Multi-Information Diversity*

Tao SUN,Zhihua ZHOU
DOI: https://doi.org/10.3778/j.issn.1673-9418.1806007
2019-01-01
Abstract:Ensemble diversity, that is, the difference among the individual learners, is a fundamental issue in ensemble learning. Multi-information diversity measures ensemble diversity based on information theory and provides a promising direction for understanding ensemble diversity. In practice, the high-order information is usually hard to be estimated. This paper proposes to estimate high-order information based on a particular kind of k-order t-cherry junction tree so as to get approximate multi-information diversity. The method includes approximating multi-information directly as well as approximating its mathematical components by the junction tree, and the relevance between the two cases is discussed. Experiments show that this estimation method is superior to established ones under the same low-order approximation.
What problem does this paper attempt to address?