Abstract:Learning high-dimensional distributions is a significant challenge in machine learning and statistics. Classical research has mostly concentrated on asymptotic analysis of such data under suitable assumptions. While existing works [Bhattacharyya et al.: SICOMP 2023, Daskalakis et al.: STOC 2021, Choo et al.: ALT 2024] focus on discrete distributions, the current work addresses the tree structure learning problem for Gaussian distributions, providing efficient algorithms with solid theoretical guarantees. This is crucial as real-world distributions are often continuous and differ from the discrete scenarios studied in prior works. In this work, we design a conditional mutual information tester for Gaussian random variables that can test whether two Gaussian random variables are independent, or their conditional mutual information is at least $\varepsilon$, for some parameter $\varepsilon \in (0,1)$ using $\mathcal{O}(\varepsilon^{-1})$ samples which we show to be near-optimal. In contrast, an additive estimation would require $\Omega(\varepsilon^{-2})$ samples. Our upper bound technique uses linear regression on a pair of suitably transformed random variables. Importantly, we show that the chain rule of conditional mutual information continues to hold for the estimated (conditional) mutual information. As an application of such a mutual information tester, we give an efficient $\varepsilon$-approximate structure-learning algorithm for an $n$-variate Gaussian tree model that takes $\widetilde{\Theta}(n\varepsilon^{-1})$ samples which we again show to be near-optimal. In contrast, when the underlying Gaussian model is not known to be tree-structured, we show that $\widetilde{\Theta}(n^2\varepsilon^{-2})$ samples are necessary and sufficient to output an $\varepsilon$-approximate tree structure. We perform extensive experiments that corroborate our theoretical convergence bounds.

Sgmit: Using Selfish Gene Theory To Construct Mutual Information Trees For Optimization

A Method for Assessing the Performance of Genetic Algorithm Optimization for Water Distribution Design

Efficient Sample-optimal Learning of Gaussian Tree Models via Sample-optimal Testing of Gaussian Mutual Information

An Integrated Genetic Algorithm To Optimize Engineering Information Flows Through Task Splitting And Partitioning

Estimation of Distribution Algorithm with Scatter Search for Dynamic Optimisation Problems

Improving Decision Trees by Tsallis Entropy Information Metric Method.

Independent component analysis based on genetic algorithms

Adaptive Memetic Computing for Evolutionary Multiobjective Optimization

Interactive Genetic Algorithm Assisted with Collective Intelligence from Group Decision Making

Improving Mutual Information Estimation with Annealed and Energy-Based Bounds

Personalized Search Inspired Fast Interactive Estimation of Distribution Algorithm and Its Application

Mutual Information Multinomial Estimation

An improved model‐based evolutionary algorithm for multi‐objective optimization

Surrogate-based Optimization using Mutual Information for Computer Experiments (optim-MICE)

Improving Evolutionary Multitasking Optimization by Leveraging Inter-Task Gene Similarity and Mirror Transformation

A less-greedy two-term Tsallis Entropy Information Metric approach for decision tree classification

Improved Information Gain Estimates for Decision Tree Induction

From Understanding Genetic Drift to a Smart-Restart Mechanism for Estimation-of-Distribution Algorithms

Headpin: a serpin with endogenous and exogenous suppression of angiogenesis.

An approach of gene regulatory network construction using mixed entropy optimizing context-related likelihood mutual information

A New Approach for Feature Selection from Microarray Data Based on Mutual Information.