Abstract:The Bayesian Ying-Yang (BYY) harmony learning acts as a general statistical learning framework, featured by not only new regularization techniques for parameter learning but also a new mechanism that implements model selection either automatically during parameter learning or via a new class of model selection criteria used after parameter learning. In this paper, further advances on BYY harmony learning by considering modular inner representations are presented in three parts. One consists of results on unsupervised mixture models, ranging from Gaussian mixture based Mean Square Error (MSE) clustering, elliptic clustering, subspace clustering to NonGaussian mixture based clustering not only with each cluster represented via either Bernoulli–Gaussian mixtures or independent real factor models, but also with independent component analysis implicitly made on each cluster. The second consists of results on supervised mixture-of-experts (ME) models, including Gaussian ME, Radial Basis Function nets, and Kernel regressions. The third consists of two strategies for extending the above structural mixtures into self-organized topological maps. All these advances are introduced with details on three issues, namely, (a) adaptive learning algorithms, especially elliptic, subspace, and structural rival penalized competitive learning algorithms, with model selection made automatically during learning; (b) model selection criteria for being used after parameter learning, and (c) how these learning algorithms and criteria are obtained from typical special cases of BYY harmony learning.

IL-Strudel : Independence-Based Learning of Structured-Decomposable Probabilistic Circuit Ensembles

Learning The Structure Of Probabilistic Sentential Decision Diagrams

Soft Learning Probabilistic Circuits

Learning Disentangled Representation with Pairwise Independence

A Unified Framework for Human-Allied Learning of Probabilistic Circuits

Leveraging Weighted Automata in Compositional Reasoning about Concurrent Probabilistic Systems

Integrating Transformations in Probabilistic Circuits

Restructuring Tractable Probabilistic Circuits

Learning the Finer Things: Bayesian Structure Learning at the Instantiation Level

Sparse Probabilistic Circuits via Pruning and Growing

Subtractive Mixture Models via Squaring: Representation and Learning

Supervised structure learning

Scaling Up Probabilistic Circuits by Latent Variable Distillation

Learning Mixtures of Plackett-Luce Models from Structured Partial Orders

A probabilistic model for learning in cortical microcircuit motifs with data-based divisive inhibition

Sum of Squares Circuits

Probabilistic Transfer Learning Through Ensemble Probabilistic Deep Neural Network

Probabilistic Circuits with Constraints via Convex Optimization

Probabilistic Modeling with Matrix Product States

BYY harmony learning, structural RPCL, and topological self-organizing on mixture models

Learning of Structurally Unambiguous Probabilistic Grammars