Abstract:Firstly proposed in 1995 and systematically developed in the past decade, Bayesian Ying-Yang learning 1) is a statistical approach for a two pathway featured intelligent system via two complementary Bayesian representations of a joint distribution on the external observation X and its inner representation R , which can be understood from a perspective of the ancient Ying-Yang philosophy. We have q ( X,R ) = q ( X | R ) q ( R ) as Ying that is primary, with its structure designed according to tasks of the system, and p ( X,R ) = p ( R | X ) p ( X ) as Yang that is secondary, with p ( X ) given by samples of X while the structure of p ( R | X ) designed from Ying according to a Ying-Yang variety preservation principle, i.e., p ( R | X ) is designed as a functional with q ( X | R ), q ( R ) as its arguments. We call this pair Bayesian Ying-Yang (BYY) system. A Ying-Yang best harmony principle is proposed for learning all the unknowns in the system, in help of an implementation featured by a five action circling under the name of A5 paradigm. Interestingly, it coincides with the famous ancient WuXing theory that provides a general guide to keep the A5 circling well balanced towards a Ying-Yang best harmony. This BYY learning provides not only a general framework that accommodates typical learning approaches from a unified perspective but also a new road that leads to improved model selection criteria, Ying-Yang alternative learning with automatic model selection, as well as coordinated implementation of Ying based model selection and Yang based learning regularization. This paper aims at an introduction of BYY learning in a twofold purpose. On one hand, we introduce fundamentals of BYY learning, including system design principles of least redundancy versus variety preservation, global learning principles of Ying-Yang harmony versus Ying-Yang matching, and local updating mechanisms of rival penalized competitive learning (RPCL) versus maximum a posteriori (MAP) competitive learning, as well as learning regularization by data smoothing and induced bias cancelation (IBC) priori. Also, we introduce basic implementing techniques, including apex approximation, primal gradient flow, Ying-Yang alternation, and Sheng-Ke-Cheng-Hui law. On the other hand, we provide a tutorial on learning algorithms for a number of typical learning tasks, including Gaussian mixture, factor analysis (FA) with independent Gaussian, binary, and non-Gaussian factors, local FA, temporal FA (TFA), hidden Markov model (HMM), hierarchical BYY, three layer networks, mixture of experts, radial basis functions (RBFs), subspace based functions (SBFs). This tutorial aims at introducing BYY learning algorithms in a comparison with typical algorithms, particularly with a benchmark of the expectation maximization (EM) algorithm for the maximum likelihood. These algorithms are summarized in a unified Ying-Yang alternation procedure with major parts in a same expression while differences simply characterized by few options in some subroutines. Additionally, a new insight is provided on the ancient Chinese philosophy of Yin-Yang and WuXing from a perspective of information science and intelligent system.

Bayesian Ying-Yang Supervised Learning, Modular Models, and Three Layer Nets

Bayesian Ying-Yang System and Theory as a Unified Statistical Learning Approach: (III) Models and Algorithms for Dependence Reduction, Data Dimension Reduction, ICA and Supervised Learning

Bayesian Ying-Yang System and Theory as A Unified Statistical Learning Approach (II): From Unsupervised Learning to Supervised Learning and Temporal Modeling

Rbf Nets, Mixture Experts, and Bayesian Ying-Yang Learning

Bayesian Ying Yang Learning (II): A New Mechanism for Model Selection and Regularization

New Advances on Bayesian Ying-Yang Learning System with Kullback and Non-Kullback Separation Functionals

BYY harmony learning, structural RPCL, and topological self-organizing on mixture models

2002 Special Issue BYY harmony learning, structural RPCL, and topological self-organizing on mixture modelsq

Bayesian ying-yang theory for empirical learning, regularization and model selection: general formulation

BYY Learning, Regularized Implementation, and Model Selection on Modular Networks with One Hidden Layer of Binary Units.

Best Harmony, Unified RPCL and Automated Model Selection for Unsupervised and Supervised Learning on Gaussian Mixtures, Three-Layer Nets and ME-RBF-SVM Models

Bayesian Ying-Yang system, best harmony learning, and five action circling

Further Advances on Bayesian Ying-Yang Harmony Learning

Bayesian Ying-Yang System and Theory As a Unified Statistical Learning Approach (VII): Data Smoothing.

Bayesian Ying Yang System, Best Harmony Learning, And Gaussian Manifold Based Family

Bayesian Ying Yang Learning (I): A Unified Perspective for Statistical Modeling

Bkyy Three Layer Net Learning, Em-Like Algorithm, And Selection Criterion For Hidden Unit Number

A Trend On Regularization And Model Selection In Statistical Learning: A Bayesian Ying Yang Learning Perspective

Finding optimal Bayesian networks by a layered learning method

Learning Bayesian Networks Using the Constrained Maximum a Posteriori Probability Method

Byy Harmony Learning, Model Selection, And Information Approach: Further Results