Abstract:A unified statistical learning approach called Bayesian Ying-Yang (BYY) system and theory has been developed by the present author in recent years. In a sister paper [14], this BYY system and theory has been shown to function as a general theory for unsupervised learning and its extension called semi-unsupervised learning, such that not only several existing popular unsupervised learning approaches are included as special cases, but also a number of new theories and models are provided for unsupervised pattern recognition and cluster analysis, data dimension reduction, and independent component analysis. In this paper, the basic system and theory in [14] is further theoretically justified and extended into a general system and theory with a general implementation technique such that not only those results [14] are kept as special cases still, but also it works for supervised learning and temporal modeling on parameter learning, regularization, structural scale or complexity selection, and architecture design. Particularly, temporal modeling and regression based on Hidden Markov Model (HMM) and the linear and nonlinear state space model are discussed in detail, with an adaptive algorithm proposed for various specific variants of HMM model and state space models. Moreover, the criteria for deciding the number of hidden states in HMM and the order of state space are also proposed. In another sister paper [2] of this proceeding, several specific models and algorithms as well as model selection criteria will be given for dependence reduction, data dimension reduction, independent component analysis, supervised classification and regression. In addition, the relation of the BW learning system and theory to a number of existing learning models and theories has been discussed in [14].

The Learning Algorithm of the Second Order HMM and Its Relationship with the First Order HMM

Incremental learning algorithm based on support vector machine

Two Phases Hybrid Algorithm for DHMM Training

A survey on HHL algorithm: From theory to application in quantum machine learning

A New Algorithm for Hidden Markov Models Learning Problem

A tutorial of techniques for improving standard Hidden Markov Model algorithms

Compensation for the Homogeneous Assumption of Classical HMM by Probability Penalty

Principles of Non-stationary Hidden Markov Model and Its Applications to Sequence Labeling Task

Learning Theory for Inferring Interaction Kernels in Second-Order Interacting Agent Systems

A Concise Information-Theoretic Derivation of the Baum-Welch algorithm

Second-Order Regular Variation and Second-Order Approximation of Hawkes Processes

Infinite Viterbi alignments in the two state hidden Markov models

Connecting First and Second Order Recurrent Networks with Deterministic Finite Automata

Bayesian Ying-Yang System and Theory as A Unified Statistical Learning Approach (II): From Unsupervised Learning to Supervised Learning and Temporal Modeling

Discriminative GMM-HMM acoustic model selection using two-level bayesian ying-yang harmony learning

On a New Kth-Order Quadratic Learning Algorithm

Second Order Asymptotics for Quantum Hypothesis Testing

On learning parametric-output HMMs

Logical Hidden Markov Models

A new quantum machine learning algorithm: split hidden quantum Markov model inspired by quantum conditional master equation

Stability and Generalization of the Learning Algorithm (Ⅱ): the Framework of the Bounded-Difference Stability