Abstract:In this thesis, we, empirically or theoretically, investigate non-Gaussian Factor Analysis (NFA) models with different underlying structures. We focus on the problem of determining the number of latent factors of NFA, from two-stage approach model selection to automatic model selection, with real applications in pattern recognition and bioinformatics. We start by a degenerate case of NFA, the conventional Factor Analysis (FA) with latent Gaussian factors. Many model selection methods have been proposed and used for FA, and it is important to examine their relative strengths and weaknesses. We develop an empirical analysis tool, to facilitate a systematic comparison on model selection performances of not only classical criteria (e.g., Akaike’s information criterion or shortly AIC) but also recently developed methods (e.g., Kritchman & Nadler’s hypothesis tests), as well as the Bayesian Ying-Yang (BYY) harmony learning. Also, we prove a theoretical relative order of underestimation tendency of four classical criteria.Then, we investigate how parameterizations affect model selection performance, an issue that has been ignored or seldom studied since traditional model selection criteria, like AIC, perform equivalently on different parameterizations that have equivalent likelihood functions. Focusing on two typical parameterizations of FA, one of which is found to be better than the other under both Variational Bayes (VB) and BYY via extensive experiments on synthetic and real data. Moreover, a family of FA parameterizations that have equivalent likelihood functions are presented, where each one is featured by an integer r, with the two known parameterizations being both ends as r varies from zero to its upper bound. To describe binary latent features, we proceed to binary factor analysis (BFA), which considers Bernoulli factors. First, we introduce a canonical dual approach to tackling a difficult Binary Quadratic Programming (BQP) problem encountered as a computational bottleneck in BFA learning. Although it is not an exact BQP solver, it improves the learning speed and model selection accuracy, which indicates that some amount of error in solving the BQP, a problem nested in the hierarchy of the whole learning process, brings gain on both computational efficiency and model selection performance. The results also imply that optimization is important in learning, but learning is not just a simple optimization. .Furthermore, we investigate NFA under a semi-blind learning framework. In practice, there exist many scenarios of knowing partially either or both of the system and the input. Here, we modify Network Component Analysis (NCA) to model gene transcriptional regulation in system biology by NFA. The previous hardcut NFA algorithm is extended here as sparse BYY-NFA by considering either or both of a priori connectivity and a priori sparse constraint. Therefore, the a priori knowledge about the connection topology of the TF-gene regulatory network required by NCA is not necessary for our NFA algorithm. The sparse BYY-NFA can be further modified to get a sparse BYY-BFA algorithm, which directly models the switching patterns of latent transcription factor (TF) activities in gene regulation, e.g., whether or not a TF is activated. Mining switching patterns provides insights into exploring regulation mechanism of many biological processes.Finally, the semi-blind NFA learning is applied to identify those single nucleotide polymorphisms (SNPs) that are significantly associated with a disease or a complex trait from exome sequencing data. By encoding each exon/gene (which may contain multiple SNPs) as a vector, an NFA classifier, obtained in a supervised way on a training set, is used for prediction on a testing set. The genes are selected according to the p-values of Fisher’s exact test on the confusion tables collected from prediction results. The selected geneson a real dataset from an exome sequencing project on psoriasis are consistent in part with published results, and some of them are probably novel susceptible genes of the disease according to the validation results. (Abstract shortened by UMI.)

Bayesian Ying-Yang Learning on Orthogonal Binary Factor Analysis

Learning Binary Factor Analysis with Automatic Model Selection.

A Comparative Investigation on Model Selection in Independent Factor Analysis

Canonical Dual Approach to Binary Factor Analysis

Bayesian Ying Yang Learning (II): A New Mechanism for Model Selection and Regularization

Further Advances on Bayesian Ying-Yang Harmony Learning

Learning local factor analysis versus mixture of factor analyzers with automatic model selection

Investigation on Bayesian Ying-Yang Learning for Model Selection in Unsupervised Learning

Bayesian Ying-Yang system, best harmony learning, and five action circling

Local Factor Analysis with Automatic Model Selection and Data Smoothing Based Regularization

Advances on BYY Harmony Learning: Information Theoretic Perspective, Generalized Projection Geometry, and Independent Factor Autodetermination

Local factor analysis with automatic model selection: a comparative study and digits recognition application

Byy Harmony Learning, Model Selection, And Information Approach: Further Results

On Essential Topics of BYY Harmony Learning: Current Status, Challenging Issues, and Gene Analysis Applications

An iterative algorithm for BYY learning on Gaussian mixture with automated model selection

Bayesian Ying Yang System, Best Harmony Learning, And Gaussian Manifold Based Family

Learning non-gaussian factor analysis with different structures: comparative investigations on model selection and applications

New Advances on Bayesian Ying-Yang Learning System with Kullback and Non-Kullback Separation Functionals

Bayesian Ying-Yang System and Theory as A Unified Statistical Learning Approach (II): From Unsupervised Learning to Supervised Learning and Temporal Modeling

Parameterizations make different model selections: Empirical findings from factor analysis

A Fast Fixed-Point Byy Harmony Learning Algorithm on Gaussian Mixture with Automated Model Selection