Abstract:The rich information contained in biological data is often distorted by multiple interacting intrinsic or extrinsic factors. Modeling the effects of these factors is necessary to uncover the underlying true signals. However, this is challenging in real applications, because biological data usually consist of tens of thousands or millions of factors, and no reliable prior knowledge is available on how these factors exert the effect, to what degree the effect is, as well as how they interact with each other. Thus, the existing approaches rely on excessive simplification or unrealistic assumptions such as the probabilistic independence among factors. In this paper, we report the finding that after reformulating the data as a contingency tensor the problem can be well addressed by a fundamental machine learning principle, Maximum Entropy, with an extra effort of designing an efficient algorithm to solve the large-scale optimization problem. Based on the principle of maximum entropy, and by constraining the marginals of the contingency tensor using the observed values, our Conditional Multifactorial Contingency (CMC) model imposes minimum but essential assumptions about the multifactorial joint effects and leads to a conceptually simple distribution, which informs how these factors exert the effects and interact with each other. By replacing hard constraints with expected values, CMC avoids the NP-hard problem and results in a theoretically solvable convex problem. However, due to the large scale of variables and constraints, the standard convex solvers do not work. Exploring the special properties of the CMC model we developed an efficient iterative optimizer, which reduces the running time from infeasible to minutes or from days to seconds. We applied CMC to quite a few cutting-edge biological applications, including the detection of driving transcription factor, scRNA-seq normalization, cancer-associated gene identification, GO-term activity transformation, and quantification of single-cell-level similarity. CMC gained much better performance than other methods with respect to various evaluation criteria. Our source code of CMC as well as its example applications can be found at .

Uncertainty Quantification in Multivariate Mixed Models for Mass Cytometry Data

Memo: Multi-Experiment Mixture Model Analysis of Censored Data

Mixture Models for Single Cell Assays with Applications to Vaccine Studies

Modeling Single-Cell Dynamics Using Unbalanced Parameterized Monge Maps

A Dirichlet-multinomial mixed model for determining differential abundance of mutational signatures

A bayesian multivariate mixture model for high throughput spatial transcriptomics

A Tractable Probabilistic Model for Affymetrix Probe-Level Analysis Across Multiple Chips.

Meeting the Challenges of High-Dimensional Single-Cell Data Analysis in Immunology

Combining Probability and Nonprobability Samples by Using Multivariate Mass Imputation Approaches with Application to Biomedical Research

A Statistical Approach for Improving Image Quality and Cell Quantification from Imaging Mass Cytometry

Function-on-scalar quantile regression with application to mass spectrometry proteomics data

Semi-Supervised Non-Parametric Bayesian Modelling of Spatial Proteomics

Uncertainty Quantification Using Ensemble Learning and Monte Carlo Sampling for Performance Prediction and Monitoring in Cell Culture Processes

Multimodal joint deconvolution and integrative signature selection in proteomics

Uncertainty quantification in flow cytometry using a cell sorter

Mathematical Modeling and Deconvolution of Molecular Heterogeneity Identifies Novel Subpopulations in Complex Tissues.

Computation of single-cell metabolite distributions using mixture models

Unified Tumor Growth Mechanisms from Multimodel Inference and Dataset Integration

An Efficient and Principled Model to Jointly Learn the Agnostic and Multifactorial Effect in Large-Scale Biological Data

Modeling and interpretation of single-cell proteogenomic data

Quantification of total uncertainty in the physics-informed reconstruction of CVSim-6 physiology