Abstract:I present efficient data mining algorithms for knowledge discovery on two types of emerging large-scale sequence-based scientific datasets: (1) static sequence data generated from SNP diversity arrays for genomic studies, and (2) dynamic sequence data collected in streaming and sensor network systems for environmental studies. The massive, noisy nature of the SNP arrays and the distributive, online nature of sensor network data pose challenging issues for knowledge discovery such as scalability, robustness, and efficiency. Despite the different characteristics of the SNP arrays and streaming sensor data, when viewed as sequences of ordered observations, both can be efficiently mined using algorithms based on block-wise decomposition methods. I present models and mining algorithms for inferring the genetic variation structure in genome-wide Single-Nucleotide Polymorphism (SNP) arrays. Genome-wide SNP arrays provide a comprehensive view of genome variation and serve as powerful resources for genetic and biomedical studies. Understanding the patterns of genetic variation in a population of individuals plays an important role in solving many genetics problems such as genealogy reconstruction and gene association studies. In this thesis, I propose data mining models and algorithms to efficiently infer genetic variation structure from the massive SNP panels of recombinant sequences resulting from meiotic recombination. I introduced the Minimum Segmentation Problem (MSP) to infer the segmentation structure of a single recombinant strain, as well as the Minimum Mosaic Problem (MMP) to infer the mosaic structure on a panel of recombinant strains. Both MSP and MMP estimate the ancestral polymorphism patterns exhibited in recombinant strains which provides important inputs for the subsequent association analysis. Efficient dynamic programming and graph algorithms based on block-wise decomposition are proposed which can solve MSP and MMP on genome-wide large-scale panels. I present efficient algorithms for mining massive streaming and sensor network data for observational sciences such as ecology and environmental studies. I proposed efficient algoirithms using block-wise synopsis construction to capture the data distribution online for the dynamic sequence data collected in the sensor network and streaming systems including clustering analysis and order-statistics computation, which is critical for real-time monitoring, anomaly detection, and other domain specific analysis.

Mining Seasonal Marine Microbial Pattern with Greedy Heuristic Clustering and Symmetrical Nonnegative Matrix Factorization

Earth microbial co-occurrence network reveals interconnection pattern across microbiomes

Integrative Analysis for Identifying Co-Modules of Microbe-Disease Data by Matrix Tri-Factorization With Phylogenetic Information

Microbial community pattern detection in human body habitats via ensemble clustering framework

Mldm: a New Hierarchical Bayesian Statistical Model for Sparse Microbioal Association Discovery

HONMF: integration analysis of multi-omics microbiome data via matrix factorization and hypergraph

Assembly Processes and Co-occurrence Patterns of Abundant and Rare Bacterial Community in the Eastern Indian Ocean

Biogeographic shifts in the microbial co-occurrence network features of three domains across complex environmental gradients in subtropical coastal waters

Microbial community assembly and co-occurrence relationship in sediments of the river-dominated estuary and the adjacent shelf in the wet season

How Communities of Marine Stramenopiles Varied with Environmental and Biological Variables in the Subtropical Northwestern Pacific Ocean

Towards more accurate microbial source tracking via non-negative matrix factorization (NMF)

Seasonal Dynamics of the Microbial Community in Two Full-Scale Wastewater Treatment Plants: Diversity, Composition, Phylogenetic Group Based Assembly and Co-Occurrence Pattern.

Seasonal variations of microbial communities and viral diversity in fishery-enhanced marine ranching sediments: insights into metabolic potentials and ecological interactions

Mining Functional Modules by Multiview-NMF of Phenome-Genome Association

Seasonal recurrence and modular assembly of an Arctic pelagic marine microbiome

Exploring seasonal variations, assembly dynamics, and relationships of bacterial communities in different habitats of marine ranching

Comprehensive non-black box classification of highly correlated ecological time series pairs containing many zeros: the case of gut microbiome of mice

Mining Geographic Episode Association Patterns of Abnormal Events in Global Earth Science Data

Cracking the dynamic code of the deep: Unexpected seasonal patterns of active protistan-bacterial microbiomes in the mesopelagic zone of the South China Sea

Microbial community diversity from nearshore to offshore in the East China Sea

Mining emerging massive scientific sequence data using block-wise decomposition methods