Abstract:Identification of cancer subtypes plays an important role in revealing useful insights into disease pathogenesis and advancing personalized therapy. The recent development of high-throughput sequencing technologies has enabled the rapid collection of multi-platform genomic data (e.g., gene expression, miRNA expression, and DNA methylation) for the same set of tumor samples. Although numerous integrative clustering approaches have been developed to analyze cancer data, few of them are particularly designed to exploit both deep intrinsic statistical properties of each input modality and complex cross-modality correlations among multi-platform input data. In this paper, we propose a new machine learning model, called multimodal deep belief network (DBN), to cluster cancer patients from multi-platform observation data. In our integrative clustering framework, relationships among inherent features of each single modality are first encoded into multiple layers of hidden variables, and then a joint latent model is employed to fuse common features derived from multiple input modalities. A practical learning algorithm, called contrastive divergence (CD), is applied to infer the parameters of our multimodal DBN model in an unsupervised manner. Tests on two available cancer datasets show that our integrative data analysis approach can effectively extract a unified representation of latent features to capture both intra- and cross-modality correlations, and identify meaningful disease subtypes from multi-platform cancer data. In addition, our approach can identify key genes and miRNAs that may play distinct roles in the pathogenesis of different cancer subtypes. Among those key miRNAs, we found that the expression level of miR-29a is highly correlated with survival time in ovarian cancer patients. These results indicate that our multimodal DBN based data analysis approach may have practical applications in cancer pathogenesis studies and provide useful guidelines for personalized cancer therapy.

Multimodal Integration for Barrett's Esophagus

Multimodal Survival Ensemble Network: Integrating Genomic and Histopathological Insights for Enhanced Cancer Prognosis.

Multimodal risk prediction with physiological signals, medical images and clinical notes

Leveraging Multimodal Semantic Fusion for Gastric Cancer Screening via Hierarchical Attention Mechanism

Local-global multiple perception based deep multi-modality learning for sub-type of esophageal cancer classification

Leveraging Historical Medical Records as a Proxy via Multimodal Modeling and Visualization to Enrich Medical Diagnostic Learning

Cross-modality Attention-based Multimodal Fusion for Non-small Cell Lung Cancer (NSCLC) Patient Survival Prediction

Artificial Intelligence in the Detection of Barrett's Esophagus: A Systematic Review

MoME: Mixture of Multimodal Experts for Cancer Survival Prediction

A multimodal deep architecture for traditional Chinese medicine diagnosis

Beyond Doctors: Future Health Prediction from Multimedia and Multimodal Observations.

A multimodal machine learning model for the stratification of breast cancer risk

Integration of Multimodal Data for Breast Cancer Classification Using a Hybrid Deep Learning Method

Artificial intelligence system for the detection of Barrett's esophagus

Integrative data analysis of multi-platform cancer data with a multimodal deep learning approach

Modality-aware Mutual Learning for Multi-modal Medical Image Segmentation

Agnostic-Specific Modality Learning for Cancer Survival Prediction from Multiple Data

Multimodal Data Hybrid Fusion and Natural Language Processing for Clinical Prediction Models

Pancancer survival prediction using a deep learning architecture with multimodal representation and integration

Stone Needle: A General Multimodal Large-scale Model Framework towards Healthcare