Abstract:Electronic health record (EHR) systems contain a wealth of multimodal clinical data including structured data like clinical codes and unstructured data such as clinical notes. However, many existing EHR-focused studies has traditionally either concentrated on an individual modality or merged different modalities in a rather rudimentary fashion. This approach often results in the perception of structured and unstructured data as separate entities, neglecting the inherent synergy between them. Specifically, the two important modalities contain clinically relevant, inextricably linked and complementary health information. A more complete picture of a patient's medical history is captured by the joint analysis of the two modalities of data. Despite the great success of multimodal contrastive learning on vision-language, its potential remains under-explored in the realm of multimodal EHR, particularly in terms of its theoretical understanding. To accommodate the statistical analysis of multimodal EHR data, in this paper, we propose a novel multimodal feature embedding generative model and design a multimodal contrastive loss to obtain the multimodal EHR feature representation. Our theoretical analysis demonstrates the effectiveness of multimodal learning compared to single-modality learning and connects the solution of the loss function to the singular value decomposition of a pointwise mutual information matrix. This connection paves the way for a privacy-preserving algorithm tailored for multimodal EHR feature representation learning. Simulation studies show that the proposed algorithm performs well under a variety of configurations. We further validate the clinical utility of the proposed algorithm in real-world EHR data.

Quantum Multimodal Contrastive Learning Framework

Quantum Ensemble Learning with a Programmable Superconducting Processor

Contrastive Quant

Quantum Visual Feature Encoding Revisited

A Unified Framework for Quantum Supervised Learning

Multimodal Quantum Natural Language Processing: A Novel Framework for using Quantum Methods to Analyse Real Data

Contrastive Quant: Quantization Makes Stronger Contrastive Learning

Quantum-Brain: Quantum-Inspired Neural Network Approach to Vision-Brain Understanding

Contrastive Learning on Multimodal Analysis of Electronic Health Records

Integrated Encoding and Quantization to Enhance Quanvolutional Neural Networks

Quantum Curriculum Learning

Quantum Data Encoding: A Comparative Analysis of Classical-to-Quantum Mapping Techniques and Their Impact on Machine Learning Accuracy

QEEGNet: Quantum Machine Learning for Enhanced Electroencephalography Encoding

Multimodal deep representation learning for quantum cross-platform verification

Quantum Hamiltonian Embedding of Images for Data Reuploading Classifiers

Hierarchical graph contrastive learning framework based on quantum neural networks for sentiment analysis

Multimodal Structure-Aware Quantum Data Processing

A quantum-search-based multi-classifier for image recognition

Understanding the effects of data encoding on quantum-classical convolutional neural networks

MAQA: A Quantum Framework for Supervised Learning