Abstract:Recently, contrastive learning (CL) has garnered wide interest because it enables unsupervised pre-training to alleviate conventional deep learning methods’ strong reliance on artificial labels. While CL-based methods have been applied to cardiovascular disease (CVD) diagnosis with non-invasive Electrocardiogram (ECG), most of these methods are limited within the 1-dimensional signal modality and primarily focus on temporal features like amplitude and time sequence. The morphological features derived from clinically significant image-like ECGs are ignored. Furthermore, the relationships among different leads are neglected as well, describing the activities and interaction of various heart regions that are essential in CVD diagnosis and lesion localization. To address these limitations, this work proposes a novel cross-modal contrastive learning framework named SIGxCL, which represents and jointly analyzes ECGs in signal, image, and graph modalities. Crucial for CL, modality-specific transformations are introduced for ECGs in the three modalities. SIGxCL enables signal-image-graph correspondence by maximizing the agreement of the accordant cross-modal ECGs in the invariant space. Consequently, SIGxCL could capture and leverage temporal, morphological, and spatially physiological features simultaneously. Compared to random initial and conventional supervised methods, SIGxCL achieves remarkable enhancements. Considering the best performances of the existing CL-based methods, SIGxCL outperforms them by up to 4.72%, 9.41%, and 4.31% across three datasets. SIGxCL is designed to be compatible with Internet of Medical Things (IoMT) and can be deployed on resource-limited portable devices. The deployment includes pre-training, online/offline tuning, and real-time inference modules. In conclusion, SIGxCL demonstrates superior performance and provides a promising approach for real-time IoMT-based diagnosis.

CPR-CLIP: Multimodal Pre-Training for Composite Error Recognition in CPR Training

CPR-Coach: Recognizing Composite Error Actions based on Single-class Training

Towards Equitable CPR: an Interactive System for Female CPR Training

Prompt-enhanced Hierarchical Transformer Elevating Cardiopulmonary Resuscitation Instruction via Temporal Action Segmentation

A Multi-Modal Unsupervised Machine Learning Approach for Biomedical Signal Processing in CPR

A Development of a Sound Recognition-Based Cardiopulmonary Resuscitation Training System

A Deep-Learning-Based CPR Action Standardization Method

CLIP in Medical Imaging: A Comprehensive Survey

A Computed Cardiopulmonary Resuscitation Teaching System: Research and Application

RadCLIP: Enhancing Radiologic Image Analysis through Contrastive Language-Image Pre-training

Detection and Evaluation for High-Quality Cardiopulmonary Resuscitation Based on a Three-Dimensional Motion Capture System: A Feasibility Study

Advancing healthcare practice and education via data sharing: demonstrating the utility of open data by training an artificial intelligence model to assess cardiopulmonary resuscitation skills

MCPL: Multi-modal Collaborative Prompt Learning for Medical Vision-Language Model

SIGxCL: A Signal-Image-Graph Cross-Modal Contrastive Learning Framework for CVD Diagnosis Based on Internet of Medical Things

CPR Emergency Assistance Through Mixed Reality Communication

Conditional Prototype Rectification Prompt Learning

A Multilayer and Multimodal-Fusion Architecture for Simultaneous Recognition of Endovascular Manipulations and Assessment of Technical Skills

Impact of Video-Based Error Correction Learning for Cardiopulmonary Resuscitation Training: Quasi-Experimental Study

COMMA: Co-Articulated Multi-Modal Learning

Unified Medical Image-Text-Label Contrastive Learning With Continuous Prompt

MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology