Abstract:Abstract Cardiovascular diseases are a leading cause of global mortality, and timely detection is essential to prevent their detrimental outcomes. Traditional diagnostic methods, such as segmentation, feature extraction, and subsequent heartbeat analysis, present inherent challenges. So far, the published deep learning methods have been designed for a single category, either finding abnormal signals or detecting murmur. Furthermore, most published deep learning methods used heartbeats recorded from different auscultations as similar, such that they are from the same origin. However, the flexibility in designing deep learning models made it possible to analyze heartbeats from various auscultation locations as distinct signals and categorize them in multiple labels. Therefore, this research proposed a novel multi-input-multi-output hybrid deep learning model for heartbeat analysis. This innovative approach integrated a Bidirectional Long Short Term Memory (BiLSTM) along with Dense layer as inputs and added UNet followed by Residual blocks as a backbone to find abnormal and murmured heartbeats simultaneously through multi-output architecture. The first four inputs used BiLSTM layers to extract temporal features independently from the four auscultation locations. The fifth input analyzed the patient's gender through the Dense layer and supported the model by focusing on gender while analyzing heartbeats. The UNet architecture transformed the temporal features into the required or more suitable format, and the Residual blocks managed the model's depth. Finally, all five networks combined and simultaneously classified the patient's heartbeats into two binary categories, providing information that either the signal was normal or the heartbeat had murmured. The Circor DigiScope dataset had missing features and, therefore, was unsuitable for experimentation. Thus, the dataset was modified by excluding samples lacking audio from essential auscultation sites. The proposed model, trained using the K-Fold cross-validation strategy, provided 95% accuracy and avoided overfitting on a small dataset. Therefore, the MIMO strategy provided an in-depth interpretation of cardiac signals by analyzing each signal separately through multiple perspectives. Consequently, it achieved better results and set the stage for future deep-learning algorithms in this domain. Furthermore, using less preprocessing and relying on the flexibility of deep learning algorithms underscored the model's potential to enhance diagnostic accuracy in real-world clinical scenarios.

Multimodal Foundation Models For Echocardiogram Interpretation

Vision–language foundation model for echocardiogram interpretation

EchoFM: Foundation Model for Generalizable Echocardiogram Analysis

EchoPrime: A Multi-Video View-Informed Vision-Language Model for Comprehensive Echocardiography Interpretation

EchoApex: A General-Purpose Vision Foundation Model for Echocardiography

Echo-Vision-FM: A Pre-training and Fine-tuning Framework for Echocardiogram Videos Vision Foundation Model

PanEcho: Complete AI-enabled echocardiography interpretation with multi-task deep learning

GEMTrans: A General, Echocardiography-based, Multi-Level Transformer Framework for Cardiovascular Diagnosis

Echocardiogram Foundation Model -- Application 1: Estimating Ejection Fraction

EchoFM: A Pre-training and Fine-tuning Framework for Echocardiogram Videos Vision Foundation Model

ECG-Chat: A Large ECG-Language Model for Cardiac Disease Diagnosis

HeartBeat: Towards Controllable Echocardiography Video Synthesis with Multimodal Conditions-Guided Diffusion Models

EyeCLIP: A visual-language foundation model for multi-modal ophthalmic image analysis

Multimodal Foundation Models Exploit Text to Make Medical Image Predictions

Fast and accurate classification of echocardiograms using deep learning

An Enhanced Cardiac Auscultation Approach for the detection of abnormal Heartbeat and murmur based on Multimodal Deep Learning

Using deep learning to predict cardiovascular magnetic resonance findings from echocardiography videos

BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs

SIGxCL: A Signal-Image-Graph Cross-Modal Contrastive Learning Framework for CVD Diagnosis Based on Internet of Medical Things

Foundation versus Domain-Specific Model for Cardiac Ultrasound Segmentation