Abstract:Background: Echocardiograms provide vital insights into cardiac health, but their complex, multi-dimensional data presents challenges for analysis and interpretation. Current deep learning models for echocardiogram analysis often rely on supervised training, limiting their generalizability and robustness across datasets and clinical environments. Objective: To develop and evaluate EchoVisionFM (Echocardiogram video Vision Foundation Model), a self-supervised video learning framework designed to pre-train a video encoder on large-scale, unlabeled echocardiogram data. EchoVisionFM aims to produce robust and transferrable spatiotemporal representations, improving downstream performance across diverse echocardiogram datasets and clinical conditions. Methods: Our framework employs Echo-VideoMAE, an autoencoder-based video transformer that compresses and reconstructs echocardiogram video data by masking non-overlapping video patches and leveraging a ViT encoder-decoder structure. For enhanced representation, we introduce STFF-Net, a Spatio Temporal Feature Fusion Network, to integrate spatial and temporal features from the manifold representations. We pre-trained EchoVisionFM using the MIMIC-IV-ECHO dataset and fine-tuned it on the EchoNet-Dynamic dataset for downstream tasks, including classification and regression of key cardiac parameters. Results: EchoVisionFM demonstrated superior performance in classifying left ventricular ejection fraction (LVEF), achieving an accuracy of 89.12%, an F1 score of 0.9323, and an AUC of 0.9364. In regression tasks, EchoVisionFM outperformed state-of-the-art models, with LVEF prediction reaching a mean absolute error (MAE) of 4.18% and an R2 of 0.8022. The model also showed significant improvements in estimating end-systolic and end-diastolic volumes, with R2 values of 0.8006 and 0.7296, respectively. Incorporating STFF-Net led to further performance gains across tasks. Conclusion: Our results indicate that large-scale self-supervised pre-training on echocardiogram videos enables the extraction of transferable and clinically relevant features, outperforming traditional CNN-based methods. The EchoVisionFM framework, particularly with STFF-Net, enhances the extraction of spatiotemporal features, improving the predictive accuracy for various cardiac parameters. EchoVisionFM offers a powerful, scalable approach for echocardiogram analysis, with potential applications in clinical diagnostics and research.

SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals

SiamQuality: a ConvNet-based foundation model for photoplethysmography signals

Foundation Models for ECG: Leveraging Hybrid Self-Supervised Learning for Advanced Cardiac Diagnostics

Signal Quality Assessment of PPG Signals using STFT Time-Frequency Spectra and Deep Learning Approaches

ECG Signal Reconstruction Based on Facial Videos Via Combined Explicit and Implicit Supervision.

EchoFM: Foundation Model for Generalizable Echocardiogram Analysis

Large-scale Training of Foundation Models for Wearable Biosignals

Promoting cross-modal representations to improve multimodal foundation models for physiological signals

Imaging foundation model for universal enhancement of non-ideal measurement CT

Image Quality Assessment Using Deep Convolutional Networks

Specialized Foundation Models Struggle to Beat Supervised Baselines

Assessing Foundation Models' Transferability to Physiological Signals in Precision Medicine

On the Foundation Model for Cardiac MRI Reconstruction

Echo-Vision-FM: A Pre-training and Fine-tuning Framework for Echocardiogram Videos Vision Foundation Model

PSC-Net: Integration of Convolutional Neural Networks and Transformers for Physiological Signal Classification

Foundation Models in Electrocardiogram: A Review

Low-Dimensional Depth Local Dual-View Features Embedded Transformer for Electrocardiogram Signal Quality Assessment.

Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals

Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train

Deep convolutional neural network-based signal quality assessment for photoplethysmogram