Transforming Heart Chamber Imaging: Self-Supervised Learning for Whole Heart Reconstruction and Segmentation

Abdul Qayyum,Hao Xu,Brian P. Halliday,Cristobal Rodero,Christopher W. Lanyon,Richard D. Wilkinson,Steven Alexander Niederer

2024-06-10

Abstract:Automated segmentation of Cardiac Magnetic Resonance (CMR) plays a pivotal role in efficiently assessing cardiac function, offering rapid clinical evaluations that benefit both healthcare practitioners and patients. While recent research has primarily focused on delineating structures in the short-axis orientation, less attention has been given to long-axis representations, mainly due to the complex nature of structures in this orientation. Performing pixel-wise segmentation of the left ventricular (LV) myocardium and the four cardiac chambers in 2-D steady-state free precession (SSFP) cine sequences is a crucial preprocessing stage for various analyses. However, the challenge lies in the significant variability in contrast, appearance, orientation, and positioning of the heart across different patients, clinical views, scanners, and imaging protocols. Consequently, achieving fully automatic semantic segmentation in this context is notoriously challenging. In recent years, several deep learning models have been proposed to accurately quantify and diagnose cardiac pathologies. These automated tools heavily rely on the accurate segmentation of cardiac structures in magnetic resonance images (MRI). Hence, there is a need for new methods to handle such structures' geometrical and textural complexities. We proposed 2D and 3D two-stage self-supervised deep learning segmentation hybrid transformer and CNN-based architectures for 4CH whole heart segmentation. Accurate segmentation of the ventricles and atria in 4CH views is crucial for analyzing heart health and reconstructing four-chamber meshes, which are essential for estimating various parameters to assess overall heart condition. Our proposed method outperformed state-of-the-art techniques, demonstrating superior performance in this domain.

Image and Video Processing

What problem does this paper attempt to address?

This paper focuses on the automatic segmentation problem in cardiac magnetic resonance imaging (CMR), specifically for long-axis views (LAX). Automatic segmentation of the ventricles and atria is crucial for assessing cardiac function and diagnosing cardiac diseases in CMR. However, there has been less research on LAX views compared to short-axis views (SAX) due to their structural complexity.

The paper proposes a two-stage self-supervised deep learning segmentation method, combining Transformer and Convolutional Neural Network (CNN) architectures, for full-heart segmentation of the four-chamber heart (4CH). This method is important for analyzing cardiac health and constructing a four-chamber heart mesh to assess the overall cardiac condition. The researchers perform pixel-level segmentation of the left ventricular myocardium and the four cardiac chambers in 2D steady-state free precession (SSFP) cine sequences, which exhibit significant contrast, appearance, orientation, and positioning variations across different patients, clinical views, scanners, and imaging protocols.

The paper compares existing methods, including UNet-based predefined models and the use of Spatial Transformer Network (STN) to learn and estimate rigid affine transformation matrix parameters for achieving consistent standardization orientation in clinical practice. The output is then fed into a 2D secondary segmentation pipeline for further segmentation. Additionally, other research works are mentioned, such as utilizing semi-supervised learning, multitask pipelines, and statistical models to improve the segmentation of LAX views.

In the experimental part, the paper uses multiple datasets, including the TRED-HF trial dataset, CMRxmotion challenge dataset, M&Ms-2 challenge dataset, Kaggle colon dataset, and coronary artery disease (CAD) dataset, for model training and validation. The proposed model demonstrates better performance than existing techniques, especially in the segmentation of LAX four-chamber heart views, on these datasets.

In conclusion, this paper aims to address the automatic segmentation of cardiac structures in LAX views of CMR images. It proposes a novel self-supervised learning approach and validates its superiority on multiple datasets, providing more accurate image analysis tools for the diagnosis and treatment of cardiac diseases.