Abstract:To facilitate diagnosis on cardiac ultrasound (US), clinical practice has established several standard views of the heart, which serve as reference points for diagnostic measurements and define viewports from which images are acquired. Automatic view recognition involves grouping those images into classes of standard views. Although deep learning techniques have been successful in achieving this, they still struggle with fully verifying the suitability of an image for specific measurements due to factors like the correct location, pose, and potential occlusions of cardiac structures. Our approach goes beyond view classification and incorporates a 3D mesh reconstruction of the heart that enables several more downstream tasks, like segmentation and pose estimation. In this work, we explore learning 3D heart meshes via graph convolutions, using similar techniques to learn 3D meshes in natural images, such as human pose estimation. As the availability of fully annotated 3D images is limited, we generate synthetic US images from 3D meshes by training an adversarial denoising diffusion model. Experiments were conducted on synthetic and clinical cases for view recognition and structure detection. The approach yielded good performance on synthetic images and, despite being exclusively trained on synthetic data, it already showed potential when applied to clinical images. With this proof-of-concept, we aim to demonstrate the benefits of graphs to improve cardiac view recognition that can ultimately lead to better efficiency in cardiac diagnosis.

What problem does this paper attempt to address?

This paper attempts to address the problem of automatically identifying standard views in cardiac ultrasound (i.e., echocardiography) and assessing whether the images are suitable for specific diagnostic measurements. Specifically, the researchers aim to go beyond traditional view classification methods by reconstructing 3D ventricular models to achieve multiple downstream tasks such as segmentation and pose estimation. This helps to improve the efficiency and accuracy of cardiac diagnosis. ### Main Problems 1. **View Recognition**: How to automatically identify standard views in echocardiography? 2. **Image Suitability Assessment**: How to assess whether an image is suitable for specific diagnostic measurements, including whether the structure is fully visible and correctly positioned? 3. **3D Reconstruction**: How to reconstruct 3D ventricular models from 2D ultrasound images to support more complex downstream tasks? ### Solutions 1. **Graph Convolutional Network (GCN)**: Use GCN for pose regression of the 3D ventricular model to determine the position of the ultrasound plane relative to the 3D model. 2. **Multi-structure Graph**: Construct a multi-structure graph that includes all ventricles and atria for locating these structures. 3. **Diffusion Model**: Generate synthetic ultrasound images to overcome the lack of 3D annotated data. ### Experiments and Results - **Data Generation**: Generate synthetic ultrasound images and corresponding 3D annotations using the diffusion model. - **View Recognition**: The GCN trained on synthetic data performs well on the synthetic test set but shows a performance drop on clinical data. - **Structure Localization**: The GCN can accurately locate the four heart chambers, but performance on clinical data still needs improvement. ### Conclusion This study proposes a novel GCN-based method for cardiac ultrasound view recognition and image suitability assessment. Although it performs well on synthetic data, there is a domain gap in performance on clinical data. Future research will focus on addressing this issue and further optimizing the accuracy of 3D reconstruction and structure localization.

Graph Convolutional Neural Networks for Automated Echocardiography View Recognition: A Holistic Approach