Abstract:Mesh reconstruction of the cardiac anatomy from medical images is useful for shape and motion measurements and biophysics simulations to facilitate the assessment of cardiac function and health. However, 3D medical images are often acquired as 2D slices that are sparsely sampled and noisy, and mesh reconstruction on such data is a challenging task. Traditional voxel-based approaches rely on pre- and post-processing that compromises image fidelity, while mesh-level deep learning approaches require mesh annotations that are difficult to get. Therefore, direct cross-domain supervision from 2D images to meshes is a key technique for advancing 3D learning in medical imaging, but it has not been well-developed. While there have been attempts to approximate the optimized meshes' slicing, few existing methods directly use 2D slices to supervise mesh reconstruction in a differentiable manner. Here, we propose a novel explicit differentiable voxelization and slicing (DVS) algorithm that allows gradient backpropagation to a mesh from its slices, facilitating refined mesh optimization directly supervised by the losses defined on 2D images. Further, we propose an innovative framework for extracting patient-specific left ventricle (LV) meshes from medical images by coupling DVS with a graph harmonic deformation (GHD) mesh morphing descriptor of cardiac shape that naturally preserves mesh quality and smoothness during optimization. Experimental results demonstrate that our method achieves state-of-the-art performance in cardiac mesh reconstruction tasks from CT and MRI, with an overall Dice score of 90% on multi-datasets, outperforming existing approaches. The proposed method can further quantify clinically useful parameters such as ejection fraction and global myocardial strains, closely matching the ground truth and surpassing the traditional voxel-based approach in sparse images.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the challenges of reconstructing 3D cardiac meshes from medical images. Specifically, the paper focuses on the following issues: 1. **Sparse Sampling and Noisy Data**: 3D medical images are obtained through sparsely sampled 2D slices, which often have large gaps and contain noise, making 3D mesh reconstruction very difficult. 2. **Limitations of Traditional Methods**: - **Voxelization Methods**: Traditional voxelization methods lose image fidelity during preprocessing and postprocessing. - **Grid-based Deep Learning Methods**: These methods require a large amount of annotated 3D mesh data, which is difficult to obtain. - **Marching Cubes Algorithm**: Although widely used, it produces staircase artifacts, uncontrollable topology, a high number of polygons, and poor smoothness, and it lacks differentiability, making it incompatible with deep learning frameworks. 3. **Lack of Cross-domain Supervision**: There is currently a lack of a differentiable supervision method that directly maps 2D images to 3D meshes, limiting the development of 3D learning in medical imaging. ### Solutions To address the above issues, the paper proposes two main technical contributions: 1. **Differentiable Voxelization and Slicing (DVS) Algorithm**: - **Direct Global Supervision**: The DVS algorithm allows direct supervision of 3D mesh reconstruction from 2D images by optimizing the mesh through backpropagation of gradients. - **Improved Convergence and Accuracy**: Compared to local distance supervision methods and indirect supervision methods, DVS improves the convergence and accuracy of reconstruction from sparsely sampled images. 2. **Graph Harmonic Deformation (GHD) Algorithm**: - **Global Deformation**: The GHD algorithm maintains different levels of geometric information by describing mesh deformation as surface Fourier waves, naturally preserving mesh quality and smoothness. - **Robust Performance**: GHD provides accurate and efficient mesh reconstruction even in sparsely sampled medical images. ### Framework By combining DVS and GHD, the paper proposes a new differentiable cardiac mesh reconstruction framework with the following features: - **Flexibility**: Can incorporate various differentiable operators as auxiliary constraints. - **No Preprocessing and Postprocessing**: Does not require additional smoothing regularization or pre-training. - **High Performance**: Achieves state-of-the-art performance on different datasets (including CT and MRI). ### Experimental Results Experimental results show that the proposed method achieves an overall Dice score of 90% when fitting multiple sparse datasets and can quantify clinically useful parameters such as ejection fraction and global myocardial strain, closely matching the true values and outperforming traditional voxelization methods. In summary, by proposing the DVS and GHD algorithms, this paper addresses the key issues of reconstructing high-quality cardiac meshes from sparsely sampled and noisy data, providing a new technical pathway for 3D learning in medical imaging.

Explicit Differentiable Slicing and Global Deformation for Cardiac Mesh Reconstruction

Image-based Meshing of Patient-Specific Data:Converting Medical Scans into Highly Accurate Computational Models

A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction

Slice-Mask Based 3D Cardiac Shape Reconstruction from CT Volume

Simultaneous Recovery of Left Ventricular Shape and Motion Using Meshfree Particle Framework

CNN-based Cardiac Motion Extraction to Generate Deformable Geometric Left Ventricle Myocardial Models from Cine MRI

Interpretable cardiac anatomy modeling using variational mesh autoencoders

Patient-specific Heart Geometry Modeling for Solid Biomechanics using Deep Learning

A Meshfree Representation for Cardiac Medical Image Computing.

Learning Whole Heart Mesh Generation From Patient Images For Computational Simulations

DVasMesh: Deep Structured Mesh Reconstruction from Vascular Images for Dynamics Modeling of Vessels

A Simultaneous Framework for Recovering Three Dimensional Shape and Nonrigid Motion from Cardiac Image Sequences

Gradient Field Based Inhomogeneous Volumetric Mesh Deformation for Maxillofacial Surgery Simulation.

Reconstruction and completion of high-resolution 3D cardiac shapes using anisotropic CMRI segmentations and continuous implicit neural representations

Deep Atlas Network for Efficient 3D Left Ventricle Segmentation on Echocardiography

4D Myocardium Reconstruction with Decoupled Motion and Shape Model

DMCVR: Morphology-Guided Diffusion Model for 3D Cardiac Volume Reconstruction

Whole Heart Mesh Generation For Image-Based Computational Simulations By Learning Free-From Deformations

Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds

Automated analysis of 3D-echocardiography using spatially registered patient-specific CMR meshes

3D Motion Modeling and Reconstruction of Left Ventricle Wall in Cardiac MRI