A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction

Fanwei Kong,Nathan Wilson,Shawn C. Shadden

DOI: https://doi.org/10.1016/j.media.2021.102222

2021-09-14

Abstract:Automated construction of surface geometries of cardiac structures from volumetric medical images is important for a number of clinical applications. While deep-learning-based approaches have demonstrated promising reconstruction precision, these approaches have mostly focused on voxel-wise segmentation followed by surface reconstruction and post-processing techniques. However, such approaches suffer from a number of limitations including disconnected regions or incorrect surface topology due to erroneous segmentation and stair-case artifacts due to limited segmentation resolution. We propose a novel deep-learning-based approach that directly predicts whole heart surface meshes from volumetric CT and MR image data. Our approach leverages a graph convolutional neural network to predict deformation on mesh vertices from a pre-defined mesh template to reconstruct multiple anatomical structures in a 3D image volume. Our method demonstrated promising performance of generating whole heart reconstructions with as good or better accuracy than prior deep-learning-based methods on both CT and MR data. Furthermore, by deforming a template mesh, our method can generate whole heart geometries with better anatomical consistency and produce high-resolution geometries from lower resolution input image data. Our method was also able to produce temporally consistent surface mesh predictions for heart motion from CT or MR cine sequences, and therefore can potentially be applied for efficiently constructing 4D whole heart dynamics. Our code and pre-trained networks are available at <a class="link-external link-https" href="https://github.com/fkong7/MeshDeformNet" rel="external noopener nofollow">this https URL</a>

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to automatically reconstruct the cardiac surface geometry directly from three - dimensional medical imaging data. Specifically, the authors point out that although current deep - learning - based methods perform well in reconstruction accuracy, most of them focus on voxel - level segmentation and then generate cardiac surface meshes through surface reconstruction and post - processing techniques. These methods have some limitations, such as discontinuous regions or surface topological errors due to incorrect segmentation, and staircase - like artifacts due to limited segmentation resolution.

To solve these problems, the paper proposes a new deep - learning - based method that can directly predict the surface mesh of the entire heart from CT and MR image data. This method uses a graph convolutional neural network to predict the deformation from a predefined mesh template to the target anatomical structure, thereby reconstructing multiple anatomical structures in the 3D image volume. This method can not only generate cardiac reconstructions with the same or better accuracy compared to previous deep - learning - based methods, but also generate high - resolution geometries with better anatomical consistency by deforming the template mesh, even when the resolution of the input image data is low. In addition, this method can also generate temporally consistent surface mesh predictions for capturing cardiac motion from CT or MR cine sequences, thereby potentially efficiently constructing a 4D whole - heart dynamic model.