FetusMap: Fetal Pose Estimation in 3D Ultrasound

Xin Yang,Wenlong Shi,Haoran Dou,Jikuan Qian,Yi Wang,Wufeng Xue,Shengli Li,Dong Ni,Pheng-Ann Heng

2024-03-03

Abstract:The 3D ultrasound (US) entrance inspires a multitude of automated prenatal examinations. However, studies about the structuralized description of the whole fetus in 3D US are still rare. In this paper, we propose to estimate the 3D pose of fetus in US volumes to facilitate its quantitative analyses in global and local scales. Given the great challenges in 3D US, including the high volume dimension, poor image quality, symmetric ambiguity in anatomical structures and large variations of fetal pose, our contribution is three-fold. (i) This is the first work about 3D pose estimation of fetus in the literature. We aim to extract the skeleton of whole fetus and assign different segments/joints with correct torso/limb labels. (ii) We propose a self-supervised learning (SSL) framework to finetune the deep network to form visually plausible pose predictions. Specifically, we leverage the landmark-based registration to effectively encode case-adaptive anatomical priors and generate evolving label proxy for supervision. (iii) To enable our 3D network perceive better contextual cues with higher resolution input under limited computing resource, we further adopt the gradient check-pointing (GCP) strategy to save GPU memory and improve the prediction. Extensively validated on a large 3D US dataset, our method tackles varying fetal poses and achieves promising results. 3D pose estimation of fetus has potentials in serving as a map to provide navigation for many advanced studies.

Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing

What problem does this paper attempt to address?

The paper primarily focuses on addressing the problem of fetal pose estimation in three-dimensional ultrasound (3D US) images. Specifically, the research team proposes a new method called FetusMap, which aims to extract the entire fetal skeleton and correctly label different parts of the trunk and limbs to achieve quantitative analysis on both global and local scales of the fetus.

The main contributions of the paper can be summarized as follows:

1. **First proposed 3D fetal pose estimation task**: This is the first study in the literature to perform 3D pose estimation of the fetus in 3D ultrasound images. The goal is to locate 16 key points on the entire fetal body and extract the overall skeletal structure of the fetus.

2. **Self-supervised learning framework**: To overcome challenges such as poor quality of 3D ultrasound images and significant variations in fetal poses, the researchers proposed a self-supervised learning (SSL) framework to fine-tune deep neural networks, enabling them to generate visually plausible pose predictions. By utilizing keypoint-based registration techniques, the framework effectively encodes anatomical priors adapted to each case and generates evolving label proxies as supervision signals.

3. **Adoption of gradient checkpointing strategy**: To enable 3D deep networks to handle higher resolution input data with limited computational resources, the researchers adopted the Gradient Check-pointing (GCP) strategy to save GPU memory. This helps the network better capture contextual information, thereby improving prediction accuracy.

Through extensive validation on a large 3D ultrasound dataset, the method successfully addresses the variations in fetal poses and achieves encouraging results. Additionally, 3D fetal pose estimation is expected to serve as a map, providing navigational support for many advanced prenatal studies.