Abstract:Reconstructing 2D freehand Ultrasound (US) frames into 3D space without using a tracker has recently seen advances with deep learning. Predicting good frame-to-frame rigid transformations is often accepted as the learning objective, especially when the ground-truth labels from spatial tracking devices are inherently rigid transformations. Motivated by a) the observed nonrigid deformation due to soft tissue motion during scanning, and b) the highly sensitive prediction of rigid transformation, this study investigates the methods and their benefits in predicting nonrigid transformations for reconstructing 3D US. We propose a novel co-optimisation algorithm for simultaneously estimating rigid transformations among US frames, supervised by ground-truth from a tracker, and a nonrigid deformation, optimised by a regularised registration network. We show that these two objectives can be either optimised using meta-learning or combined by weighting. A fast scattered data interpolation is also developed for enabling frequent reconstruction and registration of non-parallel US frames, during training. With a new data set containing over 357,000 frames in 720 scans, acquired from 60 subjects, the experiments demonstrate that, due to an expanded thus easier-to-optimise solution space, the generalisation is improved with the added deformation estimation, with respect to the rigid ground-truth. The global pixel reconstruction error (assessing accumulative prediction) is lowered from 18.48 to 16.51 mm, compared with baseline rigid-transformation-predicting methods. Using manually identified landmarks, the proposed co-optimisation also shows potentials in compensating nonrigid tissue motion at inference, which is not measurable by tracker-provided ground-truth. The code and data used in this paper are made publicly available at <a class="link-external link-https" href="https://github.com/QiLi111/NR-Rec-FUS" rel="external noopener nofollow">this https URL</a>.

RoCoSDF: Row-Column Scanned Neural Signed Distance Fields for Freehand 3D Ultrasound Imaging Shape Reconstruction

Neural Implicit Surface Reconstruction of Freehand 3D Ultrasound Volume with Geometric Constraints

RecON: Online learning for sensorless freehand 3D ultrasound reconstruction

Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction

Sensorless Freehand 3D Ultrasound Reconstruction via Deep Contextual Learning

SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images

Learning Unsigned Distance Fields from Local Shape Functions for 3D Surface Reconstruction

Nonrigid Reconstruction of Freehand Ultrasound without a Tracker

CrossSDF: 3D Reconstruction of Thin Structures From Cross-Sections

Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction

TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

Sensorless End-to-End Freehand Three-dimensional Ultrasound Reconstruction with Physics Guided Deep Learning

Unsigned Orthogonal Distance Fields: An Accurate Neural Implicit Representation for Diverse 3D Shapes

gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction

NeUDF: Leaning Neural Unsigned Distance Fields with Volume Rendering

Multi-IMU with Online Self-Consistency for Freehand 3D Ultrasound Reconstruction

Surface-Centric Modeling for High-Fidelity Generalizable Neural Surface Reconstruction

Deep Motion Network for Freehand 3D Ultrasound Reconstruction

NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies

Topology-Preserving Shape Reconstruction and Registration via Neural Diffeomorphic Flow