Abstract:Reconstructing 2D freehand Ultrasound (US) frames into 3D space without using a tracker has recently seen advances with deep learning. Predicting good frame-to-frame rigid transformations is often accepted as the learning objective, especially when the ground-truth labels from spatial tracking devices are inherently rigid transformations. Motivated by a) the observed nonrigid deformation due to soft tissue motion during scanning, and b) the highly sensitive prediction of rigid transformation, this study investigates the methods and their benefits in predicting nonrigid transformations for reconstructing 3D US. We propose a novel co-optimisation algorithm for simultaneously estimating rigid transformations among US frames, supervised by ground-truth from a tracker, and a nonrigid deformation, optimised by a regularised registration network. We show that these two objectives can be either optimised using meta-learning or combined by weighting. A fast scattered data interpolation is also developed for enabling frequent reconstruction and registration of non-parallel US frames, during training. With a new data set containing over 357,000 frames in 720 scans, acquired from 60 subjects, the experiments demonstrate that, due to an expanded thus easier-to-optimise solution space, the generalisation is improved with the added deformation estimation, with respect to the rigid ground-truth. The global pixel reconstruction error (assessing accumulative prediction) is lowered from 18.48 to 16.51 mm, compared with baseline rigid-transformation-predicting methods. Using manually identified landmarks, the proposed co-optimisation also shows potentials in compensating nonrigid tissue motion at inference, which is not measurable by tracker-provided ground-truth. The code and data used in this paper are made publicly available at <a class="link-external link-https" href="https://github.com/QiLi111/NR-Rec-FUS" rel="external noopener nofollow">this https URL</a>.

Neural Implicit Surface Reconstruction of Freehand 3D Ultrasound Volume with Geometric Constraints

Automatic Spinal Ultrasound Image Segmentation and Deployment for Real-time Spine Volumetric Reconstruction

Surface Reconstruction from Sparse & Arbitrarily Oriented Contours in Freehand 3D Ultrasound.

Surface Reconstruction from Sparse and Mutually Intersected Contours for Freehand 3D Ultrasound Using Variational Method

Variational Approach To Reconstruct Surface From Sparse & Nonparallel Contours In Freehand 3d Ultrasound Imaging

Variational surface reconstruction from sparse and nonparallel contours for freehand 3D ultrasound

Surface Reconstruction from Sparse Non-Parallel Cross-Sections for Freehand 3d Ultrasound Using Variational Implicit Functions

Efficient Neural Reconstruction for Freehand 3-D Ultrasound Imaging and Visualization in Augmented Reality

Sensorless End-to-End Freehand Three-dimensional Ultrasound Reconstruction with Physics Guided Deep Learning

Development of Implicit Representation Method for Freehand 3D Ultrasound Image Reconstruction of Carotid Vessel

Quantitative Assessment of Variational Surface Reconstruction from Sparse Point Clouds in Freehand 3d Ultrasound Imaging During Image-Guided Tumor Ablation

NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction

Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction

Sensorless volumetric reconstruction of fetal brain freehand ultrasound scans with deep implicit representation

RecON: Online learning for sensorless freehand 3D ultrasound reconstruction

RoCoSDF: Row-Column Scanned Neural Signed Distance Fields for Freehand 3D Ultrasound Imaging Shape Reconstruction

Nonrigid Reconstruction of Freehand Ultrasound without a Tracker

Neural Implicit Surface Reconstruction using Imaging Sonar

Implicit Neural Representations for Breathing-compensated Volume Reconstruction in Robotic Ultrasound

Geometry-Guided Neural Implicit Surface Reconstruction

High-Fidelity Mask-free Neural Surface Reconstruction for Virtual Reality