Abstract:In the medical field, 3D ultrasound reconstruction can visualize the internal structure of patients, which is very important for doctors to carry out correct analyses and diagnoses. Furthermore, medical 3D ultrasound images have been widely used in clinical disease diagnosis because they can more intuitively display the characteristics and spatial location information of the target. The traditional way to obtain 3D ultrasonic images is to use a 3D ultrasonic probe directly. Although freehand 3D ultrasound reconstruction is still in the research stage, a lot of research has recently been conducted on the freehand ultrasound reconstruction method based on wireless ultrasonic probe. In this paper, a wireless linear array probe is used to build a freehand acousto-optic positioning 3D ultrasonic imaging system. B-scan is considered the brightness scan. It is used for producing a 2D cross-section of the eye and its orbit. This system is used to collect and construct multiple 2D B-scans datasets for experiments. According to the experimental results, a freehand 3D ultrasonic reconstruction method based on depth learning is proposed, which is called sequence prediction reconstruction based on acoustic optical localization (SPRAO). SPRAO is an ultrasound reconstruction system which cannot be put into medical clinical use now. Compared with 3D reconstruction using a 3D ultrasound probe, SPRAO not only has a controllable scanning area, but also has a low cost. SPRAO solves some of the problems in the existing algorithms. Firstly, a 60 frames per second (FPS) B-scan sequence can be synthesized using a 12 FPS wireless ultrasonic probe through 2–3 acquisitions. It not only effectively reduces the requirement for the output frame rate of the ultrasonic probe, but also increases the moving speed of the wireless probe. Secondly, SPRAO analyzes the B-scans through speckle decorrelation to calibrate the acousto-optic auxiliary positioning information, while other algorithms have no solution to the cumulative error of the external auxiliary positioning device. Finally, long short-term memory (LSTM) is used to predict the spatial position and attitude of B-scans, and the calculation of pose deviation and speckle decorrelation is integrated into a 3D convolutional neural network (3DCNN). Prepare for real-time 3D reconstruction under the premise of accurate spatial pose of B-scans. At the end of this paper, SPRAO is compared with linear motion, IMU, speckle decorrelation, CNN and other methods. From the experimental results, it can be observed that the spatial pose deviation of B-scans output using SPRAO is the best of these methods.

3D Reconstruction and Augmented Reality Display of Bone Structure Based on Ultrasound Segmentation with Feature-Reweighted U-Net

Automatic Spinal Ultrasound Image Segmentation and Deployment for Real-time Spine Volumetric Reconstruction

Enabling Augmented Segmentation and Registration in Ultrasound-Guided Spinal Surgery via Realistic Ultrasound Synthesis from Diagnostic CT Volume

Real-time ultrasound AR 3D visualization toward better topological structure perception for hepatobiliary surgery

Three-dimensional ultrasound image reconstruction based on 3D-ResNet in the musculoskeletal system using a 1D probe: ex vivo and in vivo feasibility studies

Acoustic 3D Shape Reconstruction of Zygomatic Bone for Non-Invasive Spatial Registration

Annotation-guided encoder-decoder network for bone extraction in ultrasound-assisted orthopedic surgery

An efficient end-to-end CNN for segmentation of bone surfaces from ultrasound

Augmented Reality Navigation with Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery

High-precision Three-dimensional Space See-through Surgical Navigation

Real-Time 3D Ultrasound Imaging System Based on a Hybrid Reconstruction Algorithm

Augmented reality presentation system of skeleton image based on biomedical features

Efficient Neural Reconstruction for Freehand 3-D Ultrasound Imaging and Visualization in Augmented Reality

Spatial-Division Augmented Occupancy Field for Bone Shape Reconstruction from Biplanar X-Rays

Class-Aware Cartilage Segmentation for Autonomous US-CT Registration in Robotic Intercostal Ultrasound Imaging

RecON: Online learning for sensorless freehand 3D ultrasound reconstruction

A Freehand 3D Ultrasound Reconstruction Method Based on Deep Learning

Augmented Reality Navigation for Minimally Invasive Knee Surgery Using Enhanced Arthroscopy.

Tissue Structure Updating for in Situ Augmented Reality Navigation Using Calibrated Ultrasound and Two-Level Surface Warping

Application and Exploration of Mixed Reality Technology in Accurate Operation of Bone and Soft Tissue Tumors

Bone Feature Segmentation in Ultrasound Spine Image with Robustness to Speckle and Regular Occlusion Noise