Abstract:Grasping good understanding of the weight-bearing spatial structure of the spine of a human subject in a standing position is critical for the treatment of spinal disorders. Such disorders are commonly diagnosed via 2D X-ray imaging of the human subject in a standing position. However, 3D reconstruction techniques based on bi-planar X-ray imaging can enable better exploration and analysis of the spinal structure. In particular, compared to earlier deformable modeling approaches, the recently-developed deep-learning-based 3D reconstruction methods exhibit higher efficiency and generalizability. But these methods usually employ simple architectures with 2D encoders and 3D decoders. Thus, these methods have several drawbacks, namely, the existence of a semantic gap between dimensionally-inconsistent feature maps, the difficulty of jointly handling multi-view inputs, and the information source limitations for the decoding process. In order to better assist clinicians and tackle these problems, we propose a novel convolutional neural network framework, which we call BX2S-Net, to effectively achieve 3D spine reconstruction based on bi-planar X-ray images. In particular, a dimensionally-consistent encoder-decoder architecture is designed in conjunction with a dimensionality enhancement method in order to reduce the semantic gap between feature maps and achieve information fusion for multi-view inputs. A feature-guided progressive decoding process is developed on the decoder side, where a full-scale feature attention guidance (FFAG) module is introduced to efficiently aggregate image features and guide the decoding process at each level. In addition, a class augmentation method and a spatially-weighted cross-entropy loss function are used for network training with improved reconstruction quality for the vertebral edge region. The experimental results demonstrate the effectiveness of our model in reconstructing high-quality 3D spinal structures from bi-planar X-ray images. The code is available at https://github.com/NBU-CVMI/bx2s-net.

3D Reconstruction of Leg Bones from X-Ray Images using CNN-based Feature Analysis

End-To-End Convolutional Neural Network for 3D Reconstruction of Knee Bones From Bi-Planar X-Ray Images

2D-3D Reconstruction of a Femur by Single X-Ray Image Based on Deep Transfer Learning Network

2D–3D reconstruction of distal forearm bone from actual X-ray images of the wrist using convolutional neural networks

Estimating and abstracting the 3D structure of bones using neural networks on X-ray (2D) images

3D Reconstruction of 2D X-Ray Images

Reconstruction of patient-specific femurs using X-ray and sparse CT images

2D-to-3D: A Review for Computational 3D Image Reconstruction from X-ray Images

Deep-learning based 3D reconstruction of lower limb bones from biplanar radiographs for preoperative osteotomy planning

3D Articulated Skeleton Extraction Using a Single Consumer-Grade Depth Camera.

Spatial-Division Augmented Occupancy Field for Bone Shape Reconstruction from Biplanar X-Rays

Extraction and Analysis of Foot Bone Shape Features Based on Deep Learning

BX2S-Net: Learning to reconstruct 3D spinal structures from bi-planar X-ray images

3DSRNet: 3-D Spine Reconstruction Network Using 2-D Orthogonal X-Ray Images Based on Deep Learning

Automatic landmark detection and mapping for 2D/3D registration with BoneNet

A Bone Age Assessment System for Real-World X-ray Images Based on Convolutional Neural Networks.

Artificial Neural Networks for Long-Bone Fracture Detection Based on X-Ray Image Contour Features

Recognition and 3D Visualization of Human Body Parts and Bone Areas Using CT Images

3D femur model reconstruction from biplane X-ray images: a novel method based on Laplacian surface deformation

Long-Bone Fracture Detection using Artificial Neural Networks based on Contour Features of X-ray Images

Benchmarking Encoder-Decoder Architectures for Biplanar X-ray to 3D Shape Reconstruction