Abstract:Grasping good understanding of the weight-bearing spatial structure of the spine of a human subject in a standing position is critical for the treatment of spinal disorders. Such disorders are commonly diagnosed via 2D X-ray imaging of the human subject in a standing position. However, 3D reconstruction techniques based on bi-planar X-ray imaging can enable better exploration and analysis of the spinal structure. In particular, compared to earlier deformable modeling approaches, the recently-developed deep-learning-based 3D reconstruction methods exhibit higher efficiency and generalizability. But these methods usually employ simple architectures with 2D encoders and 3D decoders. Thus, these methods have several drawbacks, namely, the existence of a semantic gap between dimensionally-inconsistent feature maps, the difficulty of jointly handling multi-view inputs, and the information source limitations for the decoding process. In order to better assist clinicians and tackle these problems, we propose a novel convolutional neural network framework, which we call BX2S-Net, to effectively achieve 3D spine reconstruction based on bi-planar X-ray images. In particular, a dimensionally-consistent encoder-decoder architecture is designed in conjunction with a dimensionality enhancement method in order to reduce the semantic gap between feature maps and achieve information fusion for multi-view inputs. A feature-guided progressive decoding process is developed on the decoder side, where a full-scale feature attention guidance (FFAG) module is introduced to efficiently aggregate image features and guide the decoding process at each level. In addition, a class augmentation method and a spatially-weighted cross-entropy loss function are used for network training with improved reconstruction quality for the vertebral edge region. The experimental results demonstrate the effectiveness of our model in reconstructing high-quality 3D spinal structures from bi-planar X-ray images. The code is available at https://github.com/NBU-CVMI/bx2s-net.

Multi-View Fusion Convolutional Neural Network for Automatic Landmark Location on Spinal X-Rays.

Deep Fusion Network Based Sparse View CT Reconstructions for Clinical Diagnostic Scanners

Multi-input Adaptive Neural Network for Automatic Detection of Cervical Vertebral Landmarks on X-rays.

Accurate Scoliosis Vertebral Landmark Localization on X-ray Images Via Shape-constrained Multi-stage Cascaded CNNs.

Part Affinity Fields and CoordConv for Detecting Landmarks of Lumbar Vertebrae and Sacrum in X-ray Images

A Landmark Estimation and Correction Network for Automated Measurement of Sagittal Spinal Parameters

The Improved Faster-Rcnn for Spinal Fracture Lesions Detection.

Automatic Localization of Landmarks in Craniomaxillofacial CBCT Images Using a Local Attention-Based Graph Convolution Network.

Automated vertebral landmarks and spinal curvature estimation using non-directional part affinity fields

Multi-Scale Semantic Fusion of a Large Receptive Field for Irregular Pelvic X-Ray Landmark Detection

Anatomical Prior-Based Vertebral Landmark Detection for Spinal Disorder Diagnosis

Landmark detection in Cardiac Magnetic Resonance Imaging Using A Convolutional Neural Network

Multi-modal vertebrae recognition using Transformed Deep Convolution Network

Multi-Resolution Fusion for Fully Automatic Cephalometric Landmark Detection

BX2S-Net: Learning to reconstruct 3D spinal structures from bi-planar X-ray images

Exploring Neighbor Spatial Relationships for Enhanced Lumbar Vertebrae Detection in X-ray Images

Spinal Lesions Classification and Localization with ACAT-Net from X-ray Images

Fast and Robust Detection of Anatomical Landmarks Using Cascaded 3D Convolutional Networks Guided by Linear Square Regression.

Multi-modality Hierarchical Fusion Network for Lumbar Spine Segmentation with Magnetic Resonance Images

Local Graph Fusion of Multi-view MR Images for Knee Osteoarthritis Diagnosis

Less is More: Simultaneous View Classification and Landmark Detection for Abdominal Ultrasound Images