Abstract:Background and objectiveDuring percutaneous coronary intervention procedures, generally only 2D X-ray images are provided. The consequent lack of depth perception makes it difficult for interventionists to visually estimate the pose of medical tools inside the vasculature, especially for novices. Although some automatic methods have been developed to aid interventionists, it is still a challenging task to obtain stable and accurate pose estimation. In this paper, we describe a learning-based framework for estimating the pose of the catheter distal section (CDS). The main innovation of this framework is the proposal of a coarse-to-fine fusion network (CFF-Net) which can achieve the shape and orientation estimation for the CDS.MethodsBy adopting a two-step fusion, CFF-Net progressively solves the shape and orientation ambiguities. The first step is the early fusion where the 2D projection image fuses with the shape prior before input, which makes the estimated result own a specific catheter distal shape. The second step is the late fusion where CFF-Net fuse feature maps and the orientation data from Electromagnetic (EM) sensors to confirm the overall orientation of the CDS. Finally, the estimated pose in the EM space will be obtained after we combine the estimated shape and orientation from CFF-Net with the position information from the EM sensor.ResultsThe effectiveness of CFF-Net has been verified in a simulated environment where RMSE of CFF-Net is 0.706 ± 0.121 mm. This approach was further transferred from simulation to reality using the real-world data, where RMSE of CFF-Net is 1.121 ± 0.124 mm and RMSE of the whole proposed framework is 1.577 ± 0.144 mm.ConclusionIn simulated and real-world experiments, our proposed approach has been proven to achieve high accuracy while ensuring real-time processing for estimating the pose of the CDS.

Estimating the fundamental matrix based on the end-to-end convolutional network

An end-to-end convolutional network for estimating the essential matrix

FM-Net: Deep Learning Network for the Fundamental Matrix Estimation from Biplanar Radiographs.

End-to-End Spatial Transform Face Detection and Recognition

MCNet: Rethinking the Core Ingredients for Accurate and Efficient Homography Estimation

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Method for Estimating Fundamental Matrix Based on Improved Genetic Algorithm

A Deep Learning-Based Semantic Filter for RANSAC-Based Fundamental Matrix Calculation and the ORB-SLAM System

Robust Self-calibration of Focal Lengths from the Fundamental Matrix

Correspondence-free determination of the affine fundamental matrix

RFFCE: Residual Feature Fusion and Confidence Evaluation Network for 6dof Pose Estimation.

End-To-End Face Detection and Recognition

An automatic framework for estimating the pose of the catheter distal section using a coarse-to-fine network

DenseFusion-DA2: End-to-End Pose-Estimation Network Based on RGB-D Sensors and Multi-Channel Attention Mechanisms

FDCNet: Frontend-Backend Fusion Dilated Network Through Channel-Attention Mechanism

FCNet: A Convolutional Neural Network for Arbitrary-Length Exposure Estimation

A New Approach to Robust Fundamental Matrix Estimation Using an Analytic Objective Function and Adjusted Gradient Projection

CrossFuNet: RGB and Depth Cross-Fusion Network for Hand Pose Estimation

Continuous Estimation of Human Joint Angles From sEMG Using a Multi-Feature Temporal Convolutional Attention-Based Network

FusedNet: End-to-end Mobile Robot Relocalization in Dynamic Large-scale Scene