Abstract:Purpose. Precise placement of needles is a challenge in a number of clinical applications such as brachytherapy or biopsy. Forces acting at the needle cause tissue deformation and needle deflection which in turn may lead to misplacement or injury. Hence, a number of approaches to estimate the forces at the needle have been proposed. Yet, integrating sensors into the needle tip is challenging and a careful calibration is required to obtain good force estimates. Methods. We describe a fiber-optical needle tip force sensor design using a single OCT fiber for measurement. The fiber images the deformation of an epoxy layer placed below the needle tip which results in a stream of 1D depth profiles. We study different deep learning approaches to facilitate calibration between this spatio-temporal image data and the related forces. In particular, we propose a novel convGRU-CNN architecture for simultaneous spatial and temporal data processing. Results. The needle can be adapted to different operating ranges by changing the stiffness of the epoxy layer. Likewise, calibration can be adapted by training the deep learning models. Our novel convGRU-CNN architecture results in the lowest mean absolute error of 1.59 +- 1.3 mN and a cross-correlation coefficient of 0.9997, and clearly outperforms the other methods. Ex vivo experiments in human prostate tissue demonstrate the needle's application. Conclusions. Our OCT-based fiber-optical sensor presents a viable alternative for needle tip force estimation. The results indicate that the rich spatio-temporal information included in the stream of images showing the deformation throughout the epoxy layer can be effectively used by deep learning models. Particularly, we demonstrate that the convGRU-CNN architecture performs favorably, making it a promising approach for other spatio-temporal learning problems.

What problem does this paper attempt to address?

This paper attempts to solve the problem of how to accurately estimate the force on the needle tip during the needle insertion process. Specifically, needle insertion is a challenge in clinical applications such as brachytherapy or biopsy. The force acting on the needle can cause tissue deformation and needle deflection, which may lead to misalignment or damage. Therefore, accurate estimation of the force at the needle tip is crucial for ensuring the safety and effectiveness of the operation. ### Research Background 1. **Problem Description**: - In minimally invasive surgeries (such as biopsy, neurosurgery, or brachytherapy), needle insertion is often used to minimize tissue damage. - Accurate needle placement requires the use of needle - guiding, image - guiding, and force - estimation techniques. - In particular, accurate measurement of the force at the needle tip helps to track the interaction between the needle and the tissue, detect potential tissue rupture, and generate tactile and visual feedback. 2. **Limitations of Existing Methods**: - External sensors cannot effectively distinguish between the frictional force on the needle shaft and the actual force at the needle tip. - Although integrating sensors at the needle tip can improve accuracy, manufacturing and calibration are very complicated, especially for mechanical force sensors. - Although fiber - optic sensors have small size, biocompatibility, and MRI compatibility, signal processing is still difficult in different temperature ranges and under lateral forces. ### Solutions Proposed in the Paper 1. **New Design**: - A fiber - optic needle - tip force - sensor design based on optical coherence tomography (OCT) is proposed. - A single OCT fiber is used to measure the deformation of the epoxy resin layer, generating a series of 1D depth profiles (A - Scan), thereby inferring the force at the needle tip. 2. **Deep - Learning Model**: - To extract useful information from these spatio - temporal data, a novel convolutional gated recurrent unit - convolutional neural network (convGRU - CNN) architecture is proposed. - This architecture can process spatial and temporal information simultaneously, significantly outperforming other methods, achieving a minimum mean absolute error (MAE) of \(1.59\pm1.30\) mN and an extremely high cross - correlation coefficient (CC) of 0.9997. ### Experimental Verification 1. **Experimental Setup**: - By changing the stiffness of the epoxy resin layer, different operating ranges are adapted. - Experiments are carried out using ex - vivo human prostate tissue to verify the practical application feasibility of this method. 2. **Results**: - The novel convGRU - CNN architecture performs best in all tests, significantly outperforming other methods. - The experimental results show that this method can effectively capture the changes in the needle - tip force, especially in soft tissues. ### Conclusion The paper proposes a new fiber - optic needle - tip force - sensor design based on OCT and develops a novel convGRU - CNN architecture to process spatio - temporal data. The experimental results show that this method has high accuracy in needle - tip force estimation and is suitable for a variety of clinical application scenarios.

Spatio-Temporal Deep Learning Models for Tip Force Estimation During Needle Insertion

Needle Tip Force Estimation using an OCT Fiber and a Fused convGRU-CNN Architecture

Needle tip force estimation by deep learning from raw spectral OCT data

Needle Tip Tracking Based on Optical Imaging and AI

Medical needle tip tracking based on Optical Imaging and AI

Needle Detection and Localisation for Robot‐assisted Subretinal Injection Using Deep Learning

Data-Driven 3-D Tactile Cues With Intermediate Soft Interfaces Toward Training Needle Insertions

Hybrid Deep Learning and Model-Based Needle Shape Prediction

Needle tracking in low-resolution ultrasound volumes using deep learning

Optical Fiber-Based Needle Shape Sensing in Real Tissue: Single Core vs. Multicore Approaches

Exploring the Needle Tip Interaction Force with Retinal Tissue Deformation in Vitreoretinal Surgery

Three-Dimensional Force Sensor Based on Fiber Bragg Grating for Medical Puncture Robot

Epidural anesthesia needle guidance by forward-view endoscopic optical coherence tomography and deep learning

Collaborative Robotic Biopsy with Trajectory Guidance and Needle Tip Force Feedback

Bevel-Tip Needle Deflection Modeling, Simulation, and Validation in Multi-Layer Tissues

Toward FBG-Sensorized Needle Shape Prediction in Tissue Insertions

A Novel Full Prediction Model of 3D Needle Insertion Procedures.

Force Estimation from OCT Volumes using 3D CNNs

Needle Tip Pose Estimation for Ultrasound-Guided Steerable Flexible Needle With a Complicated Trajectory in Soft Tissue

Tactile needle probe for minimally invasive tissue identification during epidural space insertion

Identification of tissue types and boundaries with a fiber optic force sensor