Abstract:Accurately detecting tumors and estimating the depth of tumors is essential in the surgical removal of tumors. In robotic-assisted surgery, autonomous robotic palpation has the potential to provide more precise detection, tumors' depth estimation, and less intrusion when normal tissues surround tumors. In this article, by mimicking the human finger touch, we propose a tactile sensing-based deep recurrent neural network (DRNN) with long short-term memory (LSTM) architecture to improve the accuracy of the detection and depth estimation of tumors embedded in soft tissue. In the experimental setup, the hard inclusions simulate the tumors, while the phantom tissue is fabricated by silicon to simulate the soft tissue. During the experiment, the data from the force sensor and displacement of the robot palpation probe are for detection and depth estimation purposes. The collected sequential data set of the force and the displacement of the probe during one completed palpation process will go through the proposed DRNN network with deep LSTM architecture, in which the temporal dependencies of the sequential data will be captured in the cell states in the deep LSTM layers. Subsequently, the softmax classifier is adopted to determine if there is any hard inclusion exists and offer the depth estimation of the hard inclusions. Experiments based on 396 real data sets demonstrate that the detection accuracy for the testing data set is 99.2% and the depth estimation accuracy for the testing data set is 95.8%. The accuracy of the proposed method is best when comparing with other widely used methods. Note to Practitioners—The palpation of tumors motivated this article in the robot-assisted surgical systems through tactile feedback. In order to mimic the human touch on the soft tissue, this article presents a deep-learning-based approach to estimate the depth of the hard inclusions in the phan-om tissue through force information. The displacement of the palpation probe and the touch force during one palpation are recorded as data sequences to train the deep model, which aims to capture dynamics and long-term dependence of the palpation process. In this article, we made the first successful attempt to accurately estimate the depth of the hard inclusions buried at different locations of the phantom tissue using only force information. The proposed approach can work in different robot-assisted scenarios, such as master–slave robotic surgery. In the clinic applications, the force sensor will be integrated at the end-effector of the robotic manipulator. According to the specific requirements, the force sensor and the robotic manipulator might be different from those used in this article. For some applications, such as the laparoscopic interventions, the complete vertical contact tends to be difficult to obtain due to the laparoscopic port effects. The projection of the recorded force data and displacement can obtain the information in the normal direction. The future work is going to be extended to tissue environments with arbitrary surface and tumors with various shapes/depths for more complex and prospective clinical applications.

Real-Time Haptic Characterisation of Hunt-Crossley Model Based on Radial Basis Function Neural Network for Contact Environment

Error Compensation of Joint Angles Based on RBF Neural Networks

Separable Nonlinear Least Squares Algorithm for Robust Kinematic Calibration of Serial Robots.

Corrigendum to “an Optimized RBF Neural Network Based on Beetle Antennae Search Algorithm for Modeling the Static Friction in a Robotic Manipulator Joint”

Learning-Based Force Control of a Surgical Robot for Tool-Soft Tissue Interaction

Learning-Based High-Precision Force Estimation and Compliant Control for Small-Scale Continuum Robot

Depth Estimation of Hard Inclusions in Soft Tissue by Autonomous Robotic Palpation Using Deep Recurrent Neural Network

Dynamic measurement errors estimation of CMMs based on RBF neural network

Real-time solution of the forward kinematics for a parallel haptic device using a numerical approach based on neural networks

A Recurrent Convolutional Neural Network Approach for Sensorless Force Estimation in Robotic Surgery

Cross-Modal Reconstruction for Tactile Signal in Human-Robot Interaction

Deep Neural Network Approach in Robot Tool Dynamics Identification for Bilateral Teleoperation

Towards Robotised Palpation for Cancer Detection through Online Tissue Viscoelastic Characterisation with a Collaborative Robotic Arm

Adaptive Control of Robot Series Elastic Drive Joint Based on Optimized Radial Basis Function Neural Network

Hysteresis Modeling of Robotic Catheters Based on Long Short-Term Memory Network for Improved Environment Reconstruction

Towards Model-Free Tool Dynamic Identification And Calibration Using Multi-Layer Neural Network

An Iterative Learning Algorithm Based on RBF Neural Network in Upper Limb Rehabilitation Robot

A multi-resolution physics-informed recurrent neural network: formulation and application to musculoskeletal systems

An RNN-LSTM Enhanced Compact and Affordable Micro Force Sensing System for Interventional Continuum Robots with Interchangeable End-Effector Instruments

Real-time Prediction of Soft Tissue Deformations Using Data-driven Nonlinear Presurgical Simulations

Object Recognition, Dynamic Contact Simulation, Detection, and Control of the Flexible Musculoskeletal Hand Using a Recurrent Neural Network with Parametric Bias