Abstract:Small-scale continuum robot-assisted minimally invasive surgery has received crucial attention due to its smaller incisions and high dexterity. In medical scenarios such as radiofrequency ablation and nasal/throat swab sampling, monitoring and controlling the forces applied to human tissue can help improve the safety and comfort level of the procedure. However, the tip-sensor-based force detection method can barely be deployed due to the miniature size of the continuum robot; meanwhile, the mechanical modeling-based high-precision force estimation cannot be realized on account of the continuum robots’ complex structure with high nonlinear properties. To address the high-precision force estimation challenge for further compliant control during minimally invasive interventions, a learning-based high-precision force estimation method via long short-term memory (LSTM) is proposed in this paper. On this basis, compliance control and high-precision force tracking can be further realized for small-scale continuum robot. The compliance control ensures a smooth and stable transition during the interaction between the robot and the environment, and force tracking can be utilized for maintaining or precisely controlling the force applied to the human tissue. Finally, the contact force sensing and control experiments are carried out on a small-scale continuum robot system prototype, and a demonstration using a human nasal cavity model is conducted. The results validate that the proposed LSTM neural network fits the mechanical model of the continuum robot well with the root mean square error of 3.44mN, and the control method can significantly compensate for the instantaneous impact during contact with an attenuation of 59.3 $\%$ and rapidly respond to keep the force accurately at the expected value with the mean absolute error of 2.41mN Note to Practitioners —This research is motivated by the increasing number of applications for small-scale continuum robot-assisted minimally invasive interventional surgery, such as radiofrequency ablation, biopsy, and endoscopic submucosal dissection. These surgeries require high-precision contact force sensing and control. However, due to the small size and complex structure of the continuum robot, traditional methods such as installing force sensors and mechanical modeling are not effective. Therefore, this paper utilizes a neural network to infer contact force through more accessible information about the robot, such as the tension of the actuators. This method allows for compliant force control during the dynamic contact between the small-scale continuum robot and human tissue. Experiments conducted on a human nasal cavity model demonstrate that the proposed method can improve the safety and reliability of small-scale continuum robot-assisted minimally invasive surgery.

Robust Tracking of Dexterous Continuum Robots: Fusing FBG Shape Sensing and Stereo Vision

Accurate and Real-Time 3-D Tracking for the Following Robots by Fusing Vision and Ultrasonar Information

A Skeleton and Visual Tracking Fusion Based Person-Following System for Mobile Service Robots

Beyond Traditional Driving Scenes: A Robotic-Centric Paradigm for 2D+3D Human Tracking Using Siamese Transformer Network

A Robust 6-D Pose Tracking Approach by Fusing a Multi-Camera Tracking Device and an AHRS Module

An Effective Approach to 3D Deformable Surface Tracking

Learning-Based High-Precision Force Estimation and Compliant Control for Small-Scale Continuum Robot

Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery

FBG-Based Online Learning and 3-D Shape Control of Unmodeled Continuum and Soft Robots in Unstructured Environments

Endoscope FOV Autonomous Tracking Method for Robot-Assisted Surgery Considering Pose Control, Hand–Eye Coordination, and Image Definition

Development of a flexible endoscopic robot with autonomous tracking control ability using machine vision and deep learning

Adaptive Online Learning and Robust 3-D Shape Servoing of Continuum and Soft Robots in Unstructured Environments

A Vision-Based Path Planning and Object Tracking Framework for 6-DOF Robotic Manipulator

Robust monocular 3D object pose tracking for large visual range variation in robotic manipulation via scale-adaptive region-based method

Shape Reconstruction for Continuum Robot Based on Pythagorean Hodograph–Bézier Curve With IMU and Vision Sensors

Engineering education under stress.

Shape Sensing for Continuum Robots by Capturing Passive Tendon Displacements With Image Sensors

Continuous Feature-Based Tracking of the Inner Ear for Robot-Assisted Microsurgery

Simultaneous Estimation of Shape and Force along Highly Deformable Surgical Manipulators Using Sparse FBG Measurement

Visual servo control of endoscope-holding robot based on multi-objective optimization: System modeling and instrument tracking

Shape Reconstruction of Soft Continuum Robots Via the Fusion of Local Strains and Global Poses