Abstract:Learning manipulation skills from open surgery provides more flexible access to the organ targets in the abdomen cavity and this could make the surgical robot working in a highly intelligent and friendly manner. Teaching by demonstration (TbD) is capable of transferring the manipulation skills from human to humanoid robots by employing active learning of multiple demonstrated tasks. This work aims to transfer motion skills from multiple human demonstrations in open surgery to robot manipulators in robot-assisted minimally invasive surgery (RA-MIS) by using TbD. However, the kinematic constraint should be respected during the performing of the learned skills by using a robot for minimally invasive surgery. In this article, we propose a novel methodology by integrating the cognitive learning techniques and the developed control techniques, allowing the robot to be highly intelligent to learn senior surgeons' skills and to perform the learned surgical operations in semiautonomous surgery in the future. Finally, experiments are performed to verify the efficiency of the proposed strategy, and the results demonstrate the ability of the system to transfer human manipulation skills to a robot in RA-MIS and also shows that the remote center of motion (RCM) constraint can be guaranteed simultaneously. Note to Practitioners-This article is inspired by limited access to the manipulation of laparoscopic surgery under a kinematic constraint at the point of incision. Current commercial surgical robots are mostly operated by teleoperation, which is representing less autonomy on surgery. Assisting and enhancing the surgeon's performance by increasing the autonomy of surgical robots has fundamental importance. The technique of teaching by demonstration (TbD) is capable of transferring the manipulation skills from human to humanoid robots by employing active learning of multiple demonstrated tasks. With the improved ability to interact with humans, such as flexibility and compliance, the new generation of serial robots becomes more and more popular in nonclinical research. Thus, advanced control strategies are required by integrating cognitive functions and learning techniques into the processes of surgical operation between robots, surgeon, and minimally invasive surgery (MIS). In this article, we propose a novel methodology to model the manipulation skill from multiple demonstrations and execute the learned operations in robot-assisted minimally invasive surgery (RA-MIS) by using a decoupled controller to respect the remote center of motion (RCM) constraint exploiting the redundancy of the robot. The developed control scheme has the following functionalities: 1) it enables the 3-D manipulation skill modeling after multiple demonstrations of the surgical tasks in open surgery by integrating dynamic time warping (DTW) and Gaussian mixture model (GMM)-based dynamic movement primitive (DMP) and 2) it maintains the RCM constraint in a smaller safe area while performing the learned operation in RA-MIS. The developed control strategy can also be potentially used in other industrial applications with a similar scenario.

Robotic Constrained Imitation Learning for the Peg Transfer Task in Fundamentals of Laparoscopic Surgery

Learning-Based Force Control of a Surgical Robot for Tool-Soft Tissue Interaction

3D Perception based Imitation Learning under Limited Demonstration for Laparoscope Control in Robotic Surgery

Learning surgical skills under the RCM constraint from demonstrations in robot-assisted minimally invasive surgery

Learning Laparoscope Actions via Video Features for Proactive Robotic Field-of-View Control

Automating Surgical Peg Transfer: Calibration with Deep Learning Can Exceed Speed, Accuracy, and Consistency of Humans

Gaze gesture based human robot interaction for laparoscopic surgery

Vision-Based Real-Time Tracking of Surgical Instruments in Robot-Assisted Laparoscopic Surgery

Toward Teaching by Demonstration for Robot-Assisted Minimally Invasive Surgery

Towards Human-Robot Collaborative Surgery: Trajectory and Strategy Learning in Bimanual Peg Transfer

Laparoscope Arm Automatic Positioning for Robot-Assisted Surgery Based on Reinforcement Learning

Gaze Contingent Cartesian Control of a Robotic Arm for Laparoscopic Surgery.

Data-driven Holistic Framework for Automated Laparoscope Optimal View Control with Learning-based Depth Perception

Toward a Surgeon-in-the-Loop Ophthalmic Robotic Apprentice using Reinforcement and Imitation Learning

Automatic Tracking of Surgical Instruments with A Continuum Laparoscope Using Data-driven Control in Robotic Surgery

Applying Depth-Sensing to Automated Surgical Manipulation with a da Vinci Robot

Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks

Reinforcement Learning Based Manipulation Skill Transferring for Robot-assisted Minimally Invasive Surgery.

Vision-based hand-eye calibration for robot-assisted minimally invasive surgery

An Adaptive Auxiliary Framework for Teleoperated Laparoscopic Surgery

SGR-AutoLap: Surgical Gesture Recognition-Based Autonomous Laparoscope Control for Human-Robot Shared Control in Semi-Autonomous Minimally Invasive Surgical Robot