Abstract:Controller design for soft robots is challenging due to nonlinear deformation and high degrees of freedom of flexible material. The data-driven approach is a promising solution to the controller design problem for soft robots. However, the existing data-driven controller design methods for soft robots suffer from two drawbacks: (i) they require excessively long training time, and (ii) they may result in potentially inefficient controllers. This paper addresses these issues by developing two memory-based controllers for soft robots that can be trained in a data-driven fashion: the finite memory controller (FMC) approach and the long short-term memory (LSTM) based approach. An FMC stores the tracking errors at different time instances and computes the actuation signal according to a weighted sum of the stored tracking errors. We develop three reinforcement learning algorithms for computing the optimal weights of an FMC using the Q-learning, soft actor-critic, and deterministic policy gradient (DDPG) methods. An LSTM-based controller is composed of an LSTM network where the inputs of the network are the robot's desired configuration and current configuration. The LSTM network computes the required actuation signal for the soft robot to follow the desired configuration. We study the performance of the proposed approaches in controlling a soft finger where, as benchmarks, we use the existing reinforcement learning (RL) based controllers and proportional-integral-derivative (PID) controllers. Our numerical results show that the training time of the proposed memory-based controllers is significantly shorter than that of the classical RL-based controllers. Moreover, the proposed controllers achieve a smaller tracking error compared with the classical RL algorithms and the PID controller.

A Q-learning Control Method for a Soft Robotic Arm Utilizing Training Data from a Rough Simulator

Motion Control Of A Soft Circular Crawling Robot Via Iterative Learning Control

A Reinforcement Learning Method for Motion Control With Constraints on an HPN Arm

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Learning-Based Force Control of a Surgical Robot for Tool-Soft Tissue Interaction

Iterative Learning Control for Motion Trajectory Tracking of a Circular Soft Crawling Robot

Position Control of Cable-Driven Robotic Soft Arm Based on Deep Reinforcement Learning

Vision-based reinforcement learning control of soft robot manipulators

A Soft Artificial Muscle Driven Robot with Reinforcement Learning

Memory-based Controllers for Efficient Data-driven Control of Soft Robots

Control of Free-Floating Space Robots to Capture Targets Using Soft Q-Learning.

An Efficient Learning Control Framework With Sim-to-Real for String-Type Artificial Muscle-Driven Robotic Systems

Using First Principles for Deep Learning and Model-Based Control of Soft Robots

Robotic arm reinforcement learning control method based on autonomous visual perception

Learning Reduced-Order Soft Robot Controller

Soft Robots Learn to Crawl: Jointly Optimizing Design and Control with Sim-to-Real Transfer

Design, Control, and Applications of a Soft Robotic Arm

Fuzzy Q-Learning interaction controller design for collaborative robot

A Hybrid Controller for a Soft Pneumatic Manipulator Based on Model Predictive Control and Iterative Learning Control

Distance-directed Target Searching for a Deep Visual Servo SMA Driven Soft Robot Using Reinforcement Learning

Hybrid LMC: Hybrid Learning and Model-based Control for Wheeled Humanoid Robot via Ensemble Deep Reinforcement Learning