Abstract:Soft robots can adaptively interact with unstructured environments. Reinforcement learning‐based controllers that leverage continuum mechanics are promising for solving complex interaction tasks. A closed‐loop pose and force control policy for a dexterous soft manipulator enabling dynamic pushing tasks is trained in simulation. The sim‐to‐real transfer of the control policy showcases the potential of soft robotic manipulation via reinforcement learning. Soft robots can adaptively interact with unstructured environments. However, nonlinear soft material properties challenge modeling and control. Learning‐based controllers that leverage efficient mechanical models are promising for solving complex interaction tasks. This article develops a closed‐loop pose/force controller for a dexterous soft manipulator enabling dynamic pushing tasks using deep reinforcement learning. Force tests investigate the mechanical properties of a soft robot module, resulting in orthogonal forces of 9−13 N. Then, the policy is trained in simulation leveraging a dynamic Cosserat rod model of the soft robot. Domain randomization mitigate the sim‐to‐real gap while careful reward engineering induced pose and force control even without explicit force inputs. Despite the approximate simulation, the sim‐to‐real transfer achieved an average reaching distance of 34±14 mm (8.1%L±3.4%L ), an average orientation error of 0.40±0.29 rad (23°±17° ) and applied pushing forces up to 3 N. Such performance is reasonable for the intended assistive tasks of the manipulator. The experiments uncovered that the soft robot interacting with the environment exhibited torsional and counter‐balancing movements. Although not explicitly enforced, they emerged from the mechanical intelligence of the manipulator. The results demonstrate the potential of soft robotic manipulation via reinforcement learning.

Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Reinforcement Learning Controllers for Soft Robots using Learned Environments

Model-Based Reinforcement Learning for Position Control of Continuum Manipulators Actuated by Pneumatic Artificial Muscles

Offline Reinforcement Learning of Robotic Control Using Deep Kinematics and Dynamics

Vision-based reinforcement learning control of soft robot manipulators

Visuo-dynamic self-modelling of soft robotic systems

Reinforcement Learning Tracking Control for Robotic Manipulator With Kernel-Based Dynamic Model

Model-based dynamic feedback control of a planar soft robot: trajectory tracking and interaction with the environment

Real-Robot Deep Reinforcement Learning: Improving Trajectory Tracking of Flexible-Joint Manipulator with Reference Correction

Closing the Control Loop with Time-Variant Embedded Soft Sensors and Recurrent Neural Networks

Learning-based Nonlinear Model Predictive Control of Articulated Soft Robots using Recurrent Neural Networks

Control Of Flexible Manipulator Based On Reinforcement Learning

A Hybrid Controller for a Soft Pneumatic Manipulator Based on Model Predictive Control and Iterative Learning Control

Pushing with Soft Robotic Arms via Deep Reinforcement Learning

Kinematic-Model-Free Predictive Control for Robotic Manipulator Target Reaching With Obstacle Avoidance

Model-free control framework for multi-limb soft robots

Dynamic Control of Soft Robotic Arm

Using First Principles for Deep Learning and Model-Based Control of Soft Robots

A Cable‐Actuated Soft Manipulator for Dexterous Grasping Based on Deep Reinforcement Learning

Vibration and Position Control of a Two-Link Flexible Manipulator Using Reinforcement Learning