Abstract:Purpose Limited by the types of sensors, the state information available for musculoskeletal robots with highly redundant, nonlinear muscles is often incomplete, which makes the control face a bottleneck problem. The aim of this paper is to design a method to improve the motion performance of musculoskeletal robots in partially observable scenarios, and to leverage the ontology knowledge to enhance the algorithm’s adaptability to musculoskeletal robots that have undergone changes. Design/methodology/approach A memory and attention-based reinforcement learning method is proposed for musculoskeletal robots with prior knowledge of muscle synergies. First, to deal with partially observed states available to musculoskeletal robots, a memory and attention-based network architecture is proposed for inferring more sufficient and intrinsic states. Second, inspired by muscle synergy hypothesis in neuroscience, prior knowledge of a musculoskeletal robot’s muscle synergies is embedded in network structure and reward shaping. Findings Based on systematic validation, it is found that the proposed method demonstrates superiority over the traditional twin delayed deep deterministic policy gradients (TD3) algorithm. A musculoskeletal robot with highly redundant, nonlinear muscles is adopted to implement goal-directed tasks. In the case of 21-dimensional states, the learning efficiency and accuracy are significantly improved compared with the traditional TD3 algorithm; in the case of 13-dimensional states without velocities and information from the end effector, the traditional TD3 is unable to complete the reaching tasks, while the proposed method breaks through this bottleneck problem. Originality/value In this paper, a novel memory and attention-based reinforcement learning method with prior knowledge of muscle synergies is proposed for musculoskeletal robots to deal with partially observable scenarios. Compared with the existing methods, the proposed method effectively improves the performance. Furthermore, this paper promotes the fusion of neuroscience and robotics.

Efficient reinforcement learning control for continuum robots based on Inexplicit Prior Knowledge

Motion Control Of A Soft Circular Crawling Robot Via Iterative Learning Control

Iteratively Successive Projection: A Novel Continuous Approach for the Task-Based Control of Redundant Robots.

Iterative Learning Control for Motion Trajectory Tracking of a Circular Soft Crawling Robot

A Deep Reinforcement Learning Control Method for a Four-Link Brachiation Robot

A Reinforcement Learning Approach for Continuum Robot Control

Vision-based reinforcement learning control of soft robot manipulators

Learning-based Control for Tendon-Driven Continuum Robotic Arms

A memory and attention-based reinforcement learning for musculoskeletal robots with prior knowledge of muscle synergies

Data Informed Residual Reinforcement Learning for High-Dimensional Robotic Tracking Control

Axis-space framework for cable-driven soft continuum robot control via reinforcement learning

Behavior-predefined adaptive control for heterogeneous continuum robots

Real-Robot Deep Reinforcement Learning: Improving Trajectory Tracking of Flexible-Joint Manipulator with Reference Correction

An actuator space optimal kinematic path tracking framework for tendon-driven continuum robots: Theory, algorithm and validation

Residual Reinforcement Learning for Robot Control

Learning-Based Control for Soft Robot–Environment Interaction with Force/Position Tracking Capability

Model-free control framework for multi-limb soft robots

Characterization of continuum robot arms under reinforcement learning and derived improvements

An Efficient Learning Control Framework With Sim-to-Real for String-Type Artificial Muscle-Driven Robotic Systems

Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control

Tactile Active Inference Reinforcement Learning for Efficient Robotic Manipulation Skill Acquisition