Abstract:Purpose Limited by the types of sensors, the state information available for musculoskeletal robots with highly redundant, nonlinear muscles is often incomplete, which makes the control face a bottleneck problem. The aim of this paper is to design a method to improve the motion performance of musculoskeletal robots in partially observable scenarios, and to leverage the ontology knowledge to enhance the algorithm’s adaptability to musculoskeletal robots that have undergone changes. Design/methodology/approach A memory and attention-based reinforcement learning method is proposed for musculoskeletal robots with prior knowledge of muscle synergies. First, to deal with partially observed states available to musculoskeletal robots, a memory and attention-based network architecture is proposed for inferring more sufficient and intrinsic states. Second, inspired by muscle synergy hypothesis in neuroscience, prior knowledge of a musculoskeletal robot’s muscle synergies is embedded in network structure and reward shaping. Findings Based on systematic validation, it is found that the proposed method demonstrates superiority over the traditional twin delayed deep deterministic policy gradients (TD3) algorithm. A musculoskeletal robot with highly redundant, nonlinear muscles is adopted to implement goal-directed tasks. In the case of 21-dimensional states, the learning efficiency and accuracy are significantly improved compared with the traditional TD3 algorithm; in the case of 13-dimensional states without velocities and information from the end effector, the traditional TD3 is unable to complete the reaching tasks, while the proposed method breaks through this bottleneck problem. Originality/value In this paper, a novel memory and attention-based reinforcement learning method with prior knowledge of muscle synergies is proposed for musculoskeletal robots to deal with partially observable scenarios. Compared with the existing methods, the proposed method effectively improves the performance. Furthermore, this paper promotes the fusion of neuroscience and robotics.

Training Musculoskeletal Arm Play Taichi with Deep Reinforcement Learning

Bio-inspired Control of a Rehabilitation Robot Actuated by Pneumatic Artificial Muscles

Control of a Rehabilitation Robotic Device Driven by Antagonistic Soft Actuators

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Design and Control of a Muscle-skeleton Robot Elbow based on Reinforcement Learning

Humanoid Muscle-Skeleton Robot Arm Design and Control Based on Reinforcement Learning

Semiparametric Musculoskeletal Model for Reinforcement Learning-Based Trajectory Tracking

Deep Reinforcement Learning for an Anthropomorphic Robotic Arm under Sparse Reward Tasks

A High-Efficient Reinforcement Learning Approach for Dexterous Manipulation

A memory and attention-based reinforcement learning for musculoskeletal robots with prior knowledge of muscle synergies

Memory, Attention, and Muscle Synergies Based Reinforcement and Transfer Learning for Musculoskeletal Robots under Imperfect Observation

Getting the Ball Rolling: Learning a Dexterous Policy for a Biomimetic Tendon-Driven Hand with Rolling Contact Joints

Design and Control of Pneumatic Muscle-Driven Robot Arm Based on Bilateral Rehabilitation Training

Humanoid motion planning of robotic arm based on human arm action feature and reinforcement learning

Control of Robotic Joint Actuated by Antagonistic Pneumatic Artificial Muscles Based on Model-free Reinforcement Learning

Learning to Play Table Tennis From Scratch Using Muscular Robots

Muscle‐driven Virtual Human Motion Generation Approach Based on Deep Reinforcement Learning

A Multitasking-Oriented Robot Arm Motion Planning Scheme Based on Deep Reinforcement Learning and Twin Synchro-Control

Interactive control algorithm for shoulder-amputated prosthesis and object based on reinforcement learning

Reinforcement Learning of Musculoskeletal Control from Functional Simulations

Hitting the Gym: Reinforcement Learning Control of Exercise-Strengthened Biohybrid Robots in Simulation