Abstract:Purpose Limited by the types of sensors, the state information available for musculoskeletal robots with highly redundant, nonlinear muscles is often incomplete, which makes the control face a bottleneck problem. The aim of this paper is to design a method to improve the motion performance of musculoskeletal robots in partially observable scenarios, and to leverage the ontology knowledge to enhance the algorithm’s adaptability to musculoskeletal robots that have undergone changes. Design/methodology/approach A memory and attention-based reinforcement learning method is proposed for musculoskeletal robots with prior knowledge of muscle synergies. First, to deal with partially observed states available to musculoskeletal robots, a memory and attention-based network architecture is proposed for inferring more sufficient and intrinsic states. Second, inspired by muscle synergy hypothesis in neuroscience, prior knowledge of a musculoskeletal robot’s muscle synergies is embedded in network structure and reward shaping. Findings Based on systematic validation, it is found that the proposed method demonstrates superiority over the traditional twin delayed deep deterministic policy gradients (TD3) algorithm. A musculoskeletal robot with highly redundant, nonlinear muscles is adopted to implement goal-directed tasks. In the case of 21-dimensional states, the learning efficiency and accuracy are significantly improved compared with the traditional TD3 algorithm; in the case of 13-dimensional states without velocities and information from the end effector, the traditional TD3 is unable to complete the reaching tasks, while the proposed method breaks through this bottleneck problem. Originality/value In this paper, a novel memory and attention-based reinforcement learning method with prior knowledge of muscle synergies is proposed for musculoskeletal robots to deal with partially observable scenarios. Compared with the existing methods, the proposed method effectively improves the performance. Furthermore, this paper promotes the fusion of neuroscience and robotics.

Acquiring musculoskeletal skills with curriculum-based reinforcement learning

Control of Antagonistic McKibben Muscles Via a Bio-inspired Approach

Reinforcement Learning of Musculoskeletal Control from Functional Simulations

Sim: A goal-driven framework for elucidating the neural control of movement through musculoskeletal modeling

Dietary folate and vitamin B12 intake before diagnosis decreases gastric cancer mortality risk among susceptible MTHFR 677TT carriers.

Regulation of LPA receptor function by estrogens.

MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control

Scalable muscle-actuated human simulation and control

Learning to Control Emulated Muscles in Real Robots: Towards Exploiting Bio-Inspired Actuator Morphology

Muscle Excitation Estimation in Biomechanical Simulation Using NAF Reinforcement Learning

Reinforcement Learning for High-dimensional Continuous Control in Biomechanics: An Intro to ArtiSynth-RL

Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity

Artificial Intelligence for Prosthetics - challenge solutions.

Hitting the Gym: Reinforcement Learning Control of Exercise-Strengthened Biohybrid Robots in Simulation

Learned parametrized dynamic movement primitives with shared synergies for controlling robotic and musculoskeletal systems

A memory and attention-based reinforcement learning for musculoskeletal robots with prior knowledge of muscle synergies

Neuro-musculoskeletal modeling reveals muscle-level neural dynamics of adaptive learning in sensorimotor cortex

Local staging of anal and distal colorectal tumors with the magnetic resonance endoscope.

Exciting Action: Investigating Efficient Exploration for Learning Musculoskeletal Humanoid Locomotion

Fine-tuning Myoelectric Control through Reinforcement Learning in a Game Environment

Learning Control Policies for Imitating Human Gaits