Abstract:Purpose Limited by the types of sensors, the state information available for musculoskeletal robots with highly redundant, nonlinear muscles is often incomplete, which makes the control face a bottleneck problem. The aim of this paper is to design a method to improve the motion performance of musculoskeletal robots in partially observable scenarios, and to leverage the ontology knowledge to enhance the algorithm’s adaptability to musculoskeletal robots that have undergone changes. Design/methodology/approach A memory and attention-based reinforcement learning method is proposed for musculoskeletal robots with prior knowledge of muscle synergies. First, to deal with partially observed states available to musculoskeletal robots, a memory and attention-based network architecture is proposed for inferring more sufficient and intrinsic states. Second, inspired by muscle synergy hypothesis in neuroscience, prior knowledge of a musculoskeletal robot’s muscle synergies is embedded in network structure and reward shaping. Findings Based on systematic validation, it is found that the proposed method demonstrates superiority over the traditional twin delayed deep deterministic policy gradients (TD3) algorithm. A musculoskeletal robot with highly redundant, nonlinear muscles is adopted to implement goal-directed tasks. In the case of 21-dimensional states, the learning efficiency and accuracy are significantly improved compared with the traditional TD3 algorithm; in the case of 13-dimensional states without velocities and information from the end effector, the traditional TD3 is unable to complete the reaching tasks, while the proposed method breaks through this bottleneck problem. Originality/value In this paper, a novel memory and attention-based reinforcement learning method with prior knowledge of muscle synergies is proposed for musculoskeletal robots to deal with partially observable scenarios. Compared with the existing methods, the proposed method effectively improves the performance. Furthermore, this paper promotes the fusion of neuroscience and robotics.

A Bio-Inspired Integration Model of Basal Ganglia and Cerebellum for Motion Learning of a Musculoskeletal Robot

Motion Control of Bionic Robots Via Biomimetic Learning

Control of Antagonistic McKibben Muscles Via a Bio-inspired Approach

Bio-inspired Control of a Rehabilitation Robot Actuated by Pneumatic Artificial Muscles

A GIM-based approach for biomimetic robot motion learning.

Local staging of anal and distal colorectal tumors with the magnetic resonance endoscope.

A Cerebellum-Inspired Prediction and Correction Model for Motion Control of a Musculoskeletal Robot

Adaptive Locomotion Control of a Hexapod Robot Via Bio-Inspired Learning

Motion Control Of A Snake Robot Via Cerebellum-Inspired Learning Control

Motion Control of a Multi-Joint Robotic Fish Based on Biomimetic Learning

An Integrated Neurorobotics Model of the Cerebellar-Basal Ganglia Circuitry

A memory and attention-based reinforcement learning for musculoskeletal robots with prior knowledge of muscle synergies

Biologically Inspired Motion Modeling and Neural Control for Robot Learning From Demonstrations

Steering Motion Control of a Snake Robot Via a Biomimetic Approach

A Simplified Cerebellar Model with Priority-based Delayed Eligibility Trace Learning for Motor Control

Integrated model of cerebellal supervised learning and basal ganglia's reinforcement learning for mobile robot behavioral decision-making

Hitting the Gym: Reinforcement Learning Control of Exercise-Strengthened Biohybrid Robots in Simulation

Neuro-Inspired Motion Control of a Soft Myriapod Robot

Use of an artificial neuroadaptive robot model to describe adaptive and learning motor mechanisms in the central nervous system

Regulation of LPA receptor function by estrogens.

Biologically inspired model simulating visual pathways and cerebellum function in human - Achieving visuomotor coordination and high precision movement with learning ability