Abstract:Learning effective continuous control policies in high-dimensional systems, including musculoskeletal agents, remains a significant challenge. Over the course of biological evolution, organisms have developed robust mechanisms for overcoming this complexity to learn highly sophisticated strategies for motor control. What accounts for this robust behavioral flexibility? Modular control via muscle synergies, i.e. coordinated muscle co-contractions, is considered to be one putative mechanism that enables organisms to learn muscle control in a simplified and generalizable action space. Drawing inspiration from this evolved motor control strategy, we use physiologically accurate human hand and leg models as a testbed for determining the extent to which a Synergistic Action Representation (SAR) acquired from simpler tasks facilitates learning more complex tasks. We find in both cases that SAR-exploiting policies significantly outperform end-to-end reinforcement learning. Policies trained with SAR were able to achieve robust locomotion on a wide set of terrains with high sample efficiency, while baseline approaches failed to learn meaningful behaviors. Additionally, policies trained with SAR on a multiobject manipulation task significantly outperformed (>70% success) baseline approaches (<20% success). Both of these SAR-exploiting policies were also found to generalize zero-shot to out-of-domain environmental conditions, while policies that did not adopt SAR failed to generalize. Finally, we establish the generality of SAR on broader high-dimensional control problems using a robotic manipulation task set and a full-body humanoid locomotion task. To the best of our knowledge, this investigation is the first of its kind to present an end-to-end pipeline for discovering synergies and using this representation to learn high-dimensional continuous control across a wide diversity of tasks.

Hand-arm autonomous grasping: synergistic motions to enhance the learning process

Ensemble Bootstrapped Deep Deterministic Policy Gradient For Vision-Based Robotic Grasping

Object Manipulation with an Anthropomorphic Robotic Hand via Deep Reinforcement Learning with a Synergy Space of Natural Hand Poses

A Grasp Pose is All You Need: Learning Multi-fingered Grasping with Deep Reinforcement Learning from Vision and Touch

Reaching, Grasping and Re-grasping: Learning Multimode Grasping Skills

Vision-based grasp learning of an anthropomorphic hand-arm system in a synergy-based control framework

Extracting bimanual synergies with reinforcement learning

Multifingered Grasping Based on Multimodal Reinforcement Learning

Automatic Grasp Planning Algorithm for Synergistic Underactuated Hands.

Reinforcement learning of motor skills with policy gradients

A Multi-Agent Approach for Adaptive Finger Cooperation in Learning-based In-Hand Manipulation

Learning Dual-Arm Push and Grasp Synergy in Dense Clutter

SAR: Generalization of Physiological Agility and Dexterity via Synergistic Action Representation

Learning Cross-hand Policies for High-DOF Reaching and Grasping

On Automated Object Grasping for Intelligent Prosthetic Hands Using Machine Learning

Acquisition of synergy for low-dimensional control of multi-fingered hands by reinforcement learning

Discovering Synergies for Robot Manipulation with Multi-Task Reinforcement Learning

GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Grasping

Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds

Getting the Ball Rolling: Learning a Dexterous Policy for a Biomimetic Tendon-Driven Hand with Rolling Contact Joints

Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation