Abstract:Legged robots are better able to adapt to different terrains compared with wheeled robots. However, traditional motion controllers suffer from extremely complex dynamics properties. Reinforcement learning (RL) helps to overcome the complications of dynamics design and calculation. In addition, the high autonomy of the RL controller results in a more robust response to complex environments and terrains compared with traditional controllers. However, RL algorithms are limited by the problems of convergence and training efficiency due to the complexity of the task. Learn and outperform the reference motion (LORM), an RL based framework for gait controlling of biped robot is proposed leveraging the prior knowledge of reference motion. The proposed trained agent outperformed the reference motion and existing motion-based methods. The RL environment was finely crafted for optimal performance, including the pruning of state space and action space, reward shaping, and design of episode criterion. Several improvements were implemented to further improve the training efficiency and performance including: random state initialization (RSI), the noise of joint angles, and a novel improvement based on symmetrization of gait. To validate the proposed method, the Darwin-op robot was set as the target platform and two different tasks were designed: (I) Walking as fast as possible and (II) Tracking specific velocity. In task (I), the proposed method resulted in the walking velocity of 0.488 m/s, with a 5.8 times improvement compared with the original traditional reference controller. The directional accuracy improved by 87.3%. The velocity performance achieved 2× compared with the rated max velocity and more than 8× compared with other recent works. To our knowledge, our work achieved the best velocity performance on the platform Darwin-op. In task (II), the proposed method achieved a tracking accuracy of over 95%. Different environments are introduced including plains, slopes, uneven terrains, and walking with external force, where the robot was expected to maintain walking stability with ideal speed and little direction deviation, to validate the performance and robustness of the proposed method.

Distributed-force-feedback-based reflex with online learning for adaptive quadruped motor control

Adaptive Locomotion Control of a Hexapod Robot Via Bio-Inspired Learning

Learning Gait-conditioned Bipedal Locomotion with Motor Adaptation

Adaptive Walking of a Quadrupedal Robot Based on Layered Biological Reflexes

MorAL: Learning Morphologically Adaptive Locomotion Controller for Quadrupedal Robots on Challenging Terrains

CPG-Based Hierarchical Locomotion Control for Modular Quadrupedal Robots Using Deep Reinforcement Learning.

Learning Agile, Robust Locomotion Skills for Quadruped Robot.

Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning

An Adaptive Locomotion Controller for a Hexapod Robot: CPG, Kinematics and Force Feedback.

Learning Fast and Agile Quadrupedal Locomotion over Complex Terrain

Learning an Adaptive Fall Recovery Controller for Quadrupeds on Complex Terrains

Simultaneous locomotion and manipulation control of quadruped robots using reinforcement learning-based adaptive fractional-order sliding-mode control

Multi-Layered CPG for Adaptive Walking of Quadruped Robots

Adaptive Force-Based Control of Dynamic Legged Locomotion Over Uneven Terrain

Real-time Replanning and Control of Quadruped Robots for Blind Locomotion on Uneven Terrain

Online Optimization of Central Pattern Generators for Quadruped Locomotion

Decentralized Deep Reinforcement Learning for a Distributed and Adaptive Locomotion Controller of a Hexapod Robot

Adaptive Walking Control of Quadruped Robots Based on Central Pattern Generator （CPG） and Reflex

Combining Evolutionary and Adaptive Control Strategies for Quadruped Robotic Locomotion

LORM: a Novel Reinforcement Learning Framework for Biped Gait Control