Abstract:Legged robots are better able to adapt to different terrains compared with wheeled robots. However, traditional motion controllers suffer from extremely complex dynamics properties. Reinforcement learning (RL) helps to overcome the complications of dynamics design and calculation. In addition, the high autonomy of the RL controller results in a more robust response to complex environments and terrains compared with traditional controllers. However, RL algorithms are limited by the problems of convergence and training efficiency due to the complexity of the task. Learn and outperform the reference motion (LORM), an RL based framework for gait controlling of biped robot is proposed leveraging the prior knowledge of reference motion. The proposed trained agent outperformed the reference motion and existing motion-based methods. The RL environment was finely crafted for optimal performance, including the pruning of state space and action space, reward shaping, and design of episode criterion. Several improvements were implemented to further improve the training efficiency and performance including: random state initialization (RSI), the noise of joint angles, and a novel improvement based on symmetrization of gait. To validate the proposed method, the Darwin-op robot was set as the target platform and two different tasks were designed: (I) Walking as fast as possible and (II) Tracking specific velocity. In task (I), the proposed method resulted in the walking velocity of 0.488 m/s, with a 5.8 times improvement compared with the original traditional reference controller. The directional accuracy improved by 87.3%. The velocity performance achieved 2× compared with the rated max velocity and more than 8× compared with other recent works. To our knowledge, our work achieved the best velocity performance on the platform Darwin-op. In task (II), the proposed method achieved a tracking accuracy of over 95%. Different environments are introduced including plains, slopes, uneven terrains, and walking with external force, where the robot was expected to maintain walking stability with ideal speed and little direction deviation, to validate the performance and robustness of the proposed method.

Natural Residual Reinforcement Learning for Bicycle Robot Control

Deep Reinforcement Learning-Based Control of Bicycle Robots on Rough Terrain

Path Following and Stabilisation of a Bicycle Model using a Reinforcement Learning Approach

A Path-Integral-Based Reinforcement Learning Algorithm for Path Following of an Autoassembly Mobile Robot

High Maneuverability Control of Single-track Two-wheeled Robot in Narrow Terrain Based on Reinforcement Learning.

Residual Reinforcement Learning for Robot Control

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Combined control algorithm based on synchronous reinforcement learning for a self-balancing bicycle robot

Online Series-Parallel Reinforcement-Learning-Based Balancing Control for Reaction Wheel Bicycle Robots on a Curved Pavement

LORM: a Novel Reinforcement Learning Framework for Biped Gait Control

Modelling, Positioning, and Deep Reinforcement Learning Path Tracking Control of Scaled Robotic Vehicles: Design and Experimental Validation

Reinforcement Learning-Based Control of Single-Track Two-Wheeled Robots in Narrow Terrain

RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion

Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots

Model Learning for Two-Wheeled Robot Self-Balance Control

Model-free Neural Lyapunov Control for Safe Robot Navigation

Long-Range Indoor Navigation With PRM-RL

Autonomous Learning and Navigation of Mobile Robots Based on Deep Reinforcement Learning

SLR: Learning Quadruped Locomotion without Privileged Information

Hybrid LMC: Hybrid Learning and Model-based Control for Wheeled Humanoid Robot via Ensemble Deep Reinforcement Learning