Abstract:Legged robots are better able to adapt to different terrains compared with wheeled robots. However, traditional motion controllers suffer from extremely complex dynamics properties. Reinforcement learning (RL) helps to overcome the complications of dynamics design and calculation. In addition, the high autonomy of the RL controller results in a more robust response to complex environments and terrains compared with traditional controllers. However, RL algorithms are limited by the problems of convergence and training efficiency due to the complexity of the task. Learn and outperform the reference motion (LORM), an RL based framework for gait controlling of biped robot is proposed leveraging the prior knowledge of reference motion. The proposed trained agent outperformed the reference motion and existing motion-based methods. The RL environment was finely crafted for optimal performance, including the pruning of state space and action space, reward shaping, and design of episode criterion. Several improvements were implemented to further improve the training efficiency and performance including: random state initialization (RSI), the noise of joint angles, and a novel improvement based on symmetrization of gait. To validate the proposed method, the Darwin-op robot was set as the target platform and two different tasks were designed: (I) Walking as fast as possible and (II) Tracking specific velocity. In task (I), the proposed method resulted in the walking velocity of 0.488 m/s, with a 5.8 times improvement compared with the original traditional reference controller. The directional accuracy improved by 87.3%. The velocity performance achieved 2× compared with the rated max velocity and more than 8× compared with other recent works. To our knowledge, our work achieved the best velocity performance on the platform Darwin-op. In task (II), the proposed method achieved a tracking accuracy of over 95%. Different environments are introduced including plains, slopes, uneven terrains, and walking with external force, where the robot was expected to maintain walking stability with ideal speed and little direction deviation, to validate the performance and robustness of the proposed method.

Using Reinforcement Learning to Develop a Novel Gait for a Bio-Robotic California Sea Lion

Generation of robotic fish locomotion through biomimetic learning

A GIM-based approach for biomimetic robot motion learning.

Design and implementation of a biomimetic robotic fish with 3D locomotion

A GIM-Based Biomimetic Learning Approach for Motion Generation of a Multi-Joint Robotic Fish

Agile Gait Generation For An Anguilliform Robotic Fish

Locomotion Learning for an Anguilliform Robotic Fish Using Central Pattern Generator Approach

Guided Deep Reinforcement Learning for Articulated Swimming Robots

Maneuverable gait selection for a novel fish-inspired robot using a CMA-ES-assisted workflow

Reinforcement learning control for the swimming motions of a beaver-like, single-legged robot based on biological inspiration

Learning Agile Swimming: An End-to-End Approach without CPGs

Physics-informed reinforcement learning for motion control of a fish-like swimming robot

Autonomous Optimization of Swimming Gait in a Fish Robot with Multiple Onboard Sensors

Path-following Control of Fish-like Robots: A Deep Reinforcement Learning Approach

Enhancing efficiency and propulsion in bio-mimetic robotic fish through end-to-end deep reinforcement learning

Learning swimming via deep reinforcement learning

Leveraging Imitation Learning on Pose Regulation Problem of a Robotic Fish

Reinforcement Learning of CPG-regulated Locomotion Controller for a Soft Snake Robot

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

LORM: a Novel Reinforcement Learning Framework for Biped Gait Control

Adaptive Gait Modeling and Optimization for Principally Kinematic Systems