Abstract:Legged robots that can instantly change motor patterns at different walking speeds are useful and can accomplish various tasks efficiently. However, state-of-the-art control methods either are difficult to develop or require long training times. In this study, we present a comprehensible neural control framework to integrate probability-based black-box optimization (PIBB) and supervised learning for robot motor pattern generation at various walking speeds. The control framework structure is based on a combination of a central pattern generator (CPG), a radial basis function (RBF) -based premotor network and a hypernetwork, resulting in a so-called neural CPG-RBF-hyper control network. First, the CPG-driven RBF network, acting as a complex motor pattern generator, was trained to learn policies (multiple motor patterns) for different speeds using PIBB. We also introduce an incremental learning strategy to avoid local optima. Second, the hypernetwork, which acts as a task/behavior to control parameter mapping, was trained using supervised learning. It creates a mapping between the internal CPG frequency (reflecting the walking speed) and motor behavior. This map represents the prior knowledge of the robot, which contains the optimal motor joint patterns at various CPG frequencies. Finally, when a user-defined robot walking frequency or speed is provided, the hypernetwork generates the corresponding policy for the CPG-RBF network. The result is a versatile locomotion controller which enables a quadruped robot to perform stable and robust walking at different speeds without sensory feedback. The policy of the controller was trained in the simulation (less than 1 h) and capable of transferring to a real robot. The generalization ability of the controller was demonstrated by testing the CPG frequencies that were not encountered during training.

Learning biped locomotion based on Q-learning and neural networks

Adaptive Locomotion Control of a Hexapod Robot Via Bio-Inspired Learning

Efficient Learning of Robust Quadruped Bounding Using Pretrained Neural Networks

Efficient Learning of Control Policies for Robust Quadruped Bounding using Pretrained Neural Networks

Agile and versatile bipedal robot tracking control through reinforcement learning

Application of Neural Network in the Stability of Biped Robot and Embedded Control of Walking Mode

ZSL-RPPO: Zero-Shot Learning for Quadrupedal Locomotion in Challenging Terrains using Recurrent Proximal Policy Optimization

Motion Sequence Learning for Robot Walking Based on Pose optimization

A Q-learning approach to the continuous control problem of robot inverted pendulum balancing

Learning Robust Locomotion for Bipedal Robot via Embedded Mechanics Properties

Neural-Dynamics Optimization and Repetitive Learning Control for Robotic Leg Prostheses

Experience-Learning Inspired Two-Step Reward Method for Efficient Legged Locomotion Learning Towards Natural and Robust Gaits

Hybrid learning mechanisms under a neural control network for various walking speed generation of a quadruped robot

Estimating Probability Distribution with Q-learning for Biped Gait Generation and Optimization.

Terrain-Aware Quadrupedal Locomotion via Reinforcement Learning

Learning to Locomote with Deep Neural-Network and CPG-based Control in a Soft Snake Robot

Real-Time Control of Full Actuated Biped Robot Based on Nonlinear Model Predictive Control

Skill Latent Space Based Multigait Learning for a Legged Robot

A Motion Planning and Control Method of Quadruped Robot Based on Deep Reinforcement Learning

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Learning Contact-aware CPG-based Locomotion in a Soft Snake Robot