Learning agile and dynamic motor skills for legged robots

Jemin Hwangbo,Joonho Lee,Alexey Dosovitskiy,Dario Bellicoso,Vassilios Tsounis,Vladlen Koltun,Marco Hutter

DOI: https://doi.org/10.1126/scirobotics.aau5872

IF: 25

2019-01-30

Science Robotics

Abstract:A method for learning agile control policies uses simulated data to enable precise, efficient movements in a complex physical robot.

robotics

What problem does this paper attempt to address?

Learning and Adapting Agile Locomotion Skills by Transferring Experience

Laura Smith,J. Chase Kew,Tianyu Li,Linda Luu,Xue Bin Peng,Sehoon Ha,Jie Tan,Sergey Levine

2023-04-20

Abstract:Legged robots have enormous potential in their range of capabilities, from navigating unstructured terrains to high-speed running. However, designing robust controllers for highly agile dynamic motions remains a substantial challenge for roboticists. Reinforcement learning (RL) offers a promising data-driven approach for automatically training such controllers. However, exploration in these high-dimensional, underactuated systems remains a significant hurdle for enabling legged robots to learn performant, naturalistic, and versatile agility skills. We propose a framework for training complex robotic skills by transferring experience from existing controllers to jumpstart learning new tasks. To leverage controllers we can acquire in practice, we design this framework to be flexible in terms of their source -- that is, the controllers may have been optimized for a different objective under different dynamics, or may require different knowledge of the surroundings -- and thus may be highly suboptimal for the target task. We show that our method enables learning complex agile jumping behaviors, navigating to goal locations while walking on hind legs, and adapting to new environments. We also demonstrate that the agile behaviors learned in this way are graceful and safe enough to deploy in the real world.

Robotics,Artificial Intelligence
An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement

Haojie Shi,Tingguang Li,Qingxu Zhu,Jiapeng Sheng,Lei Han,Max Q.-H. Meng

2024-03-18

Abstract:Learning-based methods have improved locomotion skills of quadruped robots through deep reinforcement learning. However, the sim-to-real gap and low sample efficiency still limit the skill transfer. To address this issue, we propose an efficient model-based learning framework that combines a world model with a policy network. We train a differentiable world model to predict future states and use it to directly supervise a Variational Autoencoder (VAE)-based policy network to imitate real animal behaviors. This significantly reduces the need for real interaction data and allows for rapid policy updates. We also develop a high-level network to track diverse commands and trajectories. Our simulated results show a tenfold sample efficiency increase compared to reinforcement learning methods such as PPO. In real-world testing, our policy achieves proficient command-following performance with only a two-minute data collection period and generalizes well to new speeds and paths.

Robotics
Learning Whole-body Motor Skills for Humanoids

Chuanyu Yang,Kai Yuan,Wolfgang Merkt,Taku Komura,Sethu Vijayakumar,Zhibin Li

DOI: https://doi.org/10.1109/HUMANOIDS.2018.8625045

2020-02-08

Abstract:This paper presents a hierarchical framework for Deep Reinforcement Learning that acquires motor skills for a variety of push recovery and balancing behaviors, i.e., ankle, hip, foot tilting, and stepping strategies. The policy is trained in a physics simulator with realistic setting of robot model and low-level impedance control that are easy to transfer the learned skills to real robots. The advantage over traditional methods is the integration of high-level planner and feedback control all in one single coherent policy network, which is generic for learning versatile balancing and recovery motions against unknown perturbations at arbitrary locations (e.g., legs, torso). Furthermore, the proposed framework allows the policy to be learned quickly by many state-of-the-art learning algorithms. By comparing our learned results to studies of preprogrammed, special-purpose controllers in the literature, self-learned skills are comparable in terms of disturbance rejection but with additional advantages of producing a wide range of adaptive, versatile and robust behaviors.

Robotics,Machine Learning
Learning Agile Locomotion Skills with a Mentor

Atil Iscen,George Yu,Alejandro Escontrela,Deepali Jain,Jie Tan,Ken Caluwaerts

DOI: https://doi.org/10.48550/arXiv.2011.05541

2020-11-11

Abstract:Developing agile behaviors for legged robots remains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requires tedious reward shaping and careful curriculum design. We formulate agile locomotion as a multi-stage learning problem in which a mentor guides the agent throughout the training. The mentor is optimized to place a checkpoint to guide the movement of the robot's center of mass while the student (i.e. the robot) learns to reach these checkpoints. Once the student can solve the task, we teach the student to perform the task without the mentor. We evaluate our proposed learning system with a simulated quadruped robot on a course consisting of randomly generated gaps and hurdles. Our method significantly outperforms a single-stage RL baseline without a mentor, and the quadruped robot can agilely run and jump across gaps and obstacles. Finally, we present a detailed analysis of the learned behaviors' feasibility and efficiency.

Robotics
A learning-based control pipeline for generic motor skills for quadruped robots

Yecheng Shao,Yongbin Jin,Zhilong Huang,Hongtao Wang,Wei Yang

DOI: https://doi.org/10.1631/jzus.a2300128

2024-02-13

Journal of Zhejiang University SCIENCE A

Abstract:Performing diverse motor skills with a universal controller has been a longstanding challenge for legged robots. While motion imitation-based reinforcement learning (RL) has shown remarkable performance in reproducing designed motor skills, the trained controller is only suitable for one specific type of motion. Motion synthesis has been well developed to generate a variety of different motions for character animation, but those motions only contain kinematic information and cannot be used for control. In this study, we introduce a control pipeline combining motion synthesis and motion imitation-based RL for generic motor skills. We design an animation state machine to synthesize motion from various sources and feed the generated kinematic reference trajectory to the RL controller as part of the input. With the proposed method, we show that a single policy is able to learn various motor skills simultaneously. Further, we notice the ability of the policy to uncover the correlations lurking behind the reference motions to improve control performance. We analyze this ability based on the predictability of the reference trajectory and use the quantified measurements to optimize the design of the controller. To demonstrate the effectiveness of our method, we deploy the trained policy on hardware and, with a single control policy, the quadruped robot can perform various learned skills, including automatic gait transitions, high kick, and forward jump.

engineering, multidisciplinary,physics, applied
Deep Reinforcement Learning Based Co-Optimization of Morphology and Gait for Small-Scale Legged Robot

Ci Chen,Pingyu Xiang,Jingyu Zhang,Rong Xiong,Yue Wang,Haojian Lu

DOI: https://doi.org/10.1109/tmech.2023.3330427

2024-01-01

Abstract:Small-scale legged robots have found widespread utilization in various industrial and biomedical applications due to their compact size and superior locomotion capabilities. Reducing the number of actuators is often desirable to decrease the robot's size and weight, which comes at the expense of the robot's workspace. Our study proposes a method to enhance the mobility of small-scale legged robots with limited degrees of actuators (DoAs) by co-optimizing both morphology parameters and control policy. The co-optimization is formulated as a bi-level optimization problem, where the control policy is designed using deep reinforcement learning algorithms and central pattern generators (CPGs) at the lower level. The inclusion of CPGs significantly speeds up training and enables the application of simulation results in real-world scenarios. At the upper level, morphology optimization is achieved through Bayesian optimization based on dual-networks. This approach eliminates the need to train a policy for each morphology candidate from scratch, leveraging previous experience to enhance efficiency. Through simulation and physical experiments, the effectiveness of our proposed approach is demonstrated, showcasing its ability to discover optimal morphology and gait for small-scale legged robots with limited DoAs. These findings have potential long-term impacts on small-scale legged robot design and locomotion control.
Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion

Tairan He,Chong Zhang,Wenli Xiao,Guanqi He,Changliu Liu,Guanya Shi

2024-05-21

Abstract:Legged robots navigating cluttered environments must be jointly agile for efficient task execution and safe to avoid collisions with obstacles or humans. Existing studies either develop conservative controllers (< 1.0 m/s) to ensure safety, or focus on agility without considering potentially fatal collisions. This paper introduces Agile But Safe (ABS), a learning-based control framework that enables agile and collision-free locomotion for quadrupedal robots. ABS involves an agile policy to execute agile motor skills amidst obstacles and a recovery policy to prevent failures, collaboratively achieving high-speed and collision-free navigation. The policy switch in ABS is governed by a learned control-theoretic reach-avoid value network, which also guides the recovery policy as an objective function, thereby safeguarding the robot in a closed loop. The training process involves the learning of the agile policy, the reach-avoid value network, the recovery policy, and an exteroception representation network, all in simulation. These trained modules can be directly deployed in the real world with onboard sensing and computation, leading to high-speed and collision-free navigation in confined indoor and outdoor spaces with both static and dynamic obstacles.

Robotics,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning,Systems and Control
Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Yikai Wang,Zheyuan Jiang,Jianyu Chen

2023-10-06

Abstract:Recently, reinforcement learning has become a promising and polular solution for robot legged locomotion. Compared to model-based control, reinforcement learning based controllers can achieve better robustness against uncertainties of environments through sim-to-real learning. However, the corresponding learned gaits are in general overly conservative and unatural. In this paper, we propose a new framework for learning robust, agile and natural legged locomotion skills over challenging terrain. We incorporate an adversarial training branch based on real animal locomotion data upon a teacher-student training pipeline for robust sim-to-real transfer. Empirical results on both simulation and real world of a quadruped robot demonstrate that our proposed algorithm enables robustly traversing challenging terrains such as stairs, rocky ground and slippery floor with only proprioceptive perception. Meanwhile, the gaits are more agile, natural, and energy efficient compared to the baselines. Both qualitative and quantitative results are presented in this paper.

Robotics,Artificial Intelligence
Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion

Siddhant Gangapurwala,Luigi Campanaro,Ioannis Havoutis

DOI: https://doi.org/10.48550/arXiv.2209.14887

2023-02-21

Abstract:Robotic locomotion is often approached with the goal of maximizing robustness and reactivity by increasing motion control frequency. We challenge this intuitive notion by demonstrating robust and dynamic locomotion with a learned motion controller executing at as low as 8 Hz on a real ANYmal C quadruped. The robot is able to robustly and repeatably achieve a high heading velocity of 1.5 m/s, traverse uneven terrain, and resist unexpected external perturbations. We further present a comparative analysis of deep reinforcement learning (RL) based motion control policies trained and executed at frequencies ranging from 5 Hz to 200 Hz. We show that low-frequency policies are less sensitive to actuation latencies and variations in system dynamics. This is to the extent that a successful sim-to-real transfer can be performed even without any dynamics randomization or actuation modeling. We support this claim through a set of rigorous empirical evaluations. Moreover, to assist reproducibility, we provide the training and deployment code along with an extended analysis at <a class="link-external link-https" href="https://ori-drs.github.io/lfmc/" rel="external noopener nofollow">this https URL</a>.

Robotics,Artificial Intelligence
Dynamic Bipedal Maneuvers through Sim-to-Real Reinforcement Learning

Fangzhou Yu,Ryan Batke,Jeremy Dao,Jonathan Hurst,Kevin Green,Alan Fern

DOI: https://doi.org/10.48550/arXiv.2207.07835

IF: 3.7

2022-07-16

Robotics

Abstract:For legged robots to match the athletic capabilities of humans and animals, they must not only produce robust periodic walking and running, but also seamlessly switch between nominal locomotion gaits and more specialized transient maneuvers. Despite recent advancements in controls of bipedal robots, there has been little focus on producing highly dynamic behaviors. Recent work utilizing reinforcement learning to produce policies for control of legged robots have demonstrated success in producing robust walking behaviors. However, these learned policies have difficulty expressing a multitude of different behaviors on a single network. Inspired by conventional optimization-based control techniques for legged robots, this work applies a recurrent policy to execute four-step, 90 degree turns trained using reference data generated from optimized single rigid body model trajectories. We present a novel training framework using epilogue terminal rewards for learning specific behaviors from pre-computed trajectory data and demonstrate a successful transfer to hardware on the bipedal robot Cassie.
Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response

Junfeng Long,Zirui Wang,Quanyi Li,Jiawei Gao,Liu Cao,Jiangmiao Pang

2024-01-02

Abstract:Robust locomotion control depends on accurate state estimations. However, the sensors of most legged robots can only provide partial and noisy observations, making the estimation particularly challenging, especially for external states like terrain frictions and elevation maps. Inspired by the classical Internal Model Control principle, we consider these external states as disturbances and introduce Hybrid Internal Model (HIM) to estimate them according to the response of the robot. The response, which we refer to as the hybrid internal embedding, contains the robot's explicit velocity and implicit stability representation, corresponding to two primary goals for locomotion tasks: explicitly tracking velocity and implicitly maintaining stability. We use contrastive learning to optimize the embedding to be close to the robot's successor state, in which the response is naturally embedded. HIM has several appealing benefits: It only needs the robot's proprioceptions, i.e., those from joint encoders and IMU as observations. It innovatively maintains consistent observations between simulation reference and reality that avoids information loss in mimicking learning. It exploits batch-level information that is more robust to noises and keeps better sample efficiency. It only requires 1 hour of training on an RTX 4090 to enable a quadruped robot to traverse any terrain under any disturbances. A wealth of real-world experiments demonstrates its agility, even in high-difficulty tasks and cases never occurred during the training process, revealing remarkable open-world generalizability.

Robotics,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning,Systems and Control
Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC

Yiyu Chen,Quan Nguyen

2024-03-20

Abstract:In the context of legged robots, adaptive behavior involves adaptive balancing and adaptive swing foot reflection. While adaptive balancing counteracts perturbations to the robot, adaptive swing foot reflection helps the robot to navigate intricate terrains without foot entrapment. In this paper, we manage to bring both aspects of adaptive behavior to quadruped locomotion by combining RL and MPC while improving the robustness and agility of blind legged locomotion. This integration leverages MPC's strength in predictive capabilities and RL's adeptness in drawing from past experiences. Unlike traditional locomotion controls that separate stance foot control and swing foot trajectory, our innovative approach unifies them, addressing their lack of synchronization. At the heart of our contribution is the synthesis of stance foot control with swing foot reflection, improving agility and robustness in locomotion with adaptive behavior. A hallmark of our approach is robust blind stair climbing through swing foot reflection. Moreover, we intentionally designed the learning module as a general plugin for different robot platforms. We trained the policy and implemented our approach on the Unitree A1 robot, achieving impressive results: a peak turn rate of 8.5 rad/s, a peak running speed of 3 m/s, and steering at a speed of 2.5 m/s. Remarkably, this framework also allows the robot to maintain stable locomotion while bearing an unexpected load of 10 kg, or 83\% of its body mass. We further demonstrate the generalizability and robustness of the same policy where it realizes zero-shot transfer to different robot platforms like Go1 and AlienGo robots for load carrying. Code is made available for the use of the research community at <a class="link-external link-https" href="https://github.com/DRCL-USC/RL_augmented_MPC.git" rel="external noopener nofollow">this https URL</a>

Robotics
Generalized Animal Imitator: Agile Locomotion with Versatile Motion Prior

Ruihan Yang,Zhuoqun Chen,Jianhan Ma,Chongyi Zheng,Yiyu Chen,Quan Nguyen,Xiaolong Wang

2024-04-21

Abstract:The agility of animals, particularly in complex activities such as running, turning, jumping, and backflipping, stands as an exemplar for robotic system design. Transferring this suite of behaviors to legged robotic systems introduces essential inquiries: How can a robot be trained to learn multiple locomotion behaviors simultaneously? How can the robot execute these tasks with a smooth transition? How to integrate these skills for wide-range applications? This paper introduces the Versatile Instructable Motion prior (VIM) - a Reinforcement Learning framework designed to incorporate a range of agile locomotion tasks suitable for advanced robotic applications. Our framework enables legged robots to learn diverse agile low-level skills by imitating animal motions and manually designed motions. Our Functionality reward guides the robot's ability to adopt varied skills, and our Stylization reward ensures that robot motions align with reference motions. Our evaluations of the VIM framework span both simulation environments and real-world deployment. To the best of our knowledge, this is the first work that allows a robot to concurrently learn diverse agile locomotion skills using a single learning-based controller in the real world. Further details and supportive media can be found at our project site: <a class="link-external link-https" href="https://rchalyang.github.io/VIM" rel="external noopener nofollow">this https URL</a>

Robotics,Artificial Intelligence,Machine Learning
A Real-Time Planning and Control Framework for Robust and Dynamic Quadrupedal Locomotion

Jun Li,Haibo Gao,Yuhui Wan,Haitao Yu,Chengxu Zhou

DOI: https://doi.org/10.1007/s42235-023-00347-9

IF: 2.995

2023-02-10

Journal of Bionic Engineering

Abstract:Legged locomotion poses significant challenges due to its nonlinear, underactuated and hybrid dynamic properties. These challenges are exacerbated by the high-speed motion and presence of aerial phases in dynamic legged locomotion, which highlights the requirement for online planning based on current states to cope with uncertainty and disturbances. This article proposes a real-time planning and control framework integrating motion planning and whole-body control. In the framework, the designed motion planner allows a wider body rotation range and fast reactive behaviors based on the 3-D single rigid body model. In addition, the combination of a Bézier curve based trajectory interpolator and a heuristic-based foothold planner helps generate continuous and smooth foot trajectories. The developed whole-body controller uses hierarchical quadratic optimization coupled with the full system dynamics, which ensures tasks are prioritized based on importance and joint commands are physically feasible. The performance of the framework is successfully validated in experiments with a torque-controlled quadrupedal robot for generating dynamic motions.

robotics,engineering, multidisciplinary,materials science, biomaterials
Rapid locomotion via reinforcement learning

Gabriel B. Margolis,Ge Yang,Kartik Paigwar,Tao Chen,Pulkit Agrawal

DOI: https://doi.org/10.1177/02783649231224053

2024-01-03

The International Journal of Robotics Research

Abstract:The International Journal of Robotics Research, Ahead of Print. Agile maneuvers such as sprinting and high-speed turning in the wild are challenging for legged robots. We present an end-to-end learned controller that achieves record agility for the MIT Mini Cheetah, sustaining speeds up to 3.9 m/s. This system runs and turns fast on natural terrains like grass, ice, and gravel and responds robustly to disturbances. Our controller is a neural network trained in simulation via reinforcement learning and transferred to the real world. The two key components are (i) an adaptive curriculum on velocity commands and (ii) an online system identification strategy for sim-to-real transfer. Videos of the robot's behaviors are available at https://agility.csail.mit.edu/.

robotics
Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion

Zipeng Fu,Xuxin Cheng,Deepak Pathak

DOI: https://doi.org/10.48550/arXiv.2210.10044

2022-10-19

Abstract:An attached arm can significantly increase the applicability of legged robots to several mobile manipulation tasks that are not possible for the wheeled or tracked counterparts. The standard hierarchical control pipeline for such legged manipulators is to decouple the controller into that of manipulation and locomotion. However, this is ineffective. It requires immense engineering to support coordination between the arm and legs, and error can propagate across modules causing non-smooth unnatural motions. It is also biological implausible given evidence for strong motor synergies across limbs. In this work, we propose to learn a unified policy for whole-body control of a legged manipulator using reinforcement learning. We propose Regularized Online Adaptation to bridge the Sim2Real gap for high-DoF control, and Advantage Mixing exploiting the causal dependency in the action space to overcome local minima during training the whole-body system. We also present a simple design for a low-cost legged manipulator, and find that our unified policy can demonstrate dynamic and agile behaviors across several task setups. Videos are at <a class="link-external link-https" href="https://maniploco.github.io" rel="external noopener nofollow">this https URL</a>

Robotics,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning,Systems and Control
Tello Leg: The Study of Design Principles and Metrics for Dynamic Humanoid Robots

Youngwoo Sim,Joao Ramos

DOI: https://doi.org/10.48550/arXiv.2203.00644

2022-03-02

Abstract:To be useful tools in real scenarios, humanoid robots must realize tasks dynamically. This means that they must be capable of applying substantial forces, rapidly swinging their limbs, and also mitigating impacts that may occur during the motion. Towards creating capable humanoids, this letter presents the leg of the robot TELLO and demonstrates how it embodies two new fundamental design concepts for dynamic legged robots. The limbs follows the principles of: (i) Cooperative Actuation (CA), by combining motors in differential configurations to increase the force capability of the limb. We demonstrate that the CA configuration requires half the motor torque to perform a jump in comparison to conventional serial design configurations. And (ii) proximal actuation, by placing heavy motors near the body to reduce the inertia of the limb. To quantify the effect of motor placement on the robot's dynamics, we introduce a novel metric entitle Centroidal Inertia Isotropy (CII). We show that the design of state-of-the-art dynamic legged robots empirically increase the CII to improve agility and facilitate model-based control. We hope this metric will enable a quantifiable way to design these machines in the future.

Robotics
Motion Planning for a Legged Robot with Dynamic Characteristics

Liu,Yang,Chen,Zhong,Gao

DOI: https://doi.org/10.3390/s24186070

IF: 3.9

2024-09-20

Sensors

Abstract:Legged soccer robots present a significant challenge in robotics owing to the need for seamless integration of perception, manipulation, and dynamic movement. While existing models often depend on external perception or static techniques, our study aims to develop a robot with dynamic and untethered capabilities. We have introduced a motion planner that allows the robot to excel in dynamic shooting and dribbling. Initially, it identifies and predicts the position of the ball using a rolling model. The robot then pursues the ball, using a novel optimization-based cycle planner, continuously adjusting its gait cycle. This enables the robot to kick without stopping its forward motion near the ball. Each leg is assigned a specific role (stance, swing, pre-kick, or kick), as determined by a gait scheduler. Different leg controllers were used for tailored tiptoe trajectory planning and control. We validated our approach using real-world penalty shot experiments (5 out of 12 successful), cycle adjustment tests (11 out of 12 successful), and dynamic dribbling assessments. The results demonstrate that legged robots can overcome onboard capability limitations and achieve dynamic mobility and manipulation.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Perception-Driven Learning of High-Dynamic Jumping Motions for Single-Legged Robots

Nengxiang Sun,Fei Meng,Sai Gu,Botao Liu,Xuechao Chen,Zhangguo Yu,Qiang Huang

DOI: https://doi.org/10.1007/s42235-024-00541-3

IF: 2.995

2024-05-31

Journal of Bionic Engineering

Abstract:Legged robots show great potential for high-dynamic motions in continuous interaction with the physical environment, yet achieving animal-like agility remains significant challenges. Legged animals usually predict and plan their next locomotion by combining high-dimensional information from proprioception and exteroception, and adjust the stiffness of the body's skeletal muscle system to adapt to the current environment. Traditional control methods have limitations in handling high-dimensional state information or complex robot motion that are difficult to plan manually, and Deep Reinforcement Learning (DRL) algorithms provide new solutions to robot motioncontrol problems. Inspired by biomimetics theory, we propose a perception-driven high-dynamic jump adaptive learning algorithm by combining DRL algorithms with Virtual Model Control (VMC) method. The robot will be fully trained in simulation to explore its motion potential by learning the factors related to continuous jumping while knowing its real-time jumping height. The policy trained in simulation is successfully deployed on the bio-inspired single-legged robot testing platform without further adjustments. Experimental results show that the robot can achieve continuous and ideal vertical jumping motion through simple training

robotics,engineering, multidisciplinary,materials science, biomaterials
Learning to Control Emulated Muscles in Real Robots: Towards Exploiting Bio-Inspired Actuator Morphology

Pierre Schumacher,Lorenz Krause,Jan Schneider,Dieter Büchler,Georg Martius,Daniel Haeufle

DOI: https://doi.org/10.48550/arXiv.2402.05371

IF: 3.7

2024-02-08

Robotics

Abstract:Recent studies have demonstrated the immense potential of exploiting muscle actuator morphology for natural and robust movement -- in simulation. A validation on real robotic hardware is yet missing. In this study, we emulate muscle actuator properties on hardware in real-time, taking advantage of modern and affordable electric motors. We demonstrate that our setup can emulate a simplified muscle model on a real robot while being controlled by a learned policy. We improve upon an existing muscle model by deriving a damping rule that ensures that the model is not only performant and stable but also tuneable for the real hardware. Our policies are trained by reinforcement learning entirely in simulation, where we show that previously reported benefits of muscles extend to the case of quadruped locomotion and hopping: the learned policies are more robust and exhibit more regular gaits. Finally, we confirm that the learned policies can be executed on real hardware and show that sim-to-real transfer with real-time emulated muscles on a quadruped robot is possible. These results show that artificial muscles can be highly beneficial actuators for future generations of robust legged robots.

Learning agile and dynamic motor skills for legged robots

Learning and Adapting Agile Locomotion Skills by Transferring Experience

An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement

Learning Whole-body Motor Skills for Humanoids

Learning Agile Locomotion Skills with a Mentor

A learning-based control pipeline for generic motor skills for quadruped robots

Deep Reinforcement Learning Based Co-Optimization of Morphology and Gait for Small-Scale Legged Robot

Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion

Dynamic Bipedal Maneuvers through Sim-to-Real Reinforcement Learning

Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response

Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC

Generalized Animal Imitator: Agile Locomotion with Versatile Motion Prior

A Real-Time Planning and Control Framework for Robust and Dynamic Quadrupedal Locomotion

Rapid locomotion via reinforcement learning

Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion

Tello Leg: The Study of Design Principles and Metrics for Dynamic Humanoid Robots

Motion Planning for a Legged Robot with Dynamic Characteristics

Perception-Driven Learning of High-Dynamic Jumping Motions for Single-Legged Robots

Learning to Control Emulated Muscles in Real Robots: Towards Exploiting Bio-Inspired Actuator Morphology