Abstract:The automatic design of robots has existed for 30 years but has been constricted by serial non-differentiable design evaluations, premature convergence to simple bodies or clumsy behaviors, and a lack of sim2real transfer to physical machines. Thus, here we employ massively-parallel differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans and return a fitness score for each design based on the performance of its fully optimized behavior. Non-differentiable changes to the mechanical structure of each robot in the population -- mutations that rearrange, combine, add, or remove body parts -- were applied by a genetic algorithm in an outer loop of search, generating a continuous flow of novel morphologies with highly-coordinated and graceful behaviors honed by gradient descent. This enabled the exploration of several orders-of-magnitude more designs than all previous methods, despite the fact that robots here have the potential to be much more complex, in terms of number of independent motors, than those in prior studies. We found that evolution reliably produces ``increasingly differentiable'' robots: body plans that smooth the loss landscape in which learning operates and thereby provide better training paths toward performant behaviors. Finally, one of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior. This provides a cyberphysical platform to investigate the relationship between evolution and learning in biological systems and broadens our understanding of how a robot's physical structure can influence the ability to train policies for it. Videos and code at

Bettering operation of Robots by learning

Iterative Learning Control Of Varying Trajectories For Robot Manipulators

Learning control theory for robotic motion

Optimizing the Execution of Dynamic Robot Movements With Learning Control

Robot Policy Improvement With Natural Evolution Strategies for Stable Nonlinear Dynamical System

Iterative learning control for integrated system of robot and machine tool

Evolution and learning in differentiable robots

On Designing a Learning Robot: Improving Morphology for Enhanced Task Performance and Learning

Towards Modeling and Influencing the Dynamics of Human Learning

Neural-Learning Trajectory Tracking Control of Flexible-Joint Robot Manipulators with Unknown Dynamics

Iterative Learning of Human Partner’s Desired Trajectory for Proactive Human–robot Collaboration

High-Precise Robot Arm Manipulation based on Online Iterative Learning and Forward Simulation with Positioning Error Below End-Effector Physical Minimum Displacement

Learning Soft Robot Dynamics using Differentiable Kalman Filters and Spatio-Temporal Embeddings

Incremental Learning of Goal-Directed Actions in a Dynamic Environment by a Robot Using Active Inference

Industrial robot accurate trajectory generation by nested loop iterative learning control

Robot Learning Method for Human-like Arm Skills Based on the Hybrid Primitive Framework

Optimizing Execution of Dynamic Goal-Directed Robot Movements with Learning Control

Neural-Dynamics Optimization and Repetitive Learning Control for Robotic Leg Prostheses

Learning Variable Admittance Control for Human-Robot Collaborative Manipulation

Autonomously Achieving Bipedal Locomotion Skill Via Hierarchical Motion Modelling.