Abstract:The automatic design of robots has existed for 30 years but has been constricted by serial non-differentiable design evaluations, premature convergence to simple bodies or clumsy behaviors, and a lack of sim2real transfer to physical machines. Thus, here we employ massively-parallel differentiable simulations to rapidly and simultaneously optimize individual neural control of behavior across a large population of candidate body plans and return a fitness score for each design based on the performance of its fully optimized behavior. Non-differentiable changes to the mechanical structure of each robot in the population -- mutations that rearrange, combine, add, or remove body parts -- were applied by a genetic algorithm in an outer loop of search, generating a continuous flow of novel morphologies with highly-coordinated and graceful behaviors honed by gradient descent. This enabled the exploration of several orders-of-magnitude more designs than all previous methods, despite the fact that robots here have the potential to be much more complex, in terms of number of independent motors, than those in prior studies. We found that evolution reliably produces ``increasingly differentiable'' robots: body plans that smooth the loss landscape in which learning operates and thereby provide better training paths toward performant behaviors. Finally, one of the highly differentiable morphologies discovered in simulation was realized as a physical robot and shown to retain its optimized behavior. This provides a cyberphysical platform to investigate the relationship between evolution and learning in biological systems and broadens our understanding of how a robot's physical structure can influence the ability to train policies for it. Videos and code at

Evolution of Cooperative Ensemble Neural Network Controller for Autonomous Mobile Robots

Cooperative Flocking And Learning In Multi-Robot Systems For Predator Avoidance

C^2:Co-design of Robots via Concurrent Networks Coupling Online and Offline Reinforcement Learning

Parallel Learning Evolutionary Algorithm Based On Neural Network Ensemble

Evaluation of Frameworks That Combine Evolution and Learning to Design Robots in Complex Morphological Spaces

odNEAT: An Algorithm for Decentralised Online Evolution of Robotic Controllers

Towards Behavior Control for Evolutionary Robot Based on Rl with Enn

The Evolution of Cooperative Behaviours in Physically Heterogeneous Multi-Robot Systems

Evolving homogeneous neurocontrollers for a group of heterogeneous robots: coordinated motion, cooperation, and acoustic communication

GA-Aided Elman Neural Network Controller For Behavior-Based Robot

Evolutionary Reinforcement Learning via Cooperative Coevolution

A neural network-based exploratory learning and motor planning system for co-robots

A comparison of controller architectures and learning mechanisms for arbitrary robot morphologies

Elman Neural Network Controller For Behavior-Based Robot

Evolution and learning in differentiable robots

Evolutionary Approach To Navigation Learning In Autonomous Mobile Robot

Adapting Real Mobile Robots to Complex Environments Using a Pattern Association Network Controller (PAN-C).

Self-Organization of Spiking Neural Network Generating Autonomous Behavior in a Real Mobile Robot

Distributed Robust Learning based Formation Control of Mobile Robots based on Bioinspired Neural Dynamics

An Investigation of the Factors Influencing Evolutionary Dynamics in the Joint Evolution of Robot Body and Control

A method based on evolutionary neural network for mobile robot path planning