Whole-body simulation of realistic fruit fly locomotion with deep reinforcement learning

Roman Vaxenburg,Igor Siwanowicz,Josh Merel,Alice A. Robie,Carmen Morrow,Guido Novati,Zinovia Stefanidi,Gwyneth M. Card,Michael B. Reiser,Matthew M. Botvinick,Kristin M. Branson,Yuval Tassa,Srinivas C. Turaga

DOI: https://doi.org/10.1101/2024.03.11.584515

2024-03-14

Abstract:The body of an animal determines how the nervous system produces behavior. Therefore, detailed modeling of the neural control of sensorimotor behavior requires a detailed model of the body. Here we contribute an anatomically-detailed biomechanical whole-body model of the fruit fly in the MuJoCo physics engine. Our model is general-purpose, enabling the simulation of diverse fly behaviors, both on land and in the air. We demonstrate the generality of our model by simulating realistic locomotion, both flight and walking. To support these behaviors, we have extended MuJoCo with phenomenological models of fluid forces and adhesion forces. Through data-driven end-to-end reinforcement learning, we demonstrate that these advances enable the training of neural network controllers capable of realistic locomotion along complex trajectories based on high-level steering control signals. With a visually guided flight task, we demonstrate a neural controller that can use the vision sensors of the body model to control and steer flight. Our project is an open-source platform for modeling neural control of sensorimotor behavior in an embodied context.

Animal Behavior and Cognition

What problem does this paper attempt to address?

The main objective of this paper is to develop a full-body simulation model capable of mimicking the real movement behavior of fruit flies (*Drosophila melanogaster*) and to achieve this simulation by training a neural network controller through deep reinforcement learning methods. Specifically, the research team constructed a detailed, anatomically accurate biomechanical model and applied it to simulate the walking and flying behaviors of fruit flies.

To achieve this goal, the researchers first used high-resolution confocal microscopy imaging technology to construct the body model of the fruit fly. Then, they extended the MuJoCo physics engine by adding fluid dynamics and adhesion force simulation capabilities to more accurately simulate the aerodynamics generated by wing flapping during flight and the adhesion force between the feet and the ground during walking. With these improvements, they were able to use end-to-end reinforcement learning methods to train the neural network controller, enabling the model fruit fly to execute complex movement trajectories based on high-level navigation instructions.

The research team validated the effectiveness of the model, demonstrating that the model fruit fly could mimic the walking and flying behaviors of real fruit flies, including moving along complex natural trajectories. Additionally, they showcased how to use reinforcement learning to train the model fruit fly to complete visually guided flight tasks. The entire model and related research findings are open-source, providing a foundational platform for future research on how the fruit fly nervous system controls sensorimotor behavior.

In summary, this paper aims to address the problem of how to establish a fruit fly model rich in anatomical details and capable of biomechanical simulation, and to use reinforcement learning to train a controller to achieve natural movement behavior in the model.