Abstract:Consider a reinforcement learning problem where an agent has access to a very large amount of information about the environment, but it can only take very few actions to accomplish its task and to maximize its reward. Evidently, the main problem for the agent is to learn a map from a very high-dimensional space (which represents its environment) to a very low-dimensional space (which represents its actions). The high-to-low dimensional map implies that most of the information about the environment is irrelevant for the actions to be taken, and only a small fraction of information is relevant. In this paper we argue that the relevant information need not be learned by brute force (which is the standard approach), but can be identified from the intrinsic symmetries of the system. We analyze in details a reinforcement learning problem of autonomous driving, where the corresponding symmetry is the Galilean symmetry, and argue that the learning task can be accomplished with very few relevant parameters, or, more precisely, invariants. For a numerical demonstration, we show that the autonomous vehicles (which we call autonomous particles since they describe very primitive vehicles) need only four relevant invariants to learn how to drive very well without colliding with other particles. The simple model can be easily generalized to include different types of particles (e.g. for cars, for pedestrians, for buildings, for road signs, etc.) with different types of relevant invariants describing interactions between them. We also argue that there must exist a field theory description of the learning system where autonomous particles would be described by fermionic degrees of freedom and interactions mediated by the relevant invariants would be described by bosonic degrees of freedom.

Smart inertial particles

Active particles using reinforcement learning to navigate in complex motility landscapes

Steering undulatory micro-swimmers in a fluid flow through reinforcement learning

Physics-Guided Actor-Critic Reinforcement Learning for Swimming in Turbulence

Autonomous navigation of smart microswimmers in non-uniform flow fields

Reinforcement learning of optimal active particle navigation

Learning Efficient Navigation in Vortical Flow Fields

Autonomous particles

Swimming Strategy of Settling Elongated Micro-Swimmers by Reinforcement Learning

Smart navigation through a rotating barrier: Deep reinforcement learning with application to size-based separation of active microagents

Learning how to find targets in the micro-world: The case of intermittent active Brownian particles

A numerical study of fish adaption behaviors in complex environments with a deep reinforcement learning and immersed boundary–lattice Boltzmann method

Physics-informed reinforcement learning for motion control of a fish-like swimming robot

Hierarchical Control of Smart Particle Swarms

Optimal Control Strategies for Active Particle Navigation

A numerical simulation research on fish adaption behavior based on deep reinforcement learning and fluid–structure coupling: Implementation of the “perceive-feedback-memory” control system

Smart active particles learn and transcend bacterial foraging strategies

Hunting active Brownian particles: Learning optimal behavior

Efficient collective swimming by harnessing vortices through deep reinforcement learning

Position and Attitude Tracking Control of a Biomimetic Underwater Vehicle Via Deep Reinforcement Learning