Abstract:Standard approaches to controlling dynamical systems involve biologically implausible steps such as backpropagation of errors or intermediate model-based system representations. Recent advances in machine learning have shown that "imperfect" feedback of errors during training can yield test performance that is similar to using full backpropagated errors, provided that the two error signals are at least somewhat aligned. Inspired by such methods, we introduce an iterative, spatiotemporally local protocol to learn driving forces and control non-equilibrium dynamical systems using imperfect feedback signals. We present numerical experiments and theoretical justification for several examples. For systems in conservative force fields that are driven by external time-dependent protocols, our update rules resemble a dynamical version of contrastive divergence. We appeal to linear response theory to establish that our imperfect update rules are locally convergent for these conservative systems. For systems evolving under non-conservative dynamics, we derive a new theoretical result that makes possible the control of non-equilibrium steady-state probabilities through simple local update rules. Finally, we show that similar local update rules can also solve dynamical control problems for non-conservative systems, and we illustrate this in the non-trivial example of active nematics. Our updates allow learning spatiotemporal activity fields that pull topological defects along desired trajectories in the active nematic fluid. These imperfect feedback methods are information efficient and in principle biologically plausible, and they can help extend recent methods of decentralized training for physical materials into dynamical settings.

Iterative optimisation in dynamic environments via local pursuit

Human-in-the-loop Distributed Cooperative Tracking Control with Applications to Autonomous Ground Vehicles: A Data-Driven Mixed Iteration Approach

Topology-Driven Parallel Trajectory Optimization in Dynamic Environments

Collision Avoidance using Iterative Dynamic and Nonlinear Programming with Adaptive Grid Refinements

Optimal Cooperative Maneuver Planning for Multiple Nonholonomic Robots in a Tiny Environment Via Adaptive-Scaling Constrained Optimization

Optimal control of interacting active particles on complex landscapes

Approximate Optimal Strategy for Multiagent System Pursuit–Evasion Game

Decentralized trajectory optimization for multi-agent exploration

Data-driven Optimal Tracking Control of Discrete-Time Multi-Agent Systems with Two-Stage Policy Iteration Algorithm.

Nature-inspired dynamic control for pursuit-evasion of robots

Neighboring Optimal Control for Periodic Tasks for Systems with Discontinuous Dynamics

Globally Guided Trajectory Planning in Dynamic Environments

Online Trajectory Optimization for Persistent Monitoring Problems in Partitioned Environments

Trajectory Optimization Algorithm Studies

Online Decentralized Receding Horizon Trajectory Optimization for Multi-Robot systems

Optimization-based Trajectory Tracking Approach for Multi-rotor Aerial Vehicles in Unknown Environments

Iterative learning controllers with time-varying gains for large-scale industrial processes to track trajectories with different magnitudes

Optimizing the Execution of Dynamic Robot Movements With Learning Control

Learning to control non-equilibrium dynamics using local imperfect gradients

Data-efficient Learning-Based Iterative Optimization Method with Time-Varying Prediction Horizon for Multi-Agent Collaboration