Optimal control of spiking neural networks

Tiago Costa,Juan R. Castineiras de Saa,Alfonso Renart
DOI: https://doi.org/10.1101/2024.10.02.616330
2024-10-03
Abstract:Control theory provides a natural language to describe multi-areal interactions and flexible cognitive tasks such as covert attention or brain-machine interface (BMI) experiments, which require finding adequate inputs to a local circuit in order to steer its dynamics in a context-dependent manner. In optimal control, the target dynamics should maximize a notion of long-term value along trajectories, possibly subject to control costs. Because this problem is, in general, not tractable, current approaches to the control of networks mostly consider simplified settings (e.g., variations of the Linear-Quadratic Regulator). Here, we present a mathematical framework for optimal control of recurrent networks of stochastic spiking neurons with low-rank connectivity. An essential ingredient is a control-cost that penalizes deviations from the default dynamics of the network (specified by its recurrent connections), which motivates the controller to use the default dynamics as much as possible. We derive a Bellman Equation that specifies a Value function over the low-dimensional network state (LDS), and a corresponding optimal control input. The optimal control law takes the form of a feedback controller that provides external excitatory (inhibitory) synaptic input to neurons in the recurrent network if their spiking activity tends to move the LDS towards regions of higher (lower) Value. We use our theory to study the problem of steering the state of the network towards particular terminal regions which can lie either in or out of regions in the LDS with slow dynamics, in analogy to standard BMI experiments. Our results provide the foundation of a novel approach with broad applicability that unifies bottom-up and top-down perspectives on neural computation.
Neuroscience
What problem does this paper attempt to address?
The paper attempts to address the problem of achieving optimal control in random spiking neural networks. Specifically, the researchers propose a mathematical framework for controlling recurrent spiking neural networks with low-rank connectivity. The key to this framework is the introduction of a control cost to penalize deviations from the network's default dynamics, thereby incentivizing the controller to utilize the network's default dynamics as much as possible. The study derives a Bellman equation to specify the value function and its corresponding optimal control input under low-dimensional network states. Additionally, the theory is used to investigate how to steer the network state towards specific target regions, which may lie within or outside the low-dynamic regions, similar to scenarios in brain-machine interface (BMI) experiments. The main contribution of the paper is providing a new methodological foundation that unifies bottom-up and top-down perspectives of neural computation and is applicable to a wide range of scenarios. Through this approach, the researchers explore different manifestations of network dynamics under varying control gain β values, particularly noting that in low control gain scenarios, the network dynamics become richer, demonstrating how to optimally leverage default dynamics to accomplish given tasks.