Abstract:We initiate a line of investigation into biological neural networks from an algorithmic perspective. We develop a simplified but biologically plausible model for distributed computation in stochastic spiking neural networks and study tradeoffs between computation time and network complexity in this model. Our aim is to abstract real neural networks in a way that, while not capturing all interesting features, preserves high-level behavior and allows us to make biologically relevant conclusions. In this paper, we focus on the important `winner-take-all' (WTA) problem, which is analogous to a neural leader election unit: a network consisting of $n$ input neurons and $n$ corresponding output neurons must converge to a state in which a single output corresponding to a firing input (the `winner') fires, while all other outputs remain silent. Neural circuits for WTA rely on inhibitory neurons, which suppress the activity of competing outputs and drive the network towards a converged state with a single firing winner. We attempt to understand how the number of inhibitors used affects network convergence time. We show that it is possible to significantly outperform naive WTA constructions through a more refined use of inhibition, solving the problem in $O(\theta)$ rounds in expectation with just $O(\log^{1/\theta} n)$ inhibitors for any $\theta$. An alternative construction gives convergence in $O(\log^{1/\theta} n)$ rounds with $O(\theta)$ inhibitors. We compliment these upper bounds with our main technical contribution, a nearly matching lower bound for networks using $\ge \log\log n$ inhibitors. Our lower bound uses familiar indistinguishability and locality arguments from distributed computing theory. It lets us derive a number of interesting conclusions about the structure of any network solving WTA with good probability, and the use of randomness and inhibition within such a network.

Distributed Winner-Take-All in Dynamic Networks.

Distributed Consensus of Multi-Agent Systems with Nonlinear Dynamics Via Adaptive Intermittent Control

Consensus under Bounded Noise in Discrete Network Systems: an Algorithm with Fast Convergence and High Accuracy.

An Expanded Distributed Algorithm for Dynamic Resource Allocation over Strongly Connected Topologies

Distributed Gradient Method for Neural Network-based Constrained $K$-Winners-take-all

Distributed Stochastic Algorithm for Global Optimization in Networked System

Cooperative Control Based on Distributed Estimation of Network Connectivity

Computational Tradeoffs in Biological Neural Networks: Self-Stabilizing Winner-Take-All Networks

Distributed Linear Equations over Random Networks

Competitive Dynamics on Complex Networks

Finite-Time Consensus Problems for Networks of Dynamic Agents

A Winner-Take-All Neural Networks of N Linear Threshold Neurons Without Self-Excitatory Connections

Sublinear Algorithms in T-Interval Dynamic Networks

Distributed Solver for Discrete-Time Lyapunov Equations over Dynamic Networks with Linear Convergence Rate

Distributed Dynamic Task Allocation for Moving Target Tracking of Networked Mobile Robots Using $k$-WTA Network

Neural networks letter: A new approach to the stability analysis of continuous-time distributed consensus algorithms

Distributed Decisions on Optimal Load Balancing in Loss Networks

Fully Distributed Continuous-Time Algorithm for Nonconvex Optimization over Unbalanced Directed Networks

Game-Theoretic Distributed Algorithm for Virtual Tdm-Pon Formulation to Realize Adaptive Data Transmission in C-Rans

Distributed adaptive consensus algorithm for networked Euler-Lagrange systems

Distributed fixed-point algorithms for dynamic convex optimization over decentralized and unbalanced wireless networks