Abstract:We initiate a line of investigation into biological neural networks from an algorithmic perspective. We develop a simplified but biologically plausible model for distributed computation in stochastic spiking neural networks and study tradeoffs between computation time and network complexity in this model. Our aim is to abstract real neural networks in a way that, while not capturing all interesting features, preserves high-level behavior and allows us to make biologically relevant conclusions. In this paper, we focus on the important `winner-take-all' (WTA) problem, which is analogous to a neural leader election unit: a network consisting of $n$ input neurons and $n$ corresponding output neurons must converge to a state in which a single output corresponding to a firing input (the `winner') fires, while all other outputs remain silent. Neural circuits for WTA rely on inhibitory neurons, which suppress the activity of competing outputs and drive the network towards a converged state with a single firing winner. We attempt to understand how the number of inhibitors used affects network convergence time. We show that it is possible to significantly outperform naive WTA constructions through a more refined use of inhibition, solving the problem in $O(\theta)$ rounds in expectation with just $O(\log^{1/\theta} n)$ inhibitors for any $\theta$. An alternative construction gives convergence in $O(\log^{1/\theta} n)$ rounds with $O(\theta)$ inhibitors. We compliment these upper bounds with our main technical contribution, a nearly matching lower bound for networks using $\ge \log\log n$ inhibitors. Our lower bound uses familiar indistinguishability and locality arguments from distributed computing theory. It lets us derive a number of interesting conclusions about the structure of any network solving WTA with good probability, and the use of randomness and inhibition within such a network.

Synthesis of a K-Winners-take-all Neural Network Using Linear Programming with Bounded Variables

Neural Network Solution for General Nonlinear Optimization Problems

Solving the K-Winners-take-all Problem and the Oligopoly Cournot-Nash Equilibrium Problem Using the General Projection Neural Networks.

A new recurrent neural network for solving convex quadratic programming problems with an application to the κ-winners-take-all problem

An Improved Dual Neural Network for Solving A Class of Quadratic Programming Problems and Its K-Winners-Take-All Application

A Neural Network Model For General Minimax Problem

k-Winners-Take-All Ensemble Neural Network

Structured Regularization In Barrier Nlp For Optimization Models With Dependent Constraints

A discrete-time neural network for optimization problems with hybrid constraints.

A Novel Neural Network for Nonlinear Convex Programming

Barrier NLP Methods with Structured Regularization for Optimization of Degenerate Optimization Problems

WANCO: Weak Adversarial Networks for Constrained Optimization problems

Can Deep Learning Solve Parametric Mathematical Programming? an Application to 0–1 Linear Programming Through Image Representation

A Model for Quantum Stochastic Absorption in Absorbing Disordered Media

A One-Layer Dual Recurrent Neural Network with a Heaviside Step Activation Function for Linear Programming with Its Linear Assignment Application

Computational Tradeoffs in Biological Neural Networks: Self-Stabilizing Winner-Take-All Networks

KKT-Informed Neural Network

A Stochastic Composite Augmented Lagrangian Method for Reinforcement Learning.

Solving for a Quadratic Programming with a Quadratic Constraint Based on a Neural Network Frame

Linear programming using diagonal linear networks