Abstract:A key challenge for neural modeling is to explain how a continuous stream of multimodal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-fire neurons in real time. We propose a new computational model for real-time computing on time-varying input that provides an alternative to paradigms based on Turing machines or attractor neural networks. It does not require a task-dependent construction of neural circuits. Instead, it is based on principles of high-dimensional dynamical systems in combination with statistical learning theory and can be implemented on generic evolved or found recurrent circuitry. It is shown that the inherent transient dynamics of the high-dimensional dynamical system formed by a sufficiently large and heterogeneous neural circuit may serve as universal analog fading memory. Readout neurons can learn to extract in real time from the current state of such recurrent neural circuit information about current and past inputs that may be needed for diverse tasks. Stable internal states are not required for giving a stable output, since transient internal states can be transformed by readout neurons into stable target outputs due to the high dimensionality of the dynamical system. Our approach is based on a rigorous computational model, the liquid state machine, that, unlike Turing machines, does not require sequential transitions between well-defined discrete internal states. It is supported, as the Turing machine is, by rigorous mathematical results that predict universal computational power under idealized conditions, but for the biologically more realistic scenario of real-time processing of time-varying inputs. Our approach provides new perspectives for the interpretation of neural coding, the design of experiments and data analysis in neurophysiology, and the solution of problems in robotics and neurotechnology.

On the Computational Power of Neural Nets

Lower Bounds for the Computational Power of Networks of Spiking Neurons

On the Tensor Representation and Algebraic Homomorphism of the Neural State Turing Machine

State dependent computation using coupled recurrent networks

Real-time computing without stable states: a new framework for neural computation based on perturbations

On the computational power and complexity of Spiking Neural Networks

Multiscale Computations on Neural Networks: From the Individual Neuron Interactions to the Macroscopic-Level Analysis

A continuous time stochastic model for biological neural nets

Sub-realtime simulation of a neuronal network of natural density

The Computational Power of Distributed Shared-Memory Models with Bounded-Size Registers

Neural waves and computation in a neural net model I: Convolutional hierarchies

A theoretical basis for efficient computations with noisy spiking neurons

Neuromorphic Computing is Turing-Complete

'Viral' Turing Machines, Computation from Noise and Combinatorial Hierarchies

Computational Tradeoffs in Biological Neural Networks: Self-Stabilizing Winner-Take-All Networks

Binary Operations on Neuromorphic Hardware with Application to Linear Algebraic Operations and Stochastic Equations

Computation with Sequences in a Model of the Brain

Exponential scaling of neural algorithms - a future beyond Moore's Law?

Neural Computations in A Dynamical System with Multiple Time Scales

The Computational Power of Dynamic Bayesian Networks

Simulation of Large Scale Neural Networks for Evaluation Applications