Abstract:This article studies the dynamics of the mean-field approximation of continuous random networks. These networks are stochastic integrodifferential equations driven by Gaussian noise. The kernels in the integral operators are realizations of generalized Gaussian random variables. The equation controls the time evolution of a macroscopic state interpreted as neural activity, which depends on position and time. The position is an element of a measurable space. Such a network corresponds to a statistical field theory (STF) given by a momenta-generating functional. Discrete versions of the mentioned networks appeared in spin glasses and as models of artificial neural networks (NNs). Each of these discrete networks corresponds to a lattice SFT, where the action contains a finite number of neurons and two scalar fields for each neuron. Recently, it has been proposed that these networks can be used as models for deep learning. In this application, the number of neurons is astronomical; consequently, continuous models are required. In this article, we develop mathematically rigorous, continuous versions of the mean-field theory approximation and the double-copy system that allow us to derive a condition for the criticality of continuous stochastic networks via the largest Lyapunov exponent. We use two basic architectures; in the first one, the space of neurons is the real line, and then the neurons are organized in one layer; in the second one, the space of neurons is the p-adic line, and then the neurons are organized in an infinite, fractal, tree-like structure. Networks with both types of architectures exhibit a type edge of the chaos organization.

Mean field analysis of neural networks: A central limit theorem

Mean Field Analysis of Neural Networks: A Law of Large Numbers

Dynamic Mean-Field Theory for Continuous Random Networks

Fluctuations for mean field limits of interacting systems of spiking neurons

A Mean Field Theory of Batch Normalization

Laws of Large Numbers and Langevin Approximations for Stochastic Neural Field Equations

Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective

The mean-field Limit of sparse networks of integrate and fire neurons

Mean field limits for nonlinear spatially extended hawkes processes with exponential memory kernels

Central Limit Theorem for Bayesian Neural Network trained with Variational Inference

Mean-field Analysis of Generalization Errors

Universality of the mean-field equations of networks of Hopfield-like neurons

Non-exchangeable networks of integrate-and-fire neurons: spatially-extended mean-field limit of the empirical measure

The thermodynamic limit in mean field neural networks

Neural Fields and Noise-Induced Patterns in Neurons on Large Disordered Networks

Local central limit theorem for gradient field models

Mean-Field Limit of a Stochastic Particle System Smoothly Interacting Through Threshold Hitting-Times and Applications to Neural Networks with Dendritic Component

Mean-Field and Kinetic Descriptions of Neural Differential Equations

Towards a General Theory of Infinite-Width Limits of Neural Classifiers

On kernel-based statistical learning in the mean field limit

Mean-field Analysis of Batch Normalization