Abstract:This article studies the dynamics of the mean-field approximation of continuous random networks. These networks are stochastic integrodifferential equations driven by Gaussian noise. The kernels in the integral operators are realizations of generalized Gaussian random variables. The equation controls the time evolution of a macroscopic state interpreted as neural activity, which depends on position and time. The position is an element of a measurable space. Such a network corresponds to a statistical field theory (STF) given by a momenta-generating functional. Discrete versions of the mentioned networks appeared in spin glasses and as models of artificial neural networks (NNs). Each of these discrete networks corresponds to a lattice SFT, where the action contains a finite number of neurons and two scalar fields for each neuron. Recently, it has been proposed that these networks can be used as models for deep learning. In this application, the number of neurons is astronomical; consequently, continuous models are required. In this article, we develop mathematically rigorous, continuous versions of the mean-field theory approximation and the double-copy system that allow us to derive a condition for the criticality of continuous stochastic networks via the largest Lyapunov exponent. We use two basic architectures; in the first one, the space of neurons is the real line, and then the neurons are organized in one layer; in the second one, the space of neurons is the p-adic line, and then the neurons are organized in an infinite, fractal, tree-like structure. Networks with both types of architectures exhibit a type edge of the chaos organization.

Function approximation by neural nets in the mean-field regime: Entropic regularization and controlled McKean-Vlasov dynamics

Mean field approximations via log-concavity

Laws of Large Numbers and Langevin Approximations for Stochastic Neural Field Equations

Empirical approximation to invariant measures of mean-field Langevin dynamics

Improved Particle Approximation Error for Mean Field Neural Networks

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimiax Optimization

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization

Non-exchangeable networks of integrate-and-fire neurons: spatially-extended mean-field limit of the empirical measure

Uniform-in-time propagation of chaos for kinetic mean field Langevin dynamics

Entropic Fictitious Play for Mean Field Optimization Problem

Mean-Field Limit of a Stochastic Particle System Smoothly Interacting Through Threshold Hitting-Times and Applications to Neural Networks with Dendritic Component

Convergence and Approximation of Invariant Measures for Neural Field Lattice Models under Noise Perturbation

Mean-Field and Kinetic Descriptions of Neural Differential Equations

Convergence of Policy Gradient for Entropy Regularized MDPs with Neural Network Approximation in the Mean-Field Regime

The thermodynamic limit in mean field neural networks

Mean-field underdamped Langevin dynamics and its spacetime discretization

Dynamic Mean-Field Theory for Continuous Random Networks

Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics

Projected Langevin dynamics and a gradient flow for entropic optimal transport

Random Vector Functional Link Networks for Function Approximation on Manifolds

Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings