Abstract:This article studies the dynamics of the mean-field approximation of continuous random networks. These networks are stochastic integrodifferential equations driven by Gaussian noise. The kernels in the integral operators are realizations of generalized Gaussian random variables. The equation controls the time evolution of a macroscopic state interpreted as neural activity, which depends on position and time. The position is an element of a measurable space. Such a network corresponds to a statistical field theory (STF) given by a momenta-generating functional. Discrete versions of the mentioned networks appeared in spin glasses and as models of artificial neural networks (NNs). Each of these discrete networks corresponds to a lattice SFT, where the action contains a finite number of neurons and two scalar fields for each neuron. Recently, it has been proposed that these networks can be used as models for deep learning. In this application, the number of neurons is astronomical; consequently, continuous models are required. In this article, we develop mathematically rigorous, continuous versions of the mean-field theory approximation and the double-copy system that allow us to derive a condition for the criticality of continuous stochastic networks via the largest Lyapunov exponent. We use two basic architectures; in the first one, the space of neurons is the real line, and then the neurons are organized in one layer; in the second one, the space of neurons is the p-adic line, and then the neurons are organized in an infinite, fractal, tree-like structure. Networks with both types of architectures exhibit a type edge of the chaos organization.

Controllability of continuous networks and a kernel-based learning approximation

Mean-Field and Kinetic Descriptions of Neural Differential Equations

Mean-field Langevin System, Optimal Control and Deep Neural Networks

Learning Theory for Inferring Interaction Kernels in Second-Order Interacting Agent Systems

Mean field limits for discrete-time dynamical systems via kernel mean embeddings

Control of high-dimensional collective dynamics by deep neural feedback laws and kinetic modelling

On kernel-based statistical learning in the mean field limit

Dynamic Mean-Field Theory for Continuous Random Networks

Control of Partial Differential Equations via Physics-Informed Neural Networks

A deep learning algorithm for computing mean field control problems via forward-backward score dynamics

Neural Control Systems

Controllability-Constrained Deep Network Models for Enhanced Control of Dynamical Systems

A Mean-Field Optimal Control Formulation of Deep Learning

Deep Learning for Population-Dependent Controls in Mean Field Control Problems with Common Noise

Long run convergence of discrete-time interacting particle systems of the McKean-Vlasov type

Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective

Kernel Expansions for High-Dimensional Mean-Field Control with Non-local Interactions

Mean-Field Limit of a Stochastic Particle System Smoothly Interacting Through Threshold Hitting-Times and Applications to Neural Networks with Dendritic Component

Identifiability of interaction kernels in mean-field equations of interacting particles

Network Diffusions via Neural Mean-Field Dynamics

Learning effective stochastic differential equations from microscopic simulations: combining stochastic numerics and deep learning