Abstract:A large number of current machine learning methods rely upon deep neural networks. Yet, viewing neural networks as nonlinear dynamical systems, it becomes quickly apparent that mathematically rigorously establishing certain patterns generated by the nodes in the network is extremely difficult. Indeed, it is well-understood in the nonlinear dynamics of complex systems that, even in low-dimensional models, analytical techniques rooted in pencil-and-paper approaches frequently reach their limits. In this work, we propose a completely different perspective via the paradigm of validated numerical methods of nonlinear dynamics. The idea is to use computer-assisted proofs to validate mathematically the existence of nonlinear patterns in neural networks. As a case study, we consider a class of recurrent neural networks, where we prove via computer assistance the existence of several hundred Hopf bifurcation points, their non-degeneracy, and hence also the existence of several hundred periodic orbits. Our paradigm has the capability to rigorously verify complex nonlinear behaviour of neural networks, which provides a first step to explain the full abilities, as well as potential sensitivities, of machine learning methods via computer-assisted proofs. We showcase how validated numerical techniques can shed light on the internal working of recurrent neural networks (RNNs). For this, proofs of Hopf bifurcations are a first step towards an integration of dynamical system theory in practical application of RNNs, by proving the existence of periodic orbits in a variety of settings.

Stretched and measured neural predictions of complex network dynamics

Generalization of neural network models for complex network dynamics

Learning interpretable dynamics of stochastic complex systems from experimental data

Predicting network dynamics without requiring the knowledge of the interaction graph

Neural Dynamics on Complex Networks

Neural relational and dynamics inference for complex systems

Learning Dynamics and Structure of Complex Systems Using Graph Neural Networks

Do We Need an Encoder-Decoder to Model Dynamical Systems on Networks?

Inferring networks from time series: a neural approach

Can neural networks predict dynamics they have never seen?

Global Network Prediction from Local Node Dynamics

Stochastic Scaling in Loss Functions for Physics-Informed Neural Networks

Deep-learning reconstruction of complex dynamical networks from incomplete data

Reconstructing Network Dynamics of Coupled Discrete Chaotic Units from Data

Signed Graph Neural Ordinary Differential Equation for Modeling Continuous-time Dynamics

Estimating model inadequacy in ordinary differential equations with physics-informed neural networks

Inferring Neural Communication Dynamics from Field Potentials Using Graph Diffusion Autoregression

Characterizing Learning Dynamics of Deep Neural Networks via Complex Networks

Computer Validation of Neural Network Dynamics: A First Case Study

Neural network stochastic differential equation models with applications to financial data forecasting

Topological exploration of artificial neuronal network dynamics