Abstract:Neural networks are increasingly employed to model, analyze and control non-linear dynamical systems ranging from physics to biology. Owing to their universal approximation capabilities, they regularly outperform state-of-the-art model-driven methods in terms of accuracy, computational speed, and/or control capabilities. On the other hand, neural networks are very often they are taken as black boxes whose explainability is challenged, among others, by huge amounts of trainable parameters. In this paper, we tackle the outstanding issue of analyzing the inner workings of neural networks trained to classify regular-versus-chaotic time series. This setting, well-studied in dynamical systems, enables thorough formal analyses. We focus specifically on a family of networks dubbed Large Kernel Convolutional Neural Networks (LKCNN), recently introduced by Boullé et al. (2021). These non-recursive networks have been shown to outperform other established architectures (e.g. residual networks, shallow neural networks and fully convolutional networks) at this classification task. Furthermore, they outperform ``manual'' classification approaches based on direct reconstruction of the Lyapunov exponent. We find that LKCNNs use qualitative properties of the input sequence. In particular, we show that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models. Low performing models show, in fact, analogous periodic activations to random untrained models. This could give very general criteria for identifying, a priori, trained models that have poor accuracy.

Full Lyapunov Exponents spectrum with Deep Learning from single-variable time series

Finite-Time Lyapunov Exponents of Deep Neural Networks

Data-driven Method for Identifying the Expression of the Lyapunov Exponent from Random Data

How neural networks learn to classify chaotic time series

Estimating Lyapunov exponents in biomedical time series

Generalized Lyapunov exponents and aspects of the theory of deep learning

Extracting Critical Exponents by Finite-Size Scaling with Convolutional Neural Networks

Deep learning-based state prediction of the Lorenz system with control parameters.

Deep LPPLS: Forecasting of temporal critical points in natural, engineering and financial systems

Linearly Recurrent Autoencoder Networks for Learning Dynamics

Deciphering Complexity: Machine Learning Insights into Chaotic Dynamical Systems

Deep learning algorithm for data-driven simulation of noisy dynamical system

Linearly-Recurrent Autoencoder Networks for Learning Dynamics

Estimation of Carleman operator from a univariate time series

Estimation of a Lyapunov-exponent Spectrum of Plasma Chaos

A Deep Learning Approach to Analyzing Continuous-Time Systems

Learning dynamical systems: an example from open quantum system dynamics

AI-Lorenz: A physics-data-driven framework for black-box and gray-box identification of chaotic systems with symbolic regression

Inexact iterative numerical linear algebra for neural network-based spectral estimation and rare-event prediction

Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint