Abstract:Understanding real-world dynamical phenomena remains a challenging task. Across various scientific disciplines, machine learning has advanced as the go-to technology to analyze nonlinear dynamical systems, identify patterns in big data, and make decision around them. Neural networks are now consistently used as universal function approximators for data with underlying mechanisms that are incompletely understood or exceedingly complex. However, neural networks alone ignore the fundamental laws of physics and often fail to make plausible predictions. Here we integrate data, physics, and uncertainties by combining neural networks, physics-informed modeling, and Bayesian inference to improve the predictive potential of traditional neural network models. We embed the physical model of a damped harmonic oscillator into a fully-connected feed-forward neural network to explore a simple and illustrative model system, the outbreak dynamics of COVID-19. Our Physics-Informed Neural Networks can seamlessly integrate data and physics, robustly solve forward and inverse problems, and perform well for both interpolation and extrapolation, even for a small amount of noisy and incomplete data. At only minor additional cost, they can self-adaptively learn the weighting between data and physics. Combined with Bayesian Neural Networks, they can serve as priors in a Bayesian Inference, and provide credible intervals for uncertainty quantification. Our study reveals the inherent advantages and disadvantages of Neural Networks, Bayesian Inference, and a combination of both and provides valuable guidelines for model selection. While we have only demonstrated these approaches for the simple model problem of a seasonal endemic infectious disease, we anticipate that the underlying concepts and trends generalize to more complex disease conditions and, more broadly, to a wide variety of nonlinear dynamical systems.

Bayesian mechanics of self-organising systems

An approach to non-equilibrium statistical physics using variational Bayesian inference

The free energy principle made simpler but not too simple

The principles of adaptation in organisms and machines II: Thermodynamics of the Bayesian brain

Probabilistic Principles for Biophysics and Neuroscience: Entropy Production, Bayesian Mechanics & the Free-Energy Principle

Bayesian Mechanics of Synaptic Learning under the Free Energy Principle

Bayesian mechanics of perceptual inference and motor control in the brain

Bayesian Model Selection Methods and Their Application to Biological ODE Systems

Bayesian Self-Organization Driven by Prior Probability Distributions

Hamiltonian Dynamics of Bayesian Inference Formalised by Arc Hamiltonian Systems

Hierarchical Bayesian Operational Modal Analysis: Theory and Computations

An information field theory approach to Bayesian state and parameter estimation in dynamical systems

Bayesian inference in physics

Free energy and inference in living systems

Theoretical and Numerical Study of Self-Organizing Processes In a Closed System Classical Oscillator + Random Environment

From reasonable postulates to generalised Hamiltonian systems

Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems

Bayesian mechanistic inference, statistical mechanics, and a new era for Monte Carlo

Self-Organization of Complex Systems

Efficient Amortised Bayesian Inference for Hierarchical and Nonlinear Dynamical Systems

The Information Theory of Self-Organization Phenomena in Thermal Systems