Abstract:We introduce neural information field filter, a Bayesian state and parameter estimation method for high-dimensional nonlinear dynamical systems given large measurement datasets. Solving such a problem using traditional methods, such as Kalman and particle filters, is computationally expensive. Information field theory is a Bayesian approach that can efficiently reconstruct dynamical model state paths and calibrate model parameters from noisy measurement data. To apply the method, we parameterize the time evolution state path using the span of a finite linear basis. The existing method has to reparameterize the state path by initial states to satisfy the initial condition. Designing an expressive yet simple linear basis before knowing the true state path is crucial for inference accuracy but challenging. Moreover, reparameterizing the state path using the initial state is easy to perform for a linear basis, but is nontrivial for more complex and expressive function parameterizations, such as neural networks. The objective of this paper is to simplify and enrich the class of state path parameterizations using neural networks for the information field theory approach. To this end, we propose a generalized physics-informed conditional prior using an auxiliary initial state. We show the existing reparameterization is a special case. We parameterize the state path using a residual neural network that consists of a linear basis function and a Fourier encoding fully connected neural network residual function. The residual function aims to correct the error of the linear basis function. To sample from the intractable posterior distribution, we develop an optimization algorithm, nested stochastic variational inference, and a sampling algorithm, nested preconditioned stochastic gradient Langevin dynamics. A series of numerical and experimental examples verify and validate the proposed method.

Initialization of multilayer forecasting artifical neural networks

How to Initialize your Network? Robust Initialization for WeightNorm & ResNets

A weight initialization based on the linear product structure for neural networks

Multistep Prediction of Dynamic Systems With Recurrent Neural Networks

A mathematical framework for improved weight initialization of neural networks using Lagrange multipliers

State initialization for recurrent neural network modeling of time-series data

The Improved GMDH-type Neural Network and Its Application to Forecasting Chaotic Time Series

Stability of implicit neural networks for long-term forecasting in dynamical systems

Improved weight initialization for deep and narrow feedforward neural network

Back to Basics: The Power of the Multilayer Perceptron in Financial Time Series Forecasting

From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks

Critical Initialization of Wide and Deep Neural Networks through Partial Jacobians: General Theory and Applications

Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs

Application Research of Chaotic Time Series Prediction Based on PMLP Neural Network

Optical characterization of chevron texture formation in nematic electroconvection

Reconstruction, forecasting, and stability of chaotic dynamics from partial data

A Sober Look at Neural Network Initializations

Improving performance of recurrent neural network with relu nonlinearity

On weight initialization in deep neural networks

Constraining Chaos: Enforcing dynamical invariants in the training of recurrent neural networks

Neural information field filter