B-LSTM-MIONet: Bayesian LSTM-based Neural Operators for Learning the Response of Complex Dynamical Systems to Length-Variant Multiple Input Functions

Zhihao Kong,Amirhossein Mollaali,Christian Moya,Na Lu,Guang Lin

2023-11-29

Abstract:Deep Operator Network (DeepONet) is a neural network framework for learning nonlinear operators such as those from ordinary differential equations (ODEs) describing complex systems. Multiple-input deep neural operators (MIONet) extended DeepONet to allow multiple input functions in different Banach spaces. MIONet offers flexibility in training dataset grid spacing, without constraints on output location. However, it requires offline inputs and cannot handle varying sequence lengths in testing datasets, limiting its real-time application in dynamic complex systems. This work redesigns MIONet, integrating Long Short Term Memory (LSTM) to learn neural operators from time-dependent data. This approach overcomes data discretization constraints and harnesses LSTM's capability with variable-length, real-time data. Factors affecting learning performance, like algorithm extrapolation ability are presented. The framework is enhanced with uncertainty quantification through a novel Bayesian method, sampling from MIONet parameter distributions. Consequently, we develop the B-LSTM-MIONet, incorporating LSTM's temporal strengths with Bayesian robustness, resulting in a more precise and reliable model for noisy datasets.

Machine Learning,Numerical Analysis

What problem does this paper attempt to address?

The paper attempts to address the limitations of existing neural network models in handling complex dynamic systems, particularly in situations where the length of input data varies. Specifically, the paper aims to solve the following problems: 1. **Handling variable-length input data**: Existing Multi-Input Deep Neural Operators (MIONet) require input data to have a fixed length and need to know the entire input sequence in advance. This limits their application in real-time prediction of complex dynamic systems, as the length of actual observed data changes over time. 2. **Real-time data processing**: The traditional MIONet framework cannot handle real-time data, i.e., data that dynamically changes over time. The paper proposes a new framework by introducing Long Short-Term Memory networks (LSTM), enabling the model to learn from real-time data and adapt to input sequences of different lengths. 3. **Uncertainty quantification**: Existing neural network models perform poorly when dealing with noisy data, leading to reduced generalization ability. By introducing Bayesian methods, the paper provides effective uncertainty quantification, thereby improving the model's robustness and reliability in noisy environments. To achieve these goals, the paper proposes the B-LSTM-MIONet framework, which combines the temporal processing capabilities of LSTM with the uncertainty quantification capabilities of Bayesian methods, demonstrating higher accuracy and confidence in handling real-time data of complex dynamic systems.

B-LSTM-MIONet: Bayesian LSTM-based Neural Operators for Learning the Response of Complex Dynamical Systems to Length-Variant Multiple Input Functions

MIONet: Learning multiple-input operators via tensor product

State-space Modeling for Electrochemical Performance of Li-ion Batteries with Physics-Informed Deep Operator Networks

On Learning the Dynamical Response of Nonlinear Control Systems with Deep Operator Networks

Modeling of nonlinear dynamic system based on Deep Operator Network

DON-LSTM: Multi-Resolution Learning with DeepONets and Long Short-Term Memory Neural Networks

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

AP-MIONet: Asymptotic-preserving multiple-input neural operators for capturing the high-field limits of collisional kinetic equations

LeMON: Learning to Learn Multi-Operator Networks

Synergistic Learning with Multi-Task DeepONet for Efficient PDE Problem Solving

Learning in latent spaces improves the predictive accuracy of deep neural operators

Variationally mimetic operator networks

Sequential Deep Operator Networks (S-DeepONet) for predicting full-field solutions under time-dependent loads

A New Neural Network Model And Its Application

Separable Operator Networks

MODNO: Multi Operator Learning With Distributed Neural Operators

Deep Multi-Input and Multi-Output Operator Networks Method for Optimal Control of Pdes

LINOCS: Lookahead Inference of Networked Operators for Continuous Stability

Fourier-MIONet: Fourier-enhanced multiple-input neural operators for multiphase modeling of geological carbon sequestration

Bayesian Learning to Discover Mathematical Operations in Governing Equations of Dynamic Systems