Abstract:In many computational problems in engineering and science, function or model differentiation is essential, but also integration is needed. An important class of computational problems include so-called integro-differential equations which include both integrals and derivatives of a function. In another example, stochastic differential equations can be written in terms of a partial differential equation of a probability density function of the stochastic variable. To learn characteristics of the stochastic variable based on the density function, specific integral transforms, namely moments, of the density function need to be calculated. Recently, the machine learning paradigm of Physics-Informed Neural Networks emerged with increasing popularity as a method to solve differential equations by leveraging automatic differentiation. In this work, we propose to augment the paradigm of Physics-Informed Neural Networks with automatic integration in order to compute complex integral transforms on trained solutions, and to solve integro-differential equations where integrals are computed on-the-fly during training. Furthermore, we showcase the techniques in various application settings, numerically simulating quantum computer-based neural networks as well as classical neural networks.

Integral transform and its application to neural network approximation

Approximations of Continuous Functionals by Neural Networks with Application to Dynamic Systems

Neural Networks are Integrable

Function Approximation with Randomly Initialized Neural Networks for Approximate Model Reference Adaptive Control

Approximation Capability of a Novel Neural Network Model for Dynamic Systems

Approximation of RKHS Functionals by Neural Networks

On the Approximation and Complexity of Deep Neural Networks to Invariant Functions

Universal Approximation to Nonlinear Operators by Neural Networks with Arbitrary Activation Functions and Its Application to Dynamical Systems

Neural Network Approximations of Compositional Functions With Applications to Dynamical Systems

Theoretical Research on Neural Networks in Application to Dynamic System Modeling

An approximation by neural networkswith a fixed weight

Universal Approximation of Operators with Transformers and Neural Integral Operators

Universal Approximation Theorem for Neural Networks

Approximation Capability to Functions of Several Variables, Nonlinear Functionals and Operators by Radial Basis Function Neural Networks

Integral Transforms in a Physics-Informed (Quantum) Neural Network setting: Applications & Use-Cases

On the approximation of bi-Lipschitz maps by invertible neural networks

Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks

Approximation in Shift-Invariant Spaces with Deep ReLU Neural Networks

Approximation of relation functions and attention mechanisms