Abstract:Solving partial differential equations (PDEs) and their inverse problems using Physics-informed neural networks (PINNs) is a rapidly growing approach in the physics and machine learning community. Although several architectures exist for PINNs that work remarkably in practice, our theoretical understanding of their performances is somewhat limited. In this work, we study the behavior of a Bayesian PINN estimator of the solution of a PDE from $n$ independent noisy measurement of the solution. We focus on a class of equations that are linear in their parameters (with unknown coefficients $\theta_\star$). We show that when the partial differential equation admits a classical solution (say $u_\star$), differentiable to order $\beta$, the mean square error of the Bayesian posterior mean is at least of order $n^{-2\beta/(2\beta + d)}$. Furthermore, we establish a convergence rate of the linear coefficients of $\theta_\star$ depending on the order of the underlying differential operator. Last but not least, our theoretical results are validated through extensive simulations.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to estimate the parameters in a partial differential equation (PDE) given noisy measurement data of the solution of the PDE. Specifically, the researchers focus on estimating these parameters through Bayesian Physics - informed Neural Networks (Bayesian PINN) and analyze the performance of this method in solving inverse problems. The paper considers a class of partial differential equations with linear parameters, which are in the form:
\[ H_0u + \theta^T H_1u = f \]
where \( H_0 \) and \( H_1 \) are differential operators, \( \theta \) is the unknown parameter vector, \( u \) is the function to be solved, and \( f \) is a known function. The main objectives of the study are:
1. **Parameter Estimation**: Estimate the parameter \( \theta \) in the PDE from noisy data.
2. **Solution Estimation**: Simultaneously estimate the solution \( u \) of the PDE.
3. **Theoretical Analysis**: Analyze the convergence rate and error bounds of the Bayesian PINN method in estimating parameters and solutions.
### Main Contributions
1. **Convergence Rate of Parameter Estimation**:
- When the true solution \( u^\star \) of the PDE has \( \beta \)-order derivatives, the mean - square error of the Bayesian posterior mean satisfies:
\[
\mathbb{E}\left[\left\|\int_{\mathbb{R}^d} \theta \Pi(\theta)(d\theta|D) - \theta^\star\right\|^2_2\right] \lesssim n^{-2(\beta-\tau)/(m + 2\beta)}
\]
- Here \( n \) is the number of samples, \( m \) is the spatial dimension, and \( \tau \) is the highest order of the differential operator.
2. **Estimation Rate of the Solution**:
- The estimation rate of the solution \( u^\star \) of the PDE is at least the same as the non - parametric minimax optimal rate:
\[
\mathbb{E}\left[\int_{\mathbb{R}^q} \|u_W - u^\star\|^2_2 \Pi(W)(dW|D)\right] \lesssim n^{-2\beta/(m + 2\beta)}
\]
3. **Properties of the Bayesian Posterior Distribution**:
- Analyze the closeness of the Bayesian posterior distribution to the Gaussian distribution \( N(\theta^\star, \lambda^{-1}\Sigma^\star) \) through the 2 - Wasserstein distance.
### Related Work
- **Forward Problem**: Most of the existing theoretical literature focuses on the forward problem of estimating \( u \) given the parameter \( \theta \).
- **Inverse Problem**: Although PINN is very popular in practical applications, its theoretical aspects are still relatively less studied. This paper fills this gap, especially in analyzing the performance of PINN in inverse problems in the Bayesian framework.
### Numerical Experiments
The paper verifies the theoretical results through numerical experiments of the one - dimensional heat conduction equation. The experimental settings are as follows:
- The form of the heat conduction equation is \( u_t - \theta u_{xx} = 0 \), the initial condition is \( u(0, x)=\sin(x) \), and the boundary conditions are \( u(t, 0)=u(t, \pi)=0 \).
- The true parameter \( \theta^\star = 0.5 \).
- The observation data \( D \) is generated by sensor positions uniformly distributed on \( \Omega=(0, 1)\times(0, \pi) \) and contains noise.
By comparing Bayesian PINN with other methods that do not directly utilize the PDE structure, the experimental results show that Bayesian PINN has better performance in parameter estimation and solution estimation.
### Conclusion
This paper, through theoretical analysis and numerical experiments...