Abstract:Bayesian Physics Informed Neural Networks (BPINN) have received considerable attention for inferring differential equations' system states and physical parameters according to noisy observations. However, in practice, Hamiltonian Monte Carlo (HMC) used to estimate the internal parameters of BPINN often encounters troubles, including poor performance and awful convergence for a given step size used to adjust the momentum of those parameters. To improve the efficacy of HMC convergence for the BPINN method and extend its application scope to multi-scale partial differential equations (PDE), we developed a robust multi-scale Bayesian PINN (dubbed MBPINN) method by integrating multi-scale deep neural networks (MscaleDNN) and Bayesian inference. In this newly proposed MBPINN method, we reframe HMC with Stochastic Gradient Descent (SGD) to ensure the most ``likely'' estimation is always provided, and we configure its solver as a Fourier feature mapping-induced MscaleDNN. The MBPINN method offers several key advantages: (1) it is more robust than HMC, (2) it incurs less computational cost than HMC, and (3) it is more flexible for complex problems. We demonstrate the applicability and performance of the proposed method through general Poisson and multi-scale elliptic problems in one- to three-dimensional spaces. Our findings indicate that the proposed method can avoid HMC failures and provide valid results. Additionally, our method can handle complex PDE and produce comparable results for general PDE. These findings suggest that our proposed approach has excellent potential for physics-informed machine learning for parameter estimation and solution recovery in the case of ill-posed problems.

Bayesian Power Steering: An Effective Approach for Domain Adaptation of Diffusion Models

An Improved Mask Approach Based on Pointer Network for Domain Adaptation of BERT

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains.

A Bayesian Approach to Data Point Selection

BayesAdapter: Being Bayesian, Inexpensively and Reliably, via Bayesian Fine-tuning

Exploring Diffusion Models' Corruption Stage in Few-Shot Fine-tuning and Mitigating with Bayesian Neural Networks

Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction

Diffusion-Based Probabilistic Uncertainty Estimation for Active Domain Adaptation.

Divide to Adapt: Mitigating Confirmation Bias for Domain Adaptation of Black-Box Predictors

Implicit Bayes Adaptation: A Collaborative Transport Approach

Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling

On Calibrating Diffusion Probabilistic Models

Fine-Tuning Pre-Trained Language Models Effectively by Optimizing Subnetworks Adaptively

A Closer Look at Parameter-Efficient Tuning in Diffusion Models.

Adaptive Self-supervision Algorithms for Physics-informed Neural Networks

Rethinking Distributional Matching Based Domain Adaptation

Improvement of Bayesian PINN Training Convergence in Solving Multi-scale PDEs with Noise

Super-model ecosystem: A domain-adaptation perspective

Bayesian Prediction-Powered Inference

Bayesian-Optimized One-Step Diffusion Model with Knowledge Distillation for Real-Time 3D Human Motion Prediction