Abstract:We introduce a novel deep operator network (DeepONet) framework that incorporates generalised variational inference (GVI) using Rényi's $\alpha$-divergence to learn complex operators while quantifying uncertainty. By incorporating Bayesian neural networks as the building blocks for the branch and trunk networks, our framework endows DeepONet with uncertainty quantification. The use of Rényi's $\alpha$-divergence, instead of the Kullback-Leibler divergence (KLD), commonly used in standard variational inference, mitigates issues related to prior misspecification that are prevalent in Variational Bayesian DeepONets. This approach offers enhanced flexibility and robustness. We demonstrate that modifying the variational objective function yields superior results in terms of minimising the mean squared error and improving the negative log-likelihood on the test set. Our framework's efficacy is validated across various mechanical systems, where it outperforms both deterministic and standard KLD-based VI DeepONets in predictive accuracy and uncertainty quantification. The hyperparameter $\alpha$, which controls the degree of robustness, can be tuned to optimise performance for specific problems. We apply this approach to a range of mechanics problems, including gravity pendulum, advection-diffusion, and diffusion-reaction systems. Our findings underscore the potential of $\alpha$-VI DeepONet to advance the field of data-driven operator learning and its applications in engineering and scientific domains.

Alpha-divergence minimization with mixed variational posterior for Bayesian neural networks and its robustness against adversarial examples

Alpha-VI DeepONet: A prior-robust variational Bayesian approach for enhancing DeepONets with uncertainty quantification

$α$-Variational Inference with Statistical Guarantees

Improving adversarial robustness of Bayesian neural networks via multi-task adversarial training

Variational Bayes Neural Network: Posterior Consistency, Classification Accuracy and Computational Challenges

Deterministic Variational Inference for Robust Bayesian Neural Networks

AdamMCMC: Combining Metropolis Adjusted Langevin with Momentum-based Optimization

Variational Learning of Bayesian Neural Networks Via Bayesian Dark Knowledge

On permutation symmetries in Bayesian neural network posteriors: a variational perspective

Bayesian Learning with Information Gain Provably Bounds Risk for a Robust Adversarial Defense

One step closer to unbiased aleatoric uncertainty estimation

Local Competition and Uncertainty for Adversarial Robustness in Deep Learning

Dynamic ensemble selection based on Deep Neural Network Uncertainty Estimation for Adversarial Robustness

Implicit Generative Prior for Bayesian Neural Networks

On the Convergence of Locally Adaptive and Scalable Diffusion-Based Sampling Methods for Deep Bayesian Neural Network Posteriors

Attacking Bayes: On the Adversarial Robustness of Bayesian Neural Networks

What Are Bayesian Neural Network Posteriors Really Like?

alpha-Deep Probabilistic Inference (alpha-DPI): efficient uncertainty quantification from exoplanet astrometry to black hole feature extraction

Regularization for Adversarial Robust Learning

Utilizing Amari-Alpha Divergence to Stabilize the Training of Generative Adversarial Networks

Adversarial robustness of amortized Bayesian inference