Abstract:Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-task learning. It has been extensively explored in traditional machine learning to address issues such as data sparsity and overfitting in neural networks. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs). However, implementing MTL in this context is complex, as it requires task-specific modifications to accommodate various scenarios representing different physical processes. To this end, we present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session. We introduce modifications in the branch network of the vanilla DeepONet to account for various functional forms of a parameterized coefficient in a PDE. Additionally, we handle parameterized geometries by introducing a binary mask in the branch network and incorporating it into the loss term to improve convergence and generalization to new geometry tasks. Our approach is demonstrated on three benchmark problems: (1) learning different functional forms of the source term in the Fisher equation; (2) learning multiple geometries in a 2D Darcy Flow problem and showcasing better transfer learning capabilities to new geometries; and (3) learning 3D parameterized geometries for a heat transfer problem and demonstrate the ability to predict on new but similar geometries. Our MT-DeepONet framework offers a novel approach to solving PDE problems in engineering and science under a unified umbrella based on synergistic learning that reduces the overall training cost for neural operators.

DeepMartNet -- A Martingale Based Deep Neural Network Learning Method for Dirichlet BVPs and Eigenvalue Problems of Elliptic PDEs in R^d

DeepMartNet -- A Martingale based Deep Neural Network Learning Algorithm for Eigenvalue/BVP Problems and Optimal Stochastic Controls

SOC-MartNet: A Martingale Neural Network for the Hamilton-Jacobi-Bellman Equation without Explicit inf H in Stochastic Optimal Controls

A Natural Deep Ritz Method for Essential Boundary Value Problems

DeepSets and their derivative networks for solving symmetric PDEs

Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations

MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs

Positional Embeddings for Solving PDEs with Evolutional Deep Neural Networks

Deep-learning of parametric partial differential equations from sparse and noisy data

Robust SDE-Based Variational Formulations for Solving Linear PDEs via Deep Learning

A Deep Learning Approach to Data-Driven Model-Free Pricing and to Martingale Optimal Transport

Deep neural network approximation for high-dimensional elliptic PDEs with boundary conditions

Solving High-Dimensional Partial Differential Equations Using Deep Learning.

The Deep Parametric PDE Method: Application to Option Pricing

Synergistic Learning with Multi-Task DeepONet for Efficient PDE Problem Solving

Solving Partial Differential Equations Using Deep Learning and Physical Constraints

BMO Martingale Method for Backward Stochastic Differential Equations Driven by General Càdlàg Local Martingales

Variational Inverting Network for Statistical Inverse Problems of Partial Differential Equations

TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision

Numerical solutions of boundary problems in partial differential equations: A deep learning framework with Green's function