Abstract:Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-task learning. It has been extensively explored in traditional machine learning to address issues such as data sparsity and overfitting in neural networks. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs). However, implementing MTL in this context is complex, as it requires task-specific modifications to accommodate various scenarios representing different physical processes. To this end, we present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session. We introduce modifications in the branch network of the vanilla DeepONet to account for various functional forms of a parameterized coefficient in a PDE. Additionally, we handle parameterized geometries by introducing a binary mask in the branch network and incorporating it into the loss term to improve convergence and generalization to new geometry tasks. Our approach is demonstrated on three benchmark problems: (1) learning different functional forms of the source term in the Fisher equation; (2) learning multiple geometries in a 2D Darcy Flow problem and showcasing better transfer learning capabilities to new geometries; and (3) learning 3D parameterized geometries for a heat transfer problem and demonstrate the ability to predict on new but similar geometries. Our MT-DeepONet framework offers a novel approach to solving PDE problems in engineering and science under a unified umbrella based on synergistic learning that reduces the overall training cost for neural operators.

Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers

Blending neural operators and relaxation methods in PDE numerical solvers

Non-equispaced Fourier Neural Solvers for PDEs

Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solvers

Finite Operator Learning: Bridging Neural Operators and Numerical Methods for Efficient Parametric Solution and Optimization of PDEs

A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks

dNNsolve: an efficient NN-based PDE solver

Convergence Framework of Deep Learning-based Hybrid Iterative Methods and the Application to Designing a Fourier Neural Solver for Parametric PDEs

Interfacing finite elements with deep neural operators for fast multiscale modeling of mechanics problems

A hybrid iterative method based on MIONet for PDEs: Theory and numerical examples

Semi-Implicit Neural Solver for Time-dependent Partial Differential Equations

Solving forward and inverse PDE problems on unknown manifolds via physics-informed neural operators

Deep Equilibrium Based Neural Operators for Steady-State PDEs

A novel paradigm for solving PDEs: multi scale neural computing

Operator Learning Enhanced Physics-informed Neural Networks for Solving Partial Differential Equations Characterized by Sharp Solutions

Neural Q-learning for solving PDEs

Message Passing Neural PDE Solvers

Synergistic Learning with Multi-Task DeepONet for Efficient PDE Problem Solving

Large scale scattering using fast solvers based on neural operators

A hybrid physics-informed neural network based multiscale solver as a partial differential equation constrained optimization problem

Enhancing Solutions for Complex PDEs: Introducing Complementary Convolution and Equivariant Attention in Fourier Neural Operators