One-Shot Transfer Learning for Nonlinear ODEs

Wanzhou Lei,Pavlos Protopapas,Joy Parikh
DOI: https://doi.org/10.48550/arXiv.2311.14931
2023-11-25
Abstract:We introduce a generalizable approach that combines perturbation method and one-shot transfer learning to solve nonlinear ODEs with a single polynomial term, using Physics-Informed Neural Networks (PINNs). Our method transforms non-linear ODEs into linear ODE systems, trains a PINN across varied conditions, and offers a closed-form solution for new instances within the same non-linear ODE class. We demonstrate the effectiveness of this approach on the Duffing equation and suggest its applicability to similarly structured PDEs and ODE systems.
Machine Learning
What problem does this paper attempt to address?
This paper attempts to solve several key problems in solving nonlinear ordinary differential equations (ODEs): 1. **High computational cost**: Traditional physics - informed neural networks (PINNs) cannot generalize well when solving similar types of equations for different instances, resulting in the need for retraining every time a new instance is solved, increasing the computational cost. 2. **Complexity of nonlinear ODEs**: The loss function of nonlinear ODEs cannot be directly analytically optimized for weights, so it is difficult to use one - shot transfer learning to solve them quickly. 3. **Limitations of existing methods**: Although previous studies have applied transfer learning to the single - shot solution of linear ODEs, these methods cannot be directly extended to nonlinear ODEs because the nonlinear terms make the loss function complex and cannot be optimized by simple transfer learning. To solve these problems, the author proposes a new method that combines the perturbation method and one - shot transfer learning. Specifically: - **Perturbation method**: The nonlinear ODE is transformed into a series of linear ODE systems through perturbation expansion. For a nonlinear ODE of the form \( Dx+\epsilon x^{q} = f(t)\), assume \( x=\sum_{i = 0}^{p}\epsilon^{i}x_{i}\) and expand using the polynomial theorem to obtain a series of linear ODEs. The specific formula is as follows: \[ \sum_{i = 0}^{p}\epsilon^{i}D x_{i}+\epsilon\left(\sum_{k_{0}+k_{1}+\cdots + k_{p}=q}\frac{q!}{k_{1}!k_{2}!\cdots k_{p}!}\prod_{i = 0}^{p}(x_{i})^{k_{i}}\right)=f \] - **One - shot transfer learning**: During the training phase, a multi - head fully - connected neural network is constructed, with each head corresponding to a linear ODE system under a specific parameter setting. After training, the hidden - layer weights are frozen. When a new nonlinear ODE of the same type is encountered, only the weights of the last linear output layer need to be adjusted to quickly solve the new problem. Through this method, the author can not only solve nonlinear ODEs efficiently but also significantly reduce the computational cost and improve the generalization ability of the model. The experimental results show that this method performs very well on the Duffing equation and can accurately solve unseen Duffing equation instances within a few seconds.