Abstract:Recently, deep learning-based hybrid iterative methods (DL-HIM) have emerged as a promising approach for designing fast neural solvers to tackle large-scale sparse linear systems. DL-HIM combine the smoothing effect of simple iterative methods with the spectral bias of neural networks, which allows them to effectively eliminate both high-frequency and low-frequency error components. However, their efficiency may decrease if simple iterative methods can not provide effective smoothing, making it difficult for the neural network to learn mid-frequency and high-frequency components. This paper first establishes a convergence analysis framework for general DL-HIM from a spectral viewpoint, concluding that under reasonable assumptions, DL-HIM exhibit a convergence rate independent of grid size $h$ and physical parameters $\boldsymbol{\mu}$. To meet these assumptions, we design a neural network from an eigen perspective, focusing on learning the eigenvalues and eigenvectors corresponding to error components that simple iterative methods struggle to eliminate. Specifically, the eigenvalues are learned by a meta subnet, while the eigenvectors are approximated using Fourier modes with a transition matrix provided by another meta subnet. The resulting DL-HIM, termed the Fourier Neural Solver (FNS), is expected to achieve a $h, \boldsymbol{\mu}$-independent convergence rate with the guarantee of universal approximation and the discretization invariant property of the meta subnets. We verify the performance of FNS on four types of linear parametric PDEs.

Research on SIMO Fourier Neural Networks Based on Least Square Method

Simo Fourier Neural Networks Research

A Weights-Direct-Determination Method and Structure-Automatic-Determination Algorithm for SIMO Trigonometrically-Activated Fourier Neural Networks

Variable Structure and Variable Learning Rate Fourier Neural Networks Research

Convergence Framework of Deep Learning-based Hybrid Iterative Methods and the Application to Designing a Fourier Neural Solver for Parametric PDEs

On-chip 4F-System Based on Concave Mirrors for Optical Neural Networks

Non-equispaced Fourier Neural Solvers for PDEs

A NOVEL METHOD FOR IDENTIFYING NONLINEAR SYSTEMS USING ORTHOGONAL WAVELET NETWORKS

Accelerating the convergence of Newton's method for nonlinear elliptic PDEs using Fourier neural operators

Training via quantum superposition circumventing local minima and vanishing gradient of sinusoidal neural network

Fourier neural networks as function approximators and differential equation solvers

Deep Learning without Global Optimization by Random Fourier Neural Networks

Noncoherent Energy-Modulated Massive SIMO in Multipath Channels: A Machine Learning Approach

Multi-layer optical Fourier neural network based on the convolution theorem

Solving for a Quadratic Programming with a Quadratic Constraint Based on a Neural Network Frame

From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems

Enhancing Open Quantum Dynamics Simulations Using Neural Network-Based Non-Markovian Stochastic Schrödinger Equation Method

Fast Solver of 2D Maxwell's Equations Based on Fourier Neural Operator

Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective

Line Spectral Estimation Inspired by Quasi-Neural Network

Component Fourier Neural Operator for Singularly Perturbed Differential Equations