Multi-Grid Graph Neural Networks with Self-Attention for Computational Mechanics

Paul Garnier,Jonathan Viquerat,Elie Hachem

2024-09-18

Abstract:Advancement in finite element methods have become essential in various disciplines, and in particular for Computational Fluid Dynamics (CFD), driving research efforts for improved precision and efficiency. While Convolutional Neural Networks (CNNs) have found success in CFD by mapping meshes into images, recent attention has turned to leveraging Graph Neural Networks (GNNs) for direct mesh processing. This paper introduces a novel model merging Self-Attention with Message Passing in GNNs, achieving a 15\% reduction in RMSE on the well known flow past a cylinder benchmark. Furthermore, a dynamic mesh pruning technique based on Self-Attention is proposed, that leads to a robust GNN-based multigrid approach, also reducing RMSE by 15\%. Additionally, a new self-supervised training method based on BERT is presented, resulting in a 25\% RMSE reduction. The paper includes an ablation study and outperforms state-of-the-art models on several challenging datasets, promising advancements similar to those recently achieved in natural language and image processing. Finally, the paper introduces a dataset with meshes larger than existing ones by at least an order of magnitude. Code and Datasets will be released at <a class="link-external link-https" href="https://github.com/DonsetPG/multigrid-gnn" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Computational Engineering, Finance, and Science

What problem does this paper attempt to address?

The paper aims to address the issues of accuracy and efficiency in Computational Fluid Dynamics (CFD). Specifically, the authors propose a new model that combines the self-attention mechanism with the message passing of Graph Neural Networks (GNN) to handle mesh data in the finite element method. This model achieved significant improvements in the classic cylinder flow benchmark, reducing the Root Mean Square Error (RMSE) by 15%. Additionally, the paper introduces a dynamic mesh pruning technique based on the self-attention mechanism, further enhancing the effectiveness of the multigrid method and similarly reducing RMSE by 15%. The study also introduces a new self-supervised training method, based on a BERT-like approach, which predicts missing information by masking partial nodes, reducing RMSE by 25%.

The authors conducted experiments on various datasets, including CYLINDER FLOW, DEFORMING PLATE, and BEZIER SHAPES, demonstrating the superior performance of the new model on these datasets and its excellent performance on large-scale meshes (over 30,000 nodes). Furthermore, ablation studies validated the effectiveness of each component and proved that the proposed multigrid method outperforms traditional encode-process-decode architectures on datasets of varying complexity. Overall, the study showcases the model's good generalization ability in different physical scenarios and is expected to advance CFD and other related fields.