Abstract:We present an attention mechanism inspired from definition of screened Coulomb potential. This attention mechanism was used to interpret the Graph Attention (GAT) model layers and training dataset by using a flexible and scalable framework (CoulGAT) developed for this purpose. Using CoulGAT, a forest of plain and resnet models were trained and characterized using this attention mechanism against CHAMPS dataset. The learnable variables of the attention mechanism are used to extract node-node and node-feature interactions to define an empirical standard model for the graph structure and hidden layer. This representation of graph and hidden layers can be used as a tool to compare different models, optimize hidden layers and extract a compact definition of graph structure of the dataset.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to improve the interpretability of Graph Attention Networks (GAT), and by introducing an attention mechanism based on the screened Coulomb potential, to improve the understanding and prediction ability of graph - structured data. Specifically, the author proposes a framework named CoulGAT for training and interpreting graph attention models, especially for applications in the field of quantum chemistry.
### Problem Background
1. **Complexity of Graph - Structured Data**: Graph - structured data is very common in fields such as social networks, e - commerce, natural language processing, knowledge databases, quantum chemistry, and molecular biology. With the increase in the complexity and irregularity of the relationships between nodes, traditional Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) are difficult to be directly applied to these non - gridded data.
2. **Limitations of Existing Models**:
- **Convolutional Neural Networks (CNN)**: Although they can utilize local translation invariance, it is more complex to define convolution operations on graph data.
- **Recurrent Neural Networks (RNN)**: They are suitable for learning causal relationships between nodes, but it is difficult to extend them to large - scale graph structures.
- **Graph Attention Networks (GAT)**: Although they have improved the expressiveness and interpretability of the model, there is still room for improvement in the existing attention mechanisms.
### Main Contributions of the Paper
1. **Proposing a New Attention Mechanism**: The attention mechanism based on the screened Coulomb potential can better capture the interaction forces between nodes in the graph structure. This mechanism not only considers the distance between nodes but also introduces an exponential decay term, enabling the model to learn different types of short - range and long - range interactions.
2. **Constructing a Scalable Graph Attention Framework (CoulGAT)**: This framework allows for a flexible and scalable method to train graph attention models, especially suitable for large - scale graph data sets. By introducing residual connections, the model can be extended to a very deep number of layers, thereby improving the expressiveness of the model.
3. **Application and Verification**: The author conducted experiments on the CHAMPS data set, which contains scalar coupling constants between pairs of atoms in molecules. By comparing different model architectures (such as shallow and deep models), the effectiveness and superiority of the CoulGAT framework were verified.
### Summary
This paper aims to improve the interpretability and prediction performance of graph attention networks by introducing an attention mechanism based on the screened Coulomb potential, especially in applications in the field of quantum chemistry. By constructing the CoulGAT framework, the author shows how to use this new mechanism to improve the modeling and understanding of graph - structured data.