Abstract:The combination of deep learning algorithm and materials science has made significant progress in predicting novel materials and understanding various behaviours of materials. Here, we introduced a new model called as the Crystal Transformer Graph Neural Network (CTGNN), which combines the advantages of Transformer model and graph neural networks to address the complexity of structure-properties relation of material data. Compared to the state-of-the-art models, CTGNN incorporates the graph network structure for capturing local atomic interactions and the dual-Transformer structures to model intra-crystal and inter-atomic relationships comprehensively. The benchmark carried on by the proposed CTGNN indicates that CTGNN significantly outperforms existing models like CGCNN and MEGNET in the prediction of formation energy and bandgap properties. Our work highlights the potential of CTGNN to enhance the performance of properties prediction and accelerates the discovery of new materials, particularly for perovskite materials.

What problem does this paper attempt to address?

The paper aims to address a key issue in materials science—how to more accurately predict the properties of materials, particularly the formation energy and band gap of crystalline materials. To tackle this problem, the authors propose a new model called the "Crystal Transformer Graph Neural Network" (CTGNN). This model combines the advantages of the Transformer model and Graph Neural Networks (GNN), effectively capturing the complex relationships between structure and properties in material data. Specifically, CTGNN improves prediction accuracy through the following methods: 1. **Introduction of Angle Encoder**: In addition to traditional distance information, CTGNN also considers the angular information between atoms, which helps to describe the spatial relationships of atoms more precisely. 2. **Dual Transformer Structure**: CTGNN includes two Transformer structures, one for modeling the interactions within atoms (i.e., the direct chemical environment of atoms) and another for analyzing the relationships between atoms, thereby comprehensively understanding material behavior on both local and broader scales. 3. **Optimized Model Structure**: Compared to other Transformer-based models, CTGNN avoids introducing excessive complexity while ensuring model performance, which is beneficial for model training. Benchmark tests on widely used material databases show that CTGNN significantly outperforms existing models such as CGCNN and MEGNET in predicting formation energy and band gap. It performs particularly well in the prediction of perovskite materials, indicating that CTGNN has significant potential value for the discovery of such materials. Additionally, the paper validates the importance of angle encoding and the Transformer architecture in improving model accuracy through ablation studies. In summary, CTGNN provides an efficient and accurate method for predicting material properties and is expected to accelerate the discovery process of new materials.

CTGNN: Crystal Transformer Graph Neural Network for Crystal Material Property Prediction

Improving Crystal Property Prediction from a Multiplex Graph Perspective

A geometric-information-enhanced crystal graph network for predicting properties of materials

Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties

CrysToGraph: A Comprehensive Predictive Model for Crystal Materials Properties and the Benchmark

Graph Neural Networks for an Accurate and Interpretable Prediction of the Properties of Polycrystalline Materials

MT-CGCNN: Integrating Crystal Graph Convolutional Neural Network with Multitask Learning for Material Property Prediction

A machine learning‐based crystal graph network and its application in development of functional materials

Study of Crystal Property Prediction Based on Dual Attention Mechanism and Transfer Learning

Study of crystal properties based on attention mechanism and crystal graph convolutional neural network

Enhancing material property prediction with ensemble deep graph convolutional networks

ADA-GNN: Atom-Distance-Angle Graph Neural Network for Crystal Material Property Prediction

General Graph Neural Network-Based Model To Accurately Predict Cocrystal Density and Insight from Data Quality and Feature Representation

BNM-CDGNN: Batch Normalization Multilayer Perceptron Crystal Distance Graph Neural Network for Excellent-Performance Crystal Property Prediction

Orbital Graph Convolutional Neural Network for Material Property Prediction

Leveraging Large-scale Computational Database and Deep Learning for Accurate Prediction of Material Properties

Chemical Environment Adaptive Learning for Optical Band Gap Prediction of Doped Graphitic Carbon Nitride Nanosheets

Graph Neural Network for Predicting the Effective Properties of Polycrystalline Materials: A Comprehensive Analysis

Graph Neural Networks Based Deep Learning for Predicting Structural and Electronic Properties

Predicting Material Properties Using a 3D Graph Neural Network with Invariant Local Descriptors

GPT-assisted learning of structure-property relationships by graph neural networks: Application to rare-earth doped phosphors