Abstract:Thermal properties of materials are essential to many applications of thermal electronic devices. Density functional theory (DFT) has shown capability in obtaining an accurate calculation. However, the expensive computational cost limits the application of the DFT method for high-throughput screening of materials. Recently, machine learning models, especially graph neural networks (GNNs), have demonstrated high accuracy in many material properties’ prediction, such as bandgap and formation energy, but fail to accurately predict heat capacity(CV) due to the limitation in capturing crystallographic features. In our study, we have implemented the material informatics transformer (MatInFormer) framework, which has been pretrained on lattice reconstruction tasks. This approach has shown proficiency in capturing essential crystallographic features. By concatenating these features with human-designed descriptors, we achieved a mean absolute error of 4.893 and 4.505 J/(mol K) in our predictions. Our findings underscore the efficacy of the MatInFormer framework in leveraging crystallography, augmented with additional information processing capabilities.

What problem does this paper attempt to address?

This paper mainly discusses how to use the Transformer model and crystallography pre-training to improve the multi-modal learning of material heat capacity. Although traditional density functional theory (DFT) can accurately calculate the thermal properties of materials, it is computationally expensive and not suitable for large-scale screening of new materials. In recent years, machine learning models, especially graph neural networks (GNNs), have made progress in predicting material properties, but they have limitations in capturing crystal structure features, resulting in inaccurate heat capacity predictions. In this study, the authors proposed a material information Transformer framework called MatInFormer, which is pre-trained on lattice reconstruction tasks to capture key crystallographic characteristics. By combining these characteristics with artificially designed descriptors, MatInFormer achieved an average absolute error (MAE) of 4.893 and 4.505 J/(mol K) in predicting heat capacity, outperforming other GNN models. The paper also emphasizes the importance of crystal systems, periodicity, and lattice parameters in predicting heat capacity, and points out the limitations of traditional GNN models in capturing these characteristics. MatInFormer can understand and encode crystal structures through pre-training, rather than solely relying on local atomic environments, thereby better capturing global information. In conclusion, this paper aims to address the limitations of GNNs in predicting material heat capacity and proposes a novel machine learning framework that utilizes Transformers and crystallographic knowledge to improve prediction accuracy.

Multimodal learning of heat capacity based on transformers and crystallography pretraining

Predicting the Effective Thermal Conductivities of Composite Materials and Porous Media by Machine Learning Methods

Machine Learning Prediction of Heat Capacity for Solid Inorganics

Transfer learning for materials informatics using crystal graph convolutional neural network

Materials Informatics Transformer: A Language Model for Interpretable Materials Properties Prediction

Novel machine learning framework for thermal conductivity prediction by crystal graph convolution embedded ensemble

MT-CGCNN: Integrating Crystal Graph Convolutional Neural Network with Multitask Learning for Material Property Prediction

CrysGraphFormer: an equivariant graph transformer for prediction of lattice thermal conductivity with interpretability

Predicting melting temperature of inorganic crystals via crystal graph neural network enhanced by transfer learning

Leveraging Large-scale Computational Database and Deep Learning for Accurate Prediction of Material Properties

Predicting thermoelectric properties from crystal graphs and material descriptors - first application for functional materials

CTGNN: Crystal Transformer Graph Neural Network for Crystal Material Property Prediction

Predicting lattice thermal conductivity of semiconductors from atomic-information-enhanced CGCNN combined with transfer learning

Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties

Universal Machine Learning Kohn-Sham Hamiltonian for Materials

Prediction of lattice thermal conductivity with two-stage interpretable machine learning

DeepCrysTet: A Deep Learning Approach Using Tetrahedral Mesh for Predicting Properties of Crystalline Materials

Transfer Learning in Materials Informatics: structure-property relationships through minimal but highly informative multimodal input

Unified deep learning network for enhanced accuracy in predicting thermal conductivity of bilayer graphene, hexagonal boron nitride, and their heterostructures

Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding