Abstract:Self-supervised neural language models have recently achieved unprecedented success, from natural language processing to learning the languages of biological sequences and organic molecules. These models have demonstrated superior performance in the generation, structure classification, and functional predictions for proteins and molecules with learned representations. However, most of the masking-based pre-trained language models are not designed for generative design, and their black-box nature makes it difficult to interpret their design logic. Here we propose BLMM Crystal Transformer, a neural network based probabilistic generative model for generative and tinkering design of inorganic materials. Our model is built on the blank filling language model for text generation and has demonstrated unique advantages in learning the "materials grammars" together with high-quality generation, interpretability, and data efficiency. It can generate chemically valid materials compositions with as high as 89.7\% charge neutrality and 84.8\% balanced electronegativity, which are more than 4 and 8 times higher compared to a pseudo random sampling baseline. The probabilistic generation process of BLMM allows it to recommend tinkering operations based on learned materials chemistry and makes it useful for materials doping. Combined with the TCSP crysal structure prediction algorithm, We have applied our model to discover a set of new materials as validated using DFT calculations. Our work thus brings the unsupervised transformer language models based generative artificial intelligence to inorganic materials. A user-friendly web app has been developed for computational materials doping and can be accessed freely at \url{<a class="link-external link-http" href="http://www.materialsatlas.org/blmtinker" rel="external noopener nofollow">this http URL</a>}.

Generative Chemical Transformer: Neural Machine Learning of Molecular Geometric Structures from Chemical Language via Attention

PrefixMol: Target- and Chemistry-aware Molecule Design Via Prefix Embedding

Probabilistic generative transformer language models for generative design of molecules

Generative Organic Electronic Molecular Design Informed by Quantum Chemistry

GeoT: A Geometry-aware Transformer for Reliable Molecular Property Prediction and Chemically Interpretable Representation Learning

C5T5: Controllable Generation of Organic Molecules with Transformers

Crystal Composition Transformer: Self-Learning Neural Language Model for Generative and Tinkering Design of Materials

Crystal Transformer: Self-learning neural language model for Generative and Tinkering Design of Materials

CMGN: a conditional molecular generation net to design target-specific molecules with desired properties

MolGPT: Molecular Generation Using a Transformer-Decoder Model

A Transformer Based Generative Chemical Language AI Model for Structural Elucidation of Organic Compounds

Describe Molecules by a Heterogeneous Graph Neural Network with Transformer-like Attention for Supervised Property Predictions

LigGPT: Molecular Generation using a Transformer-Decoder Model

Exploration of crystal chemical space using text-guided generative artificial intelligence

ChemistGA: A Chemical Synthesizable Accessible Molecular Generation Algorithm for Real-World Drug Discovery

Exhaustive local chemical space exploration using a transformer model

cMolGPT: A Conditional Generative Pre-Trained Transformer for Target-Specific De Novo Molecular Generation

Chemical Language Models for Molecular Design

A Transformer-based Generative Model for De Novo Molecular Design

MatterGPT: A Generative Transformer for Multi-Property Inverse Design of Solid-State Materials