Abstract:Machine learning (ML) methods have great potential to transform chemical discovery by accelerating the exploration of chemical space and drawing scientific insights from data. However, modern chemical reaction ML models, such as those based on graph neural networks (GNNs), must be trained on a large amount of labelled data in order to avoid overfitting the data and thus possessing low accuracy and transferability. In this work, we propose a strategy to leverage unlabelled data to learn accurate ML models for small labelled chemical reaction data. We focus on an old and prominent problem-classifying reactions into distinct families-and build a GNN model for this task. We first pretrain the model on unlabelled reaction data using unsupervised contrastive learning and then fine-tune it on a small number of labelled reactions. The contrastive pretraining learns by making the representations of two augmented versions of a reaction similar to each other but distinct from other reactions. We propose chemically consistent reaction augmentation methods that protect the reaction center and find they are the key for the model to extract relevant information from unlabelled data to aid the reaction classification task. The transfer learned model outperforms a supervised model trained from scratch by a large margin. Further, it consistently performs better than models based on traditional rule-driven reaction fingerprints, which have long been the default choice for small datasets, as well as those based on reaction fingerprints derived from masked language modelling. In addition to reaction classification, the effectiveness of the strategy is tested on regression datasets; the learned GNN-based reaction fingerprints can also be used to navigate the chemical reaction space, which we demonstrate by querying for similar reactions. The strategy can be readily applied to other predictive reaction problems to uncover the power of unlabelled data for learning better models with a limited supply of labels.

Predicting Reaction Conditions from Limited Data through Active Transfer Learning

Uncertainty-calibrated deep learning for rapid identification of reaction mechanisms

Data augmentation and transfer learning strategies for reaction prediction in low chemical data regimes

Using Active Learning to Develop Machine Learning Models for Reaction Yield Prediction

Predicting reaction performance in C–N cross-coupling using machine learning

Predicting Three-Component Reaction Outcomes from 40k Miniaturized Reactant Combinations

Heck Reaction Prediction Using a Transformer Model Based on a Transfer Learning Strategy

Beyond the Typical: Modeling Rare Plausible Patterns in Chemical Reactions by Leveraging Sequential Mixture-of-Experts

Improving machine learning performance on small chemical reaction data with unsupervised contrastive pretraining

Predicting outcomes of catalytic reactions using machine learning

Generic Interpretable Reaction Condition Predictions with Open Reaction Condition Datasets and Unsupervised Learning of Reaction Center

Data-Efficient, Chemistry-Aware Machine Learning Predictions of Diels-Alder Reaction Outcomes

Data-efficient modeling of catalytic reactions via enhanced sampling and on-the-fly learning of machine learning potentials

Data-Efficient, Chemistry-Aware Machine Learning Predictions of Diels–Alder Reaction Outcomes

Unified Deep Learning Model for Multitask Reaction Predictions with Explanation

Machine learning models predict calculation outcomes with the transferability necessary for computational catalysis

A machine-learning tool to predict substrate-adaptive conditions for Pd-catalyzed C–N couplings

Predicting Chemical Reaction Barriers with a Machine Learning Model

Harnessing Data Augmentation and Normalization Preprocessing to Improve the Performance of Chemical Reaction Predictions of Data-Driven Model

Predictive chemistry: Machine learning for reaction deployment, reaction development, and reaction discovery

A transferable active-learning strategy for reactive molecular force fields