SolvBERT for solvation free energy and solubility prediction: a demonstration of an NLP model for predicting the properties of molecular complexes

Jiahui Yu,Chengwei Zhang,Yingying Cheng,Yun-Fang Yang,Yuan-Bin She,Fengfan Liu,Weike Su,An Su
DOI: https://doi.org/10.1039/d2dd00107a
2023-01-31
Digital Discovery
Abstract:Deep learning models based on NLP, mainly the Transformer family, have been successfully applied to solve many chemistry-related problems, but their applications are mostly limited to chemical reactions. Meanwhile, solvation is an important concept in physical and organic chemistry, describing the interaction of solutes and solvents. In this study, we introduced the SolvBERT model, which read the solute and solvent through the SMILES representation of their combination. SolvBERT was pretrained in an unsupervised learning fashion using a large database of computational solvation free energies. The pretrained model could be used to predict the experimental solvation free energy or solubility, depending on the fine-tuning database. To the best of our knowledge, this multi-task prediction capability has not been observed in previously developed graph-based models for predicting the properties of molecular complexes. Furthermore, the performance of our SolvBERT in predicting solvation free energy was comparable to the state-of-the-art graph-based model DMPNN, mainly due to the clustering feature of the pretraining phase of the model, as demonstrated by the TMAP visualization algorithm. Last but not least, our SolvBERT outperformed the recently-develop GNN-Transformer hybrid model, GROVER, in predicting a set of experimentally evaluated solubility data with out-of-sample solute-solvent combinations.
What problem does this paper attempt to address?