Abstract:Automated computational analysis of the vast chemical space is critical for numerous fields of research such as drug discovery and material science. Representation learning techniques have recently been employed with the primary objective of generating compact and informative numerical expressions of complex data. One approach to efficiently learn molecular representations is processing string-based notations of chemicals via natural language processing (NLP) algorithms. Majority of the methods proposed so far utilize SMILES notations for this purpose; however, SMILES is associated with numerous problems related to validity and robustness, which may prevent the model from effectively uncovering the knowledge hidden in the data. In this study, we propose SELFormer, a transformer architecture-based chemical language model that utilizes a 100% valid, compact and expressive notation, SELFIES, as input, in order to learn flexible and high-quality molecular representations. SELFormer is pre-trained on two million drug-like compounds and fine-tuned for diverse molecular property prediction tasks. Our performance evaluation has revealed that, SELFormer outperforms all competing methods, including graph learning-based approaches and SMILES-based chemical language models, on predicting aqueous solubility of molecules and adverse drug reactions. We also visualized molecular representations learned by SELFormer via dimensionality reduction, which indicated that even the pre-trained model can discriminate molecules with differing structural properties. We shared SELFormer as a programmatic tool, together with its datasets and pre-trained models. Overall, our research demonstrates the benefit of using the SELFIES notations in the context of chemical language modeling and opens up new possibilities for the design and discovery of novel drug candidates with desired features.

Self-referencing embedded strings (SELFIES): A 100% robust molecular string representation

Recent advances in the Self-Referencing Embedding Strings (SELFIES) library

SELFIES and the future of molecular string representations

Improving the reliability of molecular string representations for generative chemistry

fragSMILES: a Chemical String Notation for Advanced Fragment and Chirality Representation

SELFormer: Molecular Representation Learning via SELFIES Language Models

t-SMILES: a fragment-based molecular representation framework for de novo ligand design

t-SMILES: A Scalable Fragment-based Molecular Representation Framework for De Novo Molecule Generation

Fuzz testing molecular representation using deep variational anomaly generation

IMG2SMI: Translating Molecular Structure Images to Simplified Molecular-input Line-entry System

Gotta be SAFE: A New Framework for Molecular Design

SELF-BART : A Transformer-based Molecular Representation Model using SELFIES

Molecular generation by Fast Assembly of (Deep)SMILES fragments

t-SMILES 2: Hierarchical Structure Enhances the Generalizability of Linear Molecular Representation

Smirk: An Atomically Complete Tokenizer for Molecular Foundation Models

Reconstruction of lossless molecular representations from fingerprints

SAFE setup for generative molecular design

Chemical Language Model Linker: blending text and molecules with modular adapters

Is BigSMILES the Friend of Polymer Machine Learning?