Abstract:Reinforcement learning (RL) is a powerful and flexible paradigm for searching for solutions in high-dimensional action spaces. However, bridging the gap between playing computer games with thousands of simulated episodes and solving real scientific problems with complex and involved environments (up to actual laboratory experiments) requires improvements in terms of sample efficiency to make the most of expensive information. The discovery of new drugs is a major commercial application of RL, motivated by the very large nature of the chemical space and the need to perform multiparameter optimization (MPO) across different properties. In silico methods, such as virtual library screening (VS) and de-novo molecular generation with RL, show great promise in accelerating this search. However, incorporation of increasingly complex computational models in these workflows requires increasing sample efficiency. Here, we introduce an active learning system linked with an RL model (RL-AL) for molecular design, which aims to improve the sample-efficiency of the optimization process. We identity and characterize unique challenges combining RL and AL, investigate the interplay between the systems, and develop a novel AL approach to solve the MPO problem. Our approach greatly expedites the search for novel solutions relative to baseline-RL for simple ligand- and structure-based oracle functions, with a 5 – 66-fold increase in hits generated for a fixed oracle budget and a 4 - 64-fold reduction in computational time to find a specific number of hits. Furthermore, compounds discovered through RL-AL display substantial enrichment of a multi-parameter scoring objective, indicating superior efficacy in curating high-scoring compounds, without a reduction in output diversity. This significant acceleration improves the feasibility of oracle functions that have largely been overlooked in RL due to high computational costs, for example free energy perturbation methods, and in principle is applicable to any RL domain.

Modern hit-finding with structure-guided de novo design: identification of novel nanomolar A2A receptor ligands using reinforcement learning

Modern hit-finding with structure-guided de novo design: identification of novel nanomolar adenosine A2A receptor ligands using reinforcement learning

De Novo Structure-Based Drug Design Using Deep Learning

Automated design of multi-target ligands by generative deep learning

Widely Used and Fast De Novo Drug Design by a Protein Sequence-Based Reinforcement Learning Model

De Novo Design of Nurr1 Agonists via Fragment-Augmented Generative Deep Learning in Low-Data Regime

Leveraging molecular structure and bioactivity with chemical language models for de novo drug design

Molecular De Novo Design through Deep Reinforcement Learning

De novo generation of dual-target ligands using adversarial training and reinforcement learning

Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learning

De Novo Drug Design Using Transformer-Based Machine Translation and Reinforcement Learning of an Adaptive Monte Carlo Tree Search

Structure-based <i>de novo</i> drug design using 3D deep generative models

Genetic Algorithm-Based Receptor Ligand: A Genetic Algorithm-Guided Generative Model to Boost the Novelty and Drug-Likeness of Molecules in a Sampling Chemical Space.

Utilizing reinforcement learning for de novo drug design

Generation of Rational Drug-Like Molecular Structures Through a Multiple-Objective Reinforcement Learning Framework

De novo design of protein target specific scaffold-based Inhibitors via Reinforcement Learning

Molecular generation strategy and optimization based on A2C reinforcement learning in de novo drug design

Sample Efficient Reinforcement Learning with Active Learning for Molecular Design

De novo drug design as GPT language modeling: large chemistry models with supervised and reinforcement learning

Prospective de novo drug design with deep interactome learning