Abstract:Generative models are being increasingly used in drug discovery, very often coupled with absorption, distribution, metabolism, and excretion (ADME) bioassays or quantitative structure-activity relationship (QSAR) models to optimize a given set of properties. The molecules proposed by these algorithms are often revealed to be false positives; that is, they are predicted to be active and turn out to be inactive after synthesis and testing, mostly due to overoptimization of the predicted scores, which leads to an actual decrease or stagnation of the real scores. This behavior is also known as the "hacking" of the predictive models by the generative model during the optimization step. This issue is reminiscent of adversarial examples in machine learning and it can be seen as enunciated by Goodhart's law: "when a measure becomes a target, it ceases to be a good measure." This issue is even more apparent in a multiparameter optimization (MPO) case, where the models need to extrapolate outside the training set distribution because there are no known molecules satisfying all the objectives simultaneously in the initial training set. Experimental evaluation of this problem is a hard and expensive task since it requires synthesis and testing of the generated molecules. Thus, efforts have been made to develop in silico "oracles"─real-valued functions used as proxies for molecular properties─to help with the evaluation of these generative-model-based pipelines. However, these oracles have had a limited value so far because they are often too easy to model in comparison with biological assays and are usually limited to mono-objective cases. In this work, we introduce a simulator of multitarget assays using a smartly initialized neural network (NN) that returns continuous values for any input molecule. We use this oracle to replicate a real-world prospective lead optimization (LO) scenario. First, we trained predictive models on an initial small sample of molecules aimed at predicting their oracle values. Afterward, we generated new optimized molecules using the open-source GuacaMol package coupled with the previously built predictive models. Finally, we selected compounds matching the candidate drug target profile (CDTP) according to the predicted values and evaluated them by computing the true oracle values. We observed that even when the predictive models had excellent estimated performance metrics, the final selection still contained multiple false positives according to the NN-based oracle. Then, we evaluated the optimization behavior in mono- and bi-objective scenarios using either a logistic regression or a random forest predictive model. We also propose and evaluate several methods to help mitigate the hacking issue.

DrugGym: A testbed for the economics of autonomous drug discovery

Efficient Exploration of Chemical Space with Docking and Deep Learning

Drugst.One -- A plug-and-play solution for online systems medicine and network-based drug repurposing

Automating reward function configuration for drug design

Autoparty: Machine Learning-Guided Visual Inspection of Molecular Docking Results

DrugEx: Deep Learning Models and Tools for Exploration of Drug-Like Chemical Space

We Should at Least Be Able to Design Molecules That Dock Well

DrugMetric: quantitative drug-likeness scoring based on chemical space distance

Molecular Assays Simulator to Unravel Predictors Hacking in Goal-Directed Molecular Generations

DOCKSTRING: Easy Molecular Docking Yields Better Benchmarks for Ligand Design

DrugGen: Advancing Drug Discovery with Large Language Models and Reinforcement Learning Feedback

Retro Drug Design: From Target Properties to Molecular Structures

Assessing Methods and Obstacles in Chemical Space Exploration

High-throughput mechanistic screening of non-equilibrium inhibitors by a fully automated data analysis pipeline in early drug-discovery

Entering the era of computationally driven drug development

Customizable de novo design strategies for DOCK: Application to HIVgp41 and other therapeutic targets

A decision-theoretic approach to the evaluation of machine learning algorithms in computational drug discovery

IMPECCABLE: Integrated Modeling PipelinE for COVID Cure by Assessing Better LEads

A decision theoretic approach to model evaluation in computational drug discovery

Implementation of an Automated System Using Machine Learning Models to Accelerate the Process of In Silico Identification of Small Molecules As Drug Candidates

DrugSynthMC: an atom based generation of drug-like molecules with Monte Carlo Search