Abstract:While a multitude of deep generative models have recently emerged there exists no best practice for their practically relevant validation. On the one hand, novel de novo-generated molecules cannot be refuted by retrospective validation (so that this type of validation is biased); but on the other hand prospective validation is expensive and then often biased by the human selection process. In this case study, we frame retrospective validation as the ability to mimic human drug design, by answering the following question: Can a generative model trained on early-stage project compounds generate middle/late-stage compounds de novo? To this end, we used experimental data that contains the elapsed time of a synthetic expansion following hit identification from five public (where the time series was pre-processed to better reflect realistic synthetic expansions) and six in-house project datasets, and used REINVENT as a widely adopted RNN-based generative model. After splitting the dataset and training REINVENT on early-stage compounds, we found that rediscovery of middle/late-stage compounds was much higher in public projects (at 1.60%, 0.64%, and 0.21% of the top 100, 500, and 5,000 scored generated compounds) than in in-house projects (where the values were 0.00%, 0.03%, and 0.04%, respectively). Similarly, average single nearest neighbour similarity between early- and middle/late-stage compounds in public projects was higher between active compounds than inactive compounds; however, for in-house projects the converse was true, which makes rediscovery (if so desired) more difficult. We hence show that the generative model recovers very few middle/late-stage compounds from real-world drug discovery projects, highlighting the fundamental difference between purely algorithmic design and drug discovery as a real-world process. Evaluating de novo compound design approaches appears, based on the current study, difficult or even impossible to do retrospectively. "Scientific Contribution" This contribution hence illustrates aspects of evaluating the performance of generative models in a real-world setting which have not been extensively described previously and which hopefully contribute to their further future development.

Stereochemistry-aware string-based molecular generation

PrefixMol: Target- and Chemistry-aware Molecule Design Via Prefix Embedding

Advances in Deep Learning-Based 3D Molecular Generative Models

A scaffold‐based deep generative model considering molecular stereochemical information

Improving the reliability of molecular string representations for generative chemistry

Generative models for molecular discovery: Recent advances and challenges

Directly Optimizing for Synthesizability in Generative Molecular Design using Retrosynthesis Models

Stereochemically-aware bioactivity descriptors for uncharacterized chemical compounds

Molecular design in drug discovery: a comprehensive review of deep generative models

Machine learning-aided generative molecular design

Enhancing molecular design efficiency: Uniting language models and generative networks with genetic algorithms

Rethinking Molecular Design: Integrating Latent Variable and Auto-Regressive Models for Goal Directed Generation

Latent Chemical Space Searching for Plug-in Multi-objective Molecule Generation

Adaptive language model training for molecular design

Generative Organic Electronic Molecular Design Informed by Quantum Chemistry

Genetic Algorithm-Based Receptor Ligand: A Genetic Algorithm-Guided Generative Model to Boost the Novelty and Drug-Likeness of Molecules in a Sampling Chemical Space.

AnoChem: Prediction of chemical structural abnormalities based on machine learning models

On The Difficulty of Validating Molecular Generative Models Realistically: A Case Study on Public and Proprietary Data

Navigating the frontier of drug-like chemical space with cutting-edge generative AI models

Assessing Methods and Obstacles in Chemical Space Exploration