Abstract:Abstract Motivation Molecular representation learning plays an indispensable role in crucial tasks such as property prediction and drug design. Despite the notable achievements of Molecular Pre-training Models (MPMs), current methods often fail to capture both the structural and feature semantics of molecular graphs. Moreover, while graph contrastive learning has unveiled new prospects, existing augmentation techniques often struggle to retain their core semantics. To overcome these limitations, we propose a gradient-compensated encoder parameter perturbation approach, ensuring efficient and stable feature augmentation. By merging enhancement strategies grounded in attribute masking and parameter perturbation, we introduce MoleMCL, a new MOLEcular pre-training model based on Multi-level Contrastive Learning. Results Experimental results demonstrate that MoleMCL adeptly dissects the structure and feature semantics of molecular graphs, surpassing current state-of-the-art models in molecular prediction tasks, paving a novel avenue for molecular modeling. Availability and implementation The code and data underlying this work are available in GitHub at https://github.com/BioSequenceAnalysis/MoleMCL. Contact 23020221154148@stu.xmu.edu.cn Supplementary information Supplementary data are available at Bioinformatics online.

What problem does this paper attempt to address?

The paper aims to address two major issues in molecular representation learning: 1. **Insufficient capture of structural and feature semantics**: Existing molecular pre-training models often struggle to simultaneously capture the structural and feature semantics of molecular graphs. This limitation affects the model's performance in critical tasks such as property prediction and drug design. 2. **Core semantic retention challenge in contrastive learning**: Although contrastive learning offers a new perspective for molecular pre-training, existing data augmentation techniques often struggle to effectively enhance while retaining the core semantics of molecules. For example, structure-based augmentation methods may fail to handle molecular activity cliffs correctly, where structurally similar molecules may have drastically different properties. To address the above issues, the paper proposes MoleMCL, a molecular pre-training framework based on multi-level contrastive learning. Specifically, MoleMCL includes the following innovations: - **Gradient compensation parameter perturbation strategy**: To perturb model parameters more scientifically and ensure stability and effectiveness during the contrastive learning process, the paper introduces a gradient compensation scheme. This method uses gradients from previous contrastive learning stages to guide parameter perturbation, thereby avoiding semantic disruption that simple Gaussian noise perturbation might cause. - **Augmentation strategy combining attribute masking and parameter perturbation**: To simultaneously capture the structural and feature semantics of molecular graphs, the paper combines attribute masking techniques with parameter perturbation, proposing the MoleMCL framework. This integrated approach enables the model to learn molecular representations at different levels. Experimental results show that MoleMCL performs excellently in molecular property prediction tasks, outperforming current state-of-the-art models. It can serve as a foundational pre-training task for various graph neural network architectures, enhancing their performance. Additionally, molecular retrieval experiments further validate that MoleMCL can generate chemically meaningful representations, demonstrating the effectiveness of the proposed multi-level contrastive learning method.

MoleMCL: a multi-level contrastive learning framework for molecular pre-training

Molecular contrastive learning of representations via graph neural networks

Mole-BERT: Rethinking Pre-training Graph Neural Networks for Molecules

MoCL: Data-driven Molecular Fingerprint via Knowledge-aware Contrastive Learning from Molecular Graph

A Multi-view Molecular Pre-training with Generative Contrastive Learning

Molecular Graph Contrastive Learning with Parameterized Explainable Augmentations

Cross‐Modal Graph Contrastive Learning with Cellular Images

MolMVC: Enhancing molecular representations for drug-related tasks through multi-view contrastive learning

MolCloze - A Unified Cloze-style Self-supervised Molecular Structure Learning Model for Chemical Property Prediction.

Multilingual Molecular Representation Learning via Contrastive Pre-training

MMCL-CPI: A multi-modal compound-protein interaction prediction model incorporating contrastive learning pre-training

Molecular Contrastive Learning with Chemical Element Knowledge Graph

Multimodal Molecular Pretraining via Modality Blending

UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning

Pretraining Graph Transformer for Molecular Representation with Fusion of Multimodal Information

Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast

Attention-wise masked graph contrastive learning for predicting molecular property

Pre-trained Molecular Language Models with Random Functional Group Masking

MolLM : a unified language model for integrating biomedical text with 2D and 3D molecular representations

3D-Mol: A Novel Contrastive Learning Framework for Molecular Property Prediction with 3D Information