Abstract:Large Language Models (LLMs) with their strong task-handling capabilities have shown remarkable advancements across a spectrum of fields, moving beyond natural language understanding. However, their proficiency within the chemistry domain remains restricted, especially in solving professional molecule-related tasks. This challenge is attributed to their inherent limitations in comprehending molecules using only common textual representations, i.e., SMILES strings. In this study, we seek to enhance the ability of LLMs to comprehend molecules by equipping them with a multi-modal external module, namely MolX. In particular, instead of directly using a SMILES string to represent a molecule, we utilize specific encoders to extract fine-grained features from both SMILES string and 2D molecular graph representations for feeding into an LLM. Moreover, a handcrafted molecular fingerprint is incorporated to leverage its embedded domain knowledge. Then, to establish an alignment between MolX and the LLM's textual input space, the whole model in which the LLM is frozen, is pre-trained with a versatile strategy including a diverse set of tasks. Experimental evaluations show that our proposed method outperforms baselines across 4 downstream molecule-related tasks ranging from molecule-to-text translation to retrosynthesis, with and without fine-tuning the LLM, while only introducing a small number of trainable parameters 0.53% and 0.82%, respectively.

What problem does this paper attempt to address?

The paper aims to address the limitations of large language models (LLMs) in the field of chemistry, particularly in handling specialized tasks related to molecules. Specifically, although LLMs have demonstrated strong capabilities across various tasks and have made significant progress in multiple domains beyond natural language understanding, they still face limitations in understanding and executing specialized tasks in the field of chemistry. This limitation is mainly due to LLMs relying solely on common text representations—Simplified Molecular Input Line Entry System (SMILES strings) to understand molecular structures, which is insufficient for the models to fully capture the complexity and details of molecules. To address this issue, the paper proposes a new framework called MolX, which enhances LLMs to better understand molecules from multiple representations (including SMILES strings and 2D molecular graphs) and leverages handcrafted molecular fingerprints to integrate domain knowledge. MolX includes a multimodal external module and a pre-training strategy to align this module with the text input space of LLMs. This approach not only improves the performance of LLMs on molecule-related tasks but also introduces a very small number of trainable parameters, only 0.53% to 0.82%. Specifically, MolX uses a pre-trained SMILES encoder and a graph encoder based on Graph Neural Networks (GNN) to extract features from SMILES strings and 2D molecular graphs, respectively, and combines these features with handcrafted molecular fingerprints to form a unified embedding vector. Additionally, the entire model employs a diversified pre-training strategy, including a series of tasks, to further enhance the model's capability in molecular understanding. Experimental results show that in four different downstream molecule-related tasks, the proposed method significantly outperforms baseline methods, whether or not the LLMs are fine-tuned. This demonstrates the effectiveness of the MolX framework and its potential in enhancing the ability of LLMs to handle molecule-related tasks.

MolX: Enhancing Large Language Models for Molecular Learning with A Multi-Modal Extension

MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter

MolLM : a unified language model for integrating biomedical text with 2D and 3D molecular representations

Can Large Language Models Empower Molecular Property Prediction?

Y-Mol: A Multiscale Biomedical Knowledge-Guided Large Language Model for Drug Development

InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery

Benchmarking Large Language Models for Molecule Prediction Tasks

Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planning

Molecular Graph Representation Learning Integrating Large Language Models with Domain-specific Small Models

ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area

DrugAssist: A Large Language Model for Molecule Optimization

DrugLLM: Open Large Language Model for Few-shot Molecule Generation

Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective

MolMetaLM: a Physicochemical Knowledge-Guided Molecular Meta Language Model

Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation

Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation

Towards 3D Molecule-Text Interpretation in Language Models

Large Language Models are In-Context Molecule Learners

SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration

Large Language Models as Molecular Design Engines

MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild