Abstract:Background: As large language models continue to expand in size and diversity, their substantial potential and the relevance of their applications are increasingly being acknowledged. The rapid advancement of these models also holds profound implications for the long-term design of stimulus-responsive materials used in drug delivery. Methods: The large model used Hugging Face's Transformers package with BigBird, Gemma, and GPT NeoX architectures. Pre-training used the PubChem dataset, and fine-tuning used QM7b. Chemist instruction training was based on Direct Preference Optimization. Drug Likeness, Synthetic Accessibility, and PageRank Scores were used to filter molecules. All computational chemistry simulations were performed using ORCA and Time-Dependent Density-Functional Theory. Results: To optimize large models for extensive dataset processing and comprehensive learning akin to a chemist's intuition, the integration of deeper chemical insights is imperative. Our study initially compared the performance of BigBird, Gemma, GPT NeoX, and others, specifically focusing on the design of photoresponsive drug delivery molecules. We gathered excitation energy data through computational chemistry tools and further investigated light-driven isomerization reactions as a critical mechanism in drug delivery. Additionally, we explored the effectiveness of incorporating human feedback into reinforcement learning to imbue large models with chemical intuition, enhancing their understanding of relationships involving -N=N- groups in the photoisomerization transitions of photoresponsive molecules. Conclusions: We implemented an efficient design process based on structural knowledge and data, driven by large language model technology, to obtain a candidate dataset of specific photoswitchable molecules. However, the lack of specialized domain datasets remains a challenge for maximizing model performance.

From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery

DrugAssist: A Large Language Model for Molecule Optimization

Benchmarking Large Language Models for Molecule Prediction Tasks

Can Large Language Models Empower Molecular Property Prediction?

Discovering Photoswitchable Molecules for Drug Delivery with Large Language Models and Chemist Instruction Training

MolX: Enhancing Large Language Models for Molecular Learning with A Multi-Modal Extension

Exploring the Potential of Large Language Models in Molecular Tasks: An Insightful Evaluation with GPT‐4

Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective

SynAsk: Unleashing the Power of Large Language Models in Organic Synthesis

Large Language Models as Molecular Design Engines

Utilizing Large Language Models in an iterative paradigm with Domain feedback for Zero-shot Molecule optimization

InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery

Crossing New Frontiers: Knowledge-Augmented Large Language Model Prompting for Zero-Shot Text-Based De Novo Molecule Design

Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation

3D Molecular Pocket-based Generation with Token-only Large Language Model

Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation

L+M-24: Building a Dataset for Language + Molecules @ ACL 2024

Molecular Graph Representation Learning Integrating Large Language Models with Domain-specific Small Models

Large Language Models are In-Context Molecule Learners

Large Language Models for Biomolecular Analysis: from Methods to Applications