LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset

Botao Yu,Frazier N. Baker,Ziqi Chen,Xia Ning,Huan Sun
2024-08-10
Abstract:Chemistry plays a crucial role in many domains, such as drug discovery and material science. While large language models (LLMs) such as GPT-4 exhibit remarkable capabilities on natural language processing tasks, existing research indicates that their performance on chemistry tasks is discouragingly low. In this paper, however, we demonstrate that our developed LLMs can achieve very strong results on a comprehensive set of chemistry tasks, outperforming the most advanced GPT-4 and Claude 3 Opus by a substantial margin. To accomplish this, we propose SMolInstruct, a large-scale, comprehensive, and high-quality dataset for instruction tuning. It contains 14 selected chemistry tasks and over three million samples, laying a solid foundation for training and evaluating LLMs for chemistry. Using SMolInstruct, we fine-tune a set of open-source LLMs, among which, we find that Mistral serves as the best base model for chemistry tasks. Our analysis further demonstrates the critical role of the proposed dataset in driving the performance improvements.
Artificial Intelligence,Computational Engineering, Finance, and Science,Computation and Language
What problem does this paper attempt to address?
The paper aims to address the poor performance of large language models (LLMs) on chemical tasks. Although existing large language models like GPT-4 perform excellently on natural language processing tasks, their performance on chemistry-related tasks is far below expectations. To solve this problem, the researchers proposed a new large-scale, comprehensive, high-quality instruction-tuning dataset called SMolInstruct and fine-tuned multiple open-source large language models based on this dataset to improve their performance on chemical tasks. Specifically, the main contributions of the paper include: 1. **Construction of the SMolInstruct dataset**: It contains over 3 million samples covering 14 types of chemical tasks, including name conversion, property prediction, molecular description generation, and chemical reaction knowledge. 2. **Development of the LlaSMol series models**: By fine-tuning four open-source large language models (Galactica, Llama 2, Code Llama, and Mistral), a series of models optimized specifically for chemical tasks were created. 3. **Significant improvement in model performance**: Experimental results show that the fine-tuned LlaSMol models significantly outperform the untuned base models and the current state-of-the-art GPT-4 and Claude 3 Opus models on various chemical tasks. 4. **Validation of the dataset's importance**: The study demonstrates that training with the SMolInstruct dataset can significantly enhance model performance on chemical tasks, further proving the value of this dataset. Through this work, the researchers showcase the potential of large language models in the field of chemistry and lay the foundation for further research.