Abstract:The advancement of Large Language Models (LLMs) for domain applications in fields such as materials science and engineering depends on the development of fine-tuning strategies that adapt models for specialized, technical capabilities. In this work, we explore the effects of Continued Pretraining (CPT), Supervised Fine-Tuning (SFT), and various preference-based optimization approaches, including Direct Preference Optimization (DPO) and Odds Ratio Preference Optimization (ORPO), on fine-tuned LLM performance. Our analysis shows how these strategies influence model outcomes and reveals that the merging of multiple fine-tuned models can lead to the emergence of capabilities that surpass the individual contributions of the parent models. We find that model merging leads to new functionalities that neither parent model could achieve alone, leading to improved performance in domain-specific assessments. Experiments with different model architectures are presented, including Llama 3.1 8B and Mistral 7B models, where similar behaviors are observed. Exploring whether the results hold also for much smaller models, we use a tiny LLM with 1.7 billion parameters and show that very small LLMs do not necessarily feature emergent capabilities under model merging, suggesting that model scaling may be a key component. In open-ended yet consistent chat conversations between a human and AI models, our assessment reveals detailed insights into how different model variants perform and show that the smallest model achieves a high intelligence score across key criteria including reasoning depth, creativity, clarity, and quantitative precision. Other experiments include the development of image generation prompts based on disparate biological material design concepts, to create new microstructures, architectural concepts, and urban design based on biological materials-inspired construction principles.

Exploring the Benefits of Domain-Pretraining of Generative Large Language Models for Chemistry

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains.

Leveraging large language models for predictive chemistry

What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks

Exploring the Potential of Large Language Models in Molecular Tasks: An Insightful Evaluation with GPT‐4

Fine-tuning Large Language Models for Chemical Text Mining

LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset

Are large language models superhuman chemists?

Developing Large Language Models for Quantum Chemistry Simulation Input Generation

Assessment of Fine-Tuned Large Language Models for Real-World Chemistry and Material Science Applications

PharmaGPT: Domain-Specific Large Language Models for Bio-Pharmaceutical and Chemistry

On Domain-Specific Post-Training for Multimodal Large Language Models

Large Language Models are Catalyzing Chemistry Education

Specialising and Analysing Instruction-Tuned and Byte-Level Language Models for Organic Reaction Prediction

Large Language Models for Inorganic Synthesis Predictions

Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities

ChemBERTa-2: Towards Chemical Foundation Models

Augmenting large language models with chemistry tools