Abstract:The advancement of Large Language Models (LLMs) for domain applications in fields such as materials science and engineering depends on the development of fine-tuning strategies that adapt models for specialized, technical capabilities. In this work, we explore the effects of Continued Pretraining (CPT), Supervised Fine-Tuning (SFT), and various preference-based optimization approaches, including Direct Preference Optimization (DPO) and Odds Ratio Preference Optimization (ORPO), on fine-tuned LLM performance. Our analysis shows how these strategies influence model outcomes and reveals that the merging of multiple fine-tuned models can lead to the emergence of capabilities that surpass the individual contributions of the parent models. We find that model merging leads to new functionalities that neither parent model could achieve alone, leading to improved performance in domain-specific assessments. Experiments with different model architectures are presented, including Llama 3.1 8B and Mistral 7B models, where similar behaviors are observed. Exploring whether the results hold also for much smaller models, we use a tiny LLM with 1.7 billion parameters and show that very small LLMs do not necessarily feature emergent capabilities under model merging, suggesting that model scaling may be a key component. In open-ended yet consistent chat conversations between a human and AI models, our assessment reveals detailed insights into how different model variants perform and show that the smallest model achieves a high intelligence score across key criteria including reasoning depth, creativity, clarity, and quantitative precision. Other experiments include the development of image generation prompts based on disparate biological material design concepts, to create new microstructures, architectural concepts, and urban design based on biological materials-inspired construction principles.

Fine-Tuning Language Models for Enhanced Diagram Generation: A Deep Learning Approach

Enhancing Large Language Model Comprehension of Material Phase Diagrams through Prompt Engineering and Benchmark Datasets

A Fine-Tuned Large Language Model for Domain-Specific with Reinforcement Learning

Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities

DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning

LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understanding

Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning

Fine-Tuning Large Language Models in Education

Fine Tuning LLM for Enterprise: Practical Guidelines and Recommendations

DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models

Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs

Fine-Tuned Large Language Model for Visualization System: A Study on Self-Regulated Learning in Education

Large Language Models for Data Annotation and Synthesis: A Survey

Supervised Knowledge Makes Large Language Models Better In-context Learners

Leveraging the Power of LLMs: A Fine-Tuning Approach for High-Quality Aspect-Based Summarization

On the Diagram of Thought

Fine-tuning Large Enterprise Language Models via Ontological Reasoning

Curriculum-style Data Augmentation for LLM-based Metaphor Detection

LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding

Fine-grained LLM Agent: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback

Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planning