Abstract:The advancement of Large Language Models (LLMs) for domain applications in fields such as materials science and engineering depends on the development of fine-tuning strategies that adapt models for specialized, technical capabilities. In this work, we explore the effects of Continued Pretraining (CPT), Supervised Fine-Tuning (SFT), and various preference-based optimization approaches, including Direct Preference Optimization (DPO) and Odds Ratio Preference Optimization (ORPO), on fine-tuned LLM performance. Our analysis shows how these strategies influence model outcomes and reveals that the merging of multiple fine-tuned models can lead to the emergence of capabilities that surpass the individual contributions of the parent models. We find that model merging leads to new functionalities that neither parent model could achieve alone, leading to improved performance in domain-specific assessments. Experiments with different model architectures are presented, including Llama 3.1 8B and Mistral 7B models, where similar behaviors are observed. Exploring whether the results hold also for much smaller models, we use a tiny LLM with 1.7 billion parameters and show that very small LLMs do not necessarily feature emergent capabilities under model merging, suggesting that model scaling may be a key component. In open-ended yet consistent chat conversations between a human and AI models, our assessment reveals detailed insights into how different model variants perform and show that the smallest model achieves a high intelligence score across key criteria including reasoning depth, creativity, clarity, and quantitative precision. Other experiments include the development of image generation prompts based on disparate biological material design concepts, to create new microstructures, architectural concepts, and urban design based on biological materials-inspired construction principles.

Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs

Enhancing the Reasoning Capabilities of Small Language Models via Solution Guidance Fine-Tuning

Large Language Model for Causal Decision Making

Learning Global Controller in Latent Space for Parameter-Efficient Fine-Tuning

Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models

I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses

An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models

Improving Large Language Model Fine-tuning for Solving Math Problems

LLM4Causal: Democratized Causal Tools for Everyone via Large Language Model

Position-Aware Parameter Efficient Fine-Tuning Approach for Reducing Positional Bias in LLMs

Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling

Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper

STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data

MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning

MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models

Eliciting Causal Abilities in Large Language Models for Reasoning Tasks

LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models

Customizing Large Language Model Generation Style using Parameter-Efficient Finetuning

Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities