Abstract:[Context] Large Language Models (LLMs) have shown good performance in several software development-related tasks such as program repair, documentation, code refactoring, debugging, and testing. Adapters are specialized, small modules designed for parameter efficient fine-tuning of LLMs for specific tasks, domains, or applications without requiring extensive retraining of the entire model. These adapters offer a more efficient way to customize LLMs for particular needs, leveraging the pre-existing capabilities of the large model. Merging LLMs and adapters has shown promising results for various natural language domains and tasks, enabling the use of the learned models and adapters without additional training for a new task. [Objective] This research proposes continual merging and empirically studies the capabilities of merged adapters in Code LLMs, specially for the Automated Program Repair (APR) task. The goal is to gain insights into whether and how merging task-specific adapters can affect the performance of APR. [Method] In our framework, MergeRepair, we plan to merge multiple task-specific adapters using three different merging methods and evaluate the performance of the merged adapter for the APR task. Particularly, we will employ two main merging scenarios for all three techniques, (i) merging using equal-weight averaging applied on parameters of different adapters, where all adapters are of equal importance; and (ii) our proposed approach, continual merging, in which we sequentially merge the task-specific adapters and the order and weight of merged adapters matter. By exploratory study of merging techniques, we will investigate the improvement and generalizability of merged adapters for APR. Through continual merging, we will explore the capability of merged adapters and the effect of task order, as it occurs in real-world software projects.

GRLMerger: an automatic approach for integrating GRL models

Traceability Management of GRL and SysML Models

A rule-based approach for the identification of quality improvement opportunities in GRL models

A questionnaire-based survey methodology for systematically validating goal-oriented models

Model Generation with LLMs: From Requirements to UML Sequence Diagrams

Extracting goal models from natural language requirement specifications

MindMerger: Efficient Boosting LLM Reasoning in non-English Languages

Composite-based Conflict Resolution in Merging Versions of UML Models

Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities

RuleMerger: Automatic Construction of Variability-Based Model Transformation Rules

Model Merging and Safety Alignment: One Bad Model Spoils the Bunch

Arcee's MergeKit: A Toolkit for Merging Large Language Models

AI based Multiagent Approach for Requirements Elicitation and Analysis

Git Merge Conflict Resolution Leveraging Strategy Classification and LLM

A transformation approach for collaboration based requirement models

MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repair

Realistic Evaluation of Model Merging for Compositional Generalization

RE-GAINS & EnChAnT: Intelligent Tool Manipulation Systems For Enhanced Query Responses

Towards Generating Executable Metamorphic Relations Using Large Language Models

Applying Model-based Requirements Engineering in Three Large European Collaborative Projects: An Experience Report

AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Merging