From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models

Qianyu He,Jie Zeng,Qianxi He,Jiaqing Liang,Yanghua Xiao

2024-06-18

Abstract:It is imperative for Large language models (LLMs) to follow instructions with elaborate requirements (i.e. Complex Instructions Following). Yet, it remains under-explored how to enhance the ability of LLMs to follow complex instructions with multiple constraints. To bridge the gap, we initially study what training data is effective in enhancing complex constraints following abilities. We found that training LLMs with instructions containing multiple constraints enhances their understanding of complex instructions, especially those with lower complexity levels. The improvement can even generalize to compositions of out-of-domain constraints. Additionally, we further propose methods addressing how to obtain and utilize the effective training data. Finally, we conduct extensive experiments to prove the effectiveness of our methods in terms of overall performance and training efficiency. We also demonstrate that our methods improve models' ability to follow instructions generally and generalize effectively across out-of-domain, in-domain, and adversarial settings, while maintaining general capabilities.

Computation and Language

What problem does this paper attempt to address?

The problem this paper attempts to address is: How to enhance the multi-constraint instruction-following ability of large language models (LLMs) when faced with complex instructions. Specifically, the paper explores the following two main questions:

1. **What kind of training data can effectively improve the ability to follow complex instructions?**
- Through empirical research, the paper finds that training LLMs with instructions containing multiple constraints (i.e., composite data) is more effective in improving the model's understanding of complex instructions compared to training with instructions containing a single constraint (i.e., atomic data), especially when dealing with low-complexity instructions.

2. **How to obtain and utilize effective training data?**
- The paper proposes a technique based on a discriminative method to generate high-quality composite data and introduces a contrastive method that uses positive and negative samples for reinforcement learning fine-tuning to further enhance the model's ability to follow complex instructions.

Through these studies, the paper aims to systematically improve the ability of LLMs to handle complex instructions in practical applications, ensuring that the model maintains good generalization performance across various domains and adversarial settings.