Abstract:Aligning large language models (LLMs) with human expectations requires high-quality instructional dialogues, which usually require instructions that are diverse and in-depth. Existing methods leverage two LLMs to interact for automatic collection: one simulating a user to pose instructions, and the other acting as a system agent to respond. However, these user simulators struggle to model the rules behind how dialogues can pose different instructions without explicit guidance, resulting in general instructions. In this paper, we propose to explicitly capture the complex rules to help the user simulator pose diverse and in-depth instruction. Specifically, we first induce high-level instruction strategies from various real instruction dialogues serving as rules. Afterward, different possible strategies are applied to the newly given dialogue scenario deductively to pose various instructions. Experimental results show that our method can generate diverse and in-depth instructions. The constructed multi-turn instructional dialogues can outperform competitive baselines on the downstream chat model.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to generate diverse and in - depth instructions to construct high - quality multi - turn conversations, so as to better fine - tune large language models (LLMs). Specifically, the existing methods have the following problems when automatically generating instructions: 1. **Lack of diversity**: The existing user simulators are difficult to capture the complex rules behind different instructions in the conversation, resulting in the generated instructions being more general and lacking in diversity. 2. **Insufficient depth**: The generated instructions fail to fully explore the logical flow and details in the conversation history, resulting in insufficient depth of the conversation. To solve these problems, the paper proposes a method based on Inductive - Deductive strategy rE - use for instruction generation (IDEAS), aiming to generate diverse and in - depth instructions by explicitly modeling different directions of the conversation flow. Specifically, IDEAS consists of two stages: - **Inductive stage**: Extract high - level instruction strategies from real human - machine conversations. Through inductive reasoning on a large amount of conversation data, summarize the general rules that can guide the conversation flow. - **Deductive stage**: Apply these strategies in new conversation scenarios and generate specific instructions through deductive reasoning. The user simulator selects appropriate strategies according to the current conversation history to generate instructions. The experimental results show that IDEAS can generate higher - quality instructions, thereby improving the performance of downstream chat models. The following are the main contributions of the paper: 1. Proposed the first inductive - deductive strategy re - use method for generating diverse and in - depth instructions. 2. The experimental results show that the instructions generated by IDEAS are of higher quality and help to improve the performance of downstream chat models. 3. Extensive experiments have verified that providing high - quality instruction conversations can further improve model performance. Through this method, researchers hope to make progress in automatically collecting high - quality multi - turn conversation data, so as to better fine - tune LLMs and make them more in line with human expectations.

Inductive-Deductive Strategy Reuse for Multi-Turn Instructional Dialogues

Dialogue Learning with Human-in-the-Loop.

Few-shot Dialogue Strategy Learning for Motivational Interviewing via Inductive Reasoning

Learning through Dialogue Interactions by Asking Questions

Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting

StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving

Multi-Task Learning based Online Dialogic Instruction Detection with Pre-trained Language Models

CESAR: Automatic Induction of Compositional Instructions for Multi-turn Dialogs

InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems

Raw Text is All you Need: Knowledge-intensive Multi-turn Instruction Tuning for Large Language Model

The Role of Deductive and Inductive Reasoning in Large Language Models

Injecting Salesperson's Dialogue Strategies in Large Language Models with Chain-of-Thought Reasoning

Automatic Dialogic Instruction Detection for K-12 Online One-on-one Classes

ItD: Large Language Models Can Teach Themselves Induction through Deduction

DC-Instruct: an Effective Framework for Generative Multi-intent Spoken Language Understanding

Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning

Hierarchical Inductive Transfer for Continual Dialogue Learning

Diverse and Fine-Grained Instruction-Following Ability Exploration with Synthetic Data

Dialog Flow Induction for Constrainable LLM-Based Chatbots

Instruction Induction: From Few Examples to Natural Language Task Descriptions

Strategize Before Teaching: A Conversational Tutoring System with Pedagogy Self-Distillation