Abstract:Recent progress on large language models (LLMs) has enabled dialogue agents to generate highly naturalistic and plausible text. However, current LLM language generation focuses on responding accurately to questions and requests with a single effective response. In reality, many real dialogues are interactive, meaning an agent's utterances will influence their conversational partner, elicit information, or change their opinion. Accounting for how an agent can effectively steer a conversation is a crucial ability in many dialogue tasks, from healthcare to preference elicitation. Existing methods for fine-tuning dialogue agents to accomplish such tasks would rely on curating some amount of expert data. However, doing so often requires understanding the underlying cognitive processes of the conversational partner, which is a skill neither humans nor LLMs trained on human data can reliably do. Our key insight is that while LLMs may not be adept at identifying effective strategies for steering conversations a priori, or in the middle of an ongoing conversation, they can do so post-hoc, or in hindsight, after seeing how their conversational partner responds. We use this fact to rewrite and augment existing suboptimal data, and train via offline reinforcement learning (RL) an agent that outperforms both prompting and learning from unaltered human demonstrations. We apply our approach to two domains that require understanding human mental state, intelligent interaction, and persuasion: mental health support, and soliciting charitable donations. Our results in a user study with real humans show that our approach greatly outperforms existing state-of-the-art dialogue agents.

Enhancing Dialogue Generation Through Iterative Prompt with Difference Analogy Learning

Crafting a Good Prompt or Providing Exemplary Dialogues? A Study of In-Context Learning for Persona-based Dialogue Generation

Deep Reinforcement Learning for Dialogue Generation

Dialogue Learning with Human-in-the-Loop.

Data Augmentation of Multi-turn Psychological Dialogue via Knowledge-driven Progressive Thought Prompting

A New Dialogue Response Generation Agent for Large Language Models by Asking Questions to Detect User's Intentions

Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations

Self-Explanation Prompting Improves Dialogue Understanding in Large Language Models

Learning to Generate Prompts for Dialogue Generation through Reinforcement Learning

Exploring In-Context Learning for Knowledge Grounded Dialog Generation

Prompt Learning for Domain Adaptation in Task-Oriented Dialogue

Regularizing Dialogue Generation by Imitating Implicit Scenarios

Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue

Active Prompting with Chain-of-Thought for Large Language Models

Multi-level Adaptive Contrastive Learning for Knowledge Internalization in Dialogue Generation

Prompt Optimization via Adversarial In-Context Learning

Analogy Generation by Prompting Large Language Models: A Case Study of InstructGPT

Learning Towards Selective Data Augmentation for Dialogue Generation.

Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration

Contrastive Learning for Inference in Dialogue

Incremental Learning from Scratch for Task-Oriented Dialogue Systems