Abstract:Conversational search has seen increased recent attention in both the IR andNLP communities. It seeks to clarify and solve users' search needs throughmulti-turn natural language interactions. However, most existing systems aretrained and demonstrated with recorded or artificial conversation logs.Eventually, conversational search systems should be trained, evaluated, anddeployed in an open-ended setting with unseen conversation trajectories. A keychallenge is that training and evaluating such systems both require ahuman-in-the-loop, which is expensive and does not scale. One strategy is tosimulate users, thereby reducing the scaling costs. However, current usersimulators are either limited to only responding to yes-no questions from theconversational search system or unable to produce high-quality responses ingeneral. In this paper, we show that existing user simulation systems could besignificantly improved by a smaller finetuned natural language generationmodel. However, rather than merely reporting it as the new state-of-the-art, weconsider it a strong baseline and present an in-depth investigation ofsimulating user response for conversational search. Our goal is to supplementexisting work with an insightful hand-analysis of unsolved challenges by thebaseline and propose our solutions. The challenges we identified include (1) ablind spot that is difficult to learn, and (2) a specific type of misevaluationin the standard setup. We propose a new generation system to effectively coverthe training blind spot and suggest a new evaluation setup to avoidmisevaluation. Our proposed system leads to significant improvements overexisting systems and large language models such as GPT-4. Additionally, ouranalysis provides insights into the nature of user simulation to facilitatefuture work.

Learning to Start for Sequence to Sequence Based Response Generation

Learning to Start for Sequence to Sequence Architecture

Promoting Diversity for End-to-End Conversation Response Generation.

Tailored Sequence to Sequence Models to Different Conversation Scenarios

Generating Diverse Conversation Responses by Creating and Ranking Multiple Candidates

Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation.

Rich Short Text Conversation Using Semantic-Key-Controlled Sequence Generation

Neural Response Generation via GAN with an Approximate Embedding Layer.

Learning to Control the Specificity in Neural Response Generation

Towards Robust Online Dialogue Response Generation

Retrieval-guided Dialogue Response Generation Via a Matching-to-Generation Framework.

Towards Implicit Content-Introducing for Generative Short-Text Conversation Systems

Neural Response Generation with Dynamic Vocabularies

Learning To Ask More: Semi-Autoregressive Sequential Question Generation Under Dual-Graph Interaction

Local Explanation of Dialogue Response Generation

Learning to Express in Knowledge-Grounded Conversation

An In-depth Investigation of User Response Simulation for Conversational Search.

Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation

Sequential Topic Selection Model with Latent Variable for Topic-Grounded Dialogue

Small Changes Make Big Differences: Improving Multi-turn Response Selection in Dialogue Systems Via Fine-Grained Contrastive Learning

GLEU-Guided Multi-resolution Network for Short Text Conversation