Abstract:Generating follow-up questions on the fly could significantly improve conversational survey quality and user experiences by enabling a more dynamic and personalized survey structure. In this paper, we proposed a novel task for knowledge-driven follow-up question generation in conversational surveys. We constructed a new human-annotated dataset of human-written follow-up questions with dialogue history and labeled knowledge in the context of conversational surveys. Along with the dataset, we designed and validated a set of reference-free Gricean-inspired evaluation metrics to systematically evaluate the quality of generated follow-up questions. We then propose a two-staged knowledge-driven model for the task, which generates informative and coherent follow-up questions by using knowledge to steer the generation process. The experiments demonstrate that compared to GPT-based baseline models, our two-staged model generates more informative, coherent, and clear follow-up questions.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to automatically generate follow - up questions in conversational surveys. Specifically, the authors propose a knowledge - driven method to generate follow - up questions, aiming to improve the quality of conversational surveys and user experience. Through this method, a more dynamic and personalized survey structure can be achieved. The paper mainly addresses the following three challenges: 1. **Lack of data sets**: In open - domain conversational surveys, there are no ready - made data sets for generating follow - up questions. The existing relevant data sets are small in scale or very specific, such as job interviews or postgraduate entrance interviews, and these data sets do not consider background knowledge beyond the conversation history, which limits the model's ability to deeply understand the survey objectives and context. 2. **Limitations of existing methods**: Current methods are either based on template filling (Su et al., 2019; Inoue et al., 2020) or on sequence - to - sequence (seq2seq) models (Su et al., 2018; Wang et al., 2018; SB et al., 2020). The template - filling method limits the diversity of question types and it is difficult to ask personalized questions according to the participants' responses, especially in dynamic and open - ended conversational surveys. And the standard seq2seq method is unable to generate questions that are in line with the overall survey objectives and relevant to the context. 3. **Lack of effective evaluation metrics**: Currently, there are no mature metrics that can effectively evaluate the generated follow - up questions. Since the same conversation history can inspire multiple valid follow - up questions and the same question can also have different expressions, traditional text generation metrics that rely on a single ground truth usually underestimate the question quality, and manual evaluation is difficult to scale and compare. To solve these problems, the authors propose a knowledge - driven follow - up question generation task, construct a new annotated data set, and design a set of reference - free evaluation metrics (Gricean Scores) based on Gricean Maxims to systematically evaluate the quality of the generated follow - up questions. Through experimental verification, their proposed two - stage knowledge - driven model can generate more informative, coherent and clear follow - up questions compared to the GPT - based baseline model.

What should I Ask: A Knowledge-driven Approach for Follow-up Questions Generation in Conversational Surveys

FOLLOWUPQG: Towards Information-Seeking Follow-up Question Generation

Survey of Research on Knowledge-Driven Dialogue Generation Models

Designing the Conversational Agent: Asking Follow-up Questions for Information Elicitation

Question Generation in Knowledge-Driven Dialog: Explainability and Evaluation

Ask to Learn: A Study on Curiosity-driven Question Generation

Learning to Retrieve Engaging Follow-Up Queries

A Knowledge Enhanced Generative Conversational Service Agent

Select, Extend, and Generate: Generative Knowledge Selection for Open-Domain Dialogue Response Generation.

Improving Knowledge-aware Dialogue Generation Via Knowledge Base Question Answering

Learning to Ask Appropriate Questions in Conversational Recommendation

Dynamically Retrieving Knowledge via Query Generation for Informative Dialogue Generation

Asking The Crowd: Question Analysis, Evaluation And Generation For Open Discussion On Online Forums

Generating Relevant and Informative Questions for Open-domain Conversations

Prediction, Selection, and Generation: Exploration of Knowledge-Driven Conversation System

Improving Dialogue Generation via Proactively Querying Grounded Knowledge

Using Implicit Feedback to Improve Question Generation

Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations

Knowledge-enriched, Type-constrained and Grammar-guided Question Generation over Knowledge Bases.

Knowledge Augmented Dialogue Generation with Divergent Facts Selection

Improving Knowledge-Aware Dialogue Response Generation by Using Human-Written Prototype Dialogues.