Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations

Peng Qi,Yuhao Zhang,Christopher D. Manning

DOI: https://doi.org/10.48550/arXiv.2004.14530

2020-10-21

Abstract:We investigate the problem of generating informative questions in information-asymmetric conversations. Unlike previous work on question generation which largely assumes knowledge of what the answer might be, we are interested in the scenario where the questioner is not given the context from which answers are drawn, but must reason pragmatically about how to acquire new information, given the shared conversation history. We identify two core challenges: (1) formally defining the informativeness of potential questions, and (2) exploring the prohibitively large space of potential questions to find the good candidates. To generate pragmatic questions, we use reinforcement learning to optimize an informativeness metric we propose, combined with a reward function designed to promote more specific questions. We demonstrate that the resulting pragmatic questioner substantially improves the informativeness and specificity of questions generated over a baseline model, as evaluated by our metrics as well as humans.

Computation and Language

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to generate informative and specific questions in open - domain information - seeking conversations. Different from previous work, this paper focuses on how questioners generate questions without the background of answers, that is, questioners cannot access the context containing answers, but can only reason about how to obtain new information based on the history of the conversation. The paper points out that generating questions in this situation faces two core challenges:

1. **Formalizing the informativeness of potential questions**: How to quantify how much new information a potential question can reveal.
2. **Exploring the large space of potential questions**: How to find good candidate questions among a large number of potential questions.

To address these challenges, the authors propose using reinforcement learning to optimize their proposed informativeness measure and combine the designed reward function to promote the generation of more specific questions. In this way, the authors hope that the question - asking system they build can obtain information more effectively in conversations and improve communication efficiency. Experimental results show that, compared with the baseline model, the proposed system significantly improves the informativeness and specificity of generated questions in both custom - made metrics and human evaluations.