Abstract:Information-seeking conversation systems are increasingly popular in real-world applications, especially for e-commerce companies. To retrieve appropriate responses for users, it is necessary to compute the matching degrees between candidate responses and users' queries with historical dialogue utterances. As the contexts are usually much longer than responses, it is thus necessary to expand the responses (usually short) with richer information. Recent studies on pseudo-relevance feedback (PRF) have demonstrated its effectiveness in query expansion for search engines, hence we consider expanding response using PRF information. However, existing PRF approaches are either based on heuristic rules or require heavy manual labeling, which are not suitable for solving our task. To alleviate this problem, we treat the PRF selection for response expansion as a learning task and propose a reinforced learning method that can be trained in an end-to-end manner without any human annotations. More specifically, we propose a reinforced selector to extract useful PRF terms to enhance response candidates and a BERT-based response ranker to rank the PRF-enhanced responses. The performance of the ranker serves as a reward to guide the selector to extract useful PRF terms, which boosts the overall task performance. Extensive experiments on both standard benchmarks and commercial datasets prove the superiority of our reinforced PRF term selector compared with other potential soft or hard selection methods. Both case studies and quantitative analysis show that our model is capable of selecting meaningful PRF terms to expand response candidates and also achieving the best results compared with all baselines on a variety of evaluation metrics. We have also deployed our method on online production in an e-commerce company, which shows a significant improvement over the existing online ranking system.

A Framework Integrating Different Relevance Feedback Scenarios and Approaches for Spoken Term Detection.

Improved Spoken Term Detection by Discriminative Training of Acoustic Models Based on User Relevance Feedback.

Improved Spoken Term Detection by Feature Space Pseudo-Relevance Feedback.

Integrating Recognition and Retrieval with User Feedback: A New Framework for Spoken Term Detection.

Learning A Semantic Space from User'S Relevance Feedback for Image Retrieval

Improved spoken term detection using support vector machines with acoustic and context features from pseudo-relevance feedback

Improved Spoken Term Detection with Graph-Based Re-Ranking in Feature Space

Improved spoken term detection using support vector machines based on lattice context consistency

Fully utilize feedbacks: language model based relevance feedback in information retrieval

Improved open-vocabulary spoken content retrieval with word and subword lattices using acoustic feature similarity

Enhanced Spoken Term Detection Using Support Vector Machines and Weighted Pseudo Examples

Open-Vocabulary Retrieval of Spoken Content with Shorter/Longer Queries Considering Word/Subword-based Acoustic Feature Similarity.

A Linguistic Study on Relevance Modeling in Information Retrieval

SPRF: A semantic pseudo-relevance feedback enhancement for information retrieval via ConceptNet

Relevance Feedback in Content-Based Image Retrieval: Bayesian Framework, Feature Subspaces, and Progressive Learning

Pseudo-Relevance Feedback Based On Mrmr Criteria

Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations

Relevance Feedback by Exploring the Different Feedback Sources and Collection Structure.

Speech Retrieval Engine Based on Relevance Feedback

Improved Semantic Retrieval of Spoken Content by Document/Query Expansion with Random Walk Over Acoustic Similarity Graphs

A Powerful Relevance Feedback Mechanism for Content-Based 3D Model Retrieval