How to Represent Context Better? an Empirical Study on Context Modeling for Multi-turn Response Selection.

Jiazhan Feng,Chongyang Tao,Chang Liu,Rui Yan,Dongyan Zhao
DOI: https://doi.org/10.18653/v1/2022.findings-emnlp.539
2022-01-01
Abstract:Building retrieval-based dialogue models that can predict appropriate responses based on the understanding of multi-turn context messages is a challenging problem.Early models usually concatenate all utterances or independently encode each dialogue turn, which may lead to an inadequate understanding of dialogue status.Although a few researchers have noticed the importance of context modeling in multiturn response prediction, there is no systematic comparison to analyze how to model context effectively and no framework to unify those methods.In this paper, instead of configuring new architectures, we investigate how to improve existing models with a better context modeling method.Specifically, we heuristically summarize three categories of turn-aware context modeling strategies which model the context messages from the perspective of sequential relationship, local relationship, and query-aware manner respectively.A Turn-Aware Context Modeling (TACM) layer is explored to flexibly adapt and unify these context modeling strategies to several advanced response selection models.Evaluation results on three public data sets indicate that employing each individual context modeling strategy or multiple strategies can consistently improve the performance of existing models.
What problem does this paper attempt to address?