Self-supervised attention flow for dialogue state tracking

Boyuan Pan,Yazheng Yang,Bo Li,Deng Cai

DOI: https://doi.org/10.1016/j.neucom.2021.01.118

IF: 6

2021-06-01

Neurocomputing

Abstract:<p>The performance of existing approaches for dialogue state tracking (DST) is often limited by the deficiency of labeled datasets, and inefficient utilization of data is also a practical yet tough problem of the DST task. In this paper, we aim to tackle these challenges in a self-supervised manner by introducing an auxiliary pre-training task that learns to pick up the correct dialogue response from a group of candidates. Moreover, we propose an attention flow mechanism that is augmented with a soft-threshold function in a dynamic way to better understand the user intent and filter out the redundant information. Extensive experiments on the multi-domain dialogue state tracking dataset MultiWOZ 2.1 demonstrate the effectiveness of our proposed method, and we also show that it is able to adapt to zero/few-shot cases under the proposed self-supervised framework.</p>

computer science, artificial intelligence

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on two challenges in the Dialogue State Tracking (DST) task:

1. **Insufficient Annotated Data**: The performance of existing DST methods is usually limited by the lack of annotated data sets. Due to the complexity and high cost of the annotation process, it is very difficult to obtain a large amount of high - quality annotated data, which leads to the inefficiency of existing methods in data utilization.
2. **Data Sparsity Problem**: In practical applications, especially in multi - domain dialogue systems, the slot values of the dialogue state may be very sparse. That is, in a dialogue turn, only a few slots are mentioned, which makes it difficult for the model to learn effective representations from limited data.

To address these challenges, the author proposes a self - supervised learning framework - the Self - Supervised Attention Flow (SAF) network. Specifically, this framework improves the model's understanding ability of the dialogue state by introducing an auxiliary pre - training task - Dialogue Response Selection (DRS). The goal of the DRS task is to select the correct next system response from a set of candidate responses. This process does not require additional manual annotation data, thereby improving the data utilization efficiency. In addition, the SAF framework also introduces a dynamically updated attention flow mechanism. This mechanism filters redundant information through a soft - threshold function, enhancing the model's understanding ability of user intentions.

In this way, the SAF framework not only achieves the current best results on the standard multi - domain dialogue state tracking data set MultiWOZ 2.1, but also shows strong generalization ability in zero - shot / few - shot learning scenarios.