Unknown Intent Detection Method Based on Model Post-Processing

Hua Xu,Hanlei Zhang,Ting-En Lin

2023-01-01

Abstract:With the maturity and popularity of dialogue systems, detecting user’s unknown intent in dialogue systems has become an important task. It is also one of the most challenging tasks, as it is difficult to obtain examples, prior knowledge or the exact numbers of unknown intents. In this chapter, we introduce SofterMax and deep novelty detection (SMDN), a simple yet effective post-processing method for detecting unknown intent in dialogue systems based on pre-trained deep neural network classifiers. Our method can be flexibly applied on top of any classifiers trained in deep neural networks without changing the model architecture. By calibrating the confidence of the softmax outputs to compute the calibrated confidence score (i.e., SofterMax) and use it to calculate the decision boundary for unknown intent detection. Furthermore, the feature representations learned by the deep neural networks are fed into a traditional novelty detection algorithm to detect unknown intents from different perspectives. Finally, the methods mentioned above are combined to facilitate joint prediction. The proposed method classifies examples that differ from known intents as unknown and does not require any examples or prior knowledge of it. Extensive experiments have been conducted on three benchmark dialogue datasets. The results show that our method can yield significant improvements compared with the state-of-the-art baselines.