UTMGAT: a unified transformer with memory encoder and graph attention networks for multidomain dialogue state tracking
Prasad, Bhuyan Kaibalya,Song, Wei,Kefalas, Pavlos
DOI: https://doi.org/10.1007/s10489-024-05571-2
IF: 5.3
2024-06-25
Applied Intelligence
Abstract:Spoken dialogue systems (SDS) heavily rely on dialogue state tracking (DST) for success. However, providing sufficient computational power for training proves challenging, given that DST involves tracking states from both user and system utterances. While machine learning approaches have improved DST, they have notable limitations. These approaches often overlook unseen slot values during training and use two separate modules to extract, generate, or match slot values, leading to high time and resource consumption. Moreover, learning and deducing relevant values for related slots remain understudied challenges. To address these gaps, this paper introduces UTMGAT-a Unified Transformer with Memory Encoder and Graph Attention Networks (GAT) for Multidomain DST. UTMGAT employs a BERT tokenizer to construct user utterances and a candidate sets vocabulary, reducing the need for constant retraining when dealing with unseen values. It utilizes a single transformer to gather dialogue context for slots and generate slot values, enhancing prediction accuracy while reducing memory and computation time. UTMGAT incorporates an embedding layer aggregator to filter out unnecessary values, identify required nodes for GAT, and establish relationships among relevant values associated with related slots. This approach simplifies graph representation and diminishes required computation power. The input to the GAT maintains equal size with batch sizes, generated through padding. Finally, we have experimentally evaluated our model against several models including LLM approaches over four popular datasets with our approach outperforming all competing models except two approaches on one dataset.
computer science, artificial intelligence