Clarification question generation diversity and specificity enhancement based on question keyword prediction

Mingtao Zhou,Juxiang Zhou,Jianhou Gan,Wei Gao,Jian Xu
DOI: https://doi.org/10.1007/s10489-024-05316-1
IF: 5.3
2024-02-17
Applied Intelligence
Abstract:Clarification question generation is used mainly to solve the potential missing product information problem in e-commerce. However, the problem of not being able to better understand contextual semantic information has arisen when studying the diversity and specificity of clarification questions. This further leads to imprecise prediction of question keywords. With the aim of solving the current constraints, we propose the Transformer Encoder of Keyword Predictor Keyword-Conditioned Network (TEKPCNet). First, the transformer encoder has been proven to perform well in predicting external question keywords by encoding context. Then, the Text Convolutional Neural Network (TextCNN) and multilayer perceptron are combined to enhance the input of the encoder-decoder model. The diversity and specificity of the clarification question generation process should be further enhanced. Finally, the decoding process is completed by combining the attention mechanism. The generated clarification questions were evaluated in terms of two datasets using both automatic assessment and human evaluation. The experimental results show that the proposed model outperforms the other models.
computer science, artificial intelligence
What problem does this paper attempt to address?