Question classification task based on deep learning models with self-attention mechanism
DOI: https://doi.org/10.1007/s11042-024-19239-z
IF: 2.577
2024-04-27
Multimedia Tools and Applications
Abstract:Question classification (QC) is a process that involves classifying questions based on their type to enable systems to provide accurate responses by matching the question type with relevant information. To understand and respond to natural language questions posed by humans, machine learning models or systems must comprehend the type of information requested, which can often be inferred from the structure and wording of the question. The high dimensionality and sparse nature of text data lead to challenges for text classification. These tasks can be improved using deep learning (DL) approaches to process complex patterns and features within input data. By training on large amounts of labeled data, deep learning algorithms can automatically extract relevant features and representations from text, resulting in more accurate and robust classification. This study utilizes a dataset comprising 5452 instances of questions and six output labels and uses two different word embedding techniques, like GloVe and Word2Vec, tested on the dataset using three deep learning models, LSTM, BiLSTM, and GRU, followed by a convolution layer. Additionally, a self-attention layer is included, which helps the model to focus on more relevant information when making predictions. Finally, an analytical discussion of the proposed models and their performance results provide insight into how GloVe and Word2Vec perform on the above-mentioned models. The GloVe embedding outperforms by achieving 97.68% accuracy and a moderate loss of 16.98 with the GRU model.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering