BERT‐TriF: An inductive short text classification model for power equipment defect records

Zhenhao Ye,Bingyan Guo,Donglian Qi
DOI: https://doi.org/10.1002/eng2.12671
2023-06-11
Engineering Reports
Abstract:This paper proposes an inductive model named bidirectional encoder representations from transformers with family feature fusion (BERT‐TriF) for short text classification in the power inspection field. The model has better classification accuracy, robustness, and general applicability than benchmark deep learning models in specific and generic datasets. The descriptions of power equipment defect records are often characterized by colloquial short texts. Standardized classification of a large number of colloquial defect descriptions has laid a solid foundation for building a power equipment knowledge graph and improving the level of intelligence in the field of power inspection. Using deep learning and natural language processing technology, this paper proposes a text classification model for power equipment defect records named Bidirectional Encoder Representations from Transformers with Family Feature Fusion (BERT‐TriF). The model firstly leverages BERT to semantically represent the input text. To extract family history as well as text implicit information, we creatively propose a family feature fusion algorithm for training. An improved multi‐head attention mechanism is developed subsequently to enhance text semantic category features and strengthen the learning ability of the model. By comparing BERT‐TriF and baseline models such as TextCNN, TextRNN, and fastText on the specified and generic text dataset, the experimental results demonstrate that it has better performance, robustness, and universality for short text classification.
What problem does this paper attempt to address?