An Optimized Chinese Filtering Model Using Value Scale Extended Text Vector
Siyu Lu,Ligao Cai,Zhixin Liu,Shan Liu,Bo Yang,Lirong Yin,Mingzhe Liu,Wenfeng Zheng
DOI: https://doi.org/10.32604/csse.2023.034853
IF: 4.397
2023-01-01
Computer Systems Science and Engineering
Abstract:With the development of Internet technology, the explosive growth of Internet information presentation has led to difficulty in filtering effective information. Finding a model with high accuracy for text classification has become a critical problem to be solved by text filtering, especially for Chinese texts. This paper selected the manually calibrated Douban movie website comment data for research. First, a text filtering model based on the BP neural network has been built; Second, based on the Term Frequency-Inverse Document Frequency (TF-IDF) vector space model and the doc2vec method, the text word frequency vector and the text semantic vector were obtained respectively, and the text word frequency vector was linearly reduced by the Principal Component Analysis (PCA) method. Third, the text word frequency vector after dimensionality reduction and the text semantic vector were combined, add the text value degree, and the text synthesis vector was constructed. Experiments show that the model combined with text word frequency vector degree after dimensionality reduction, text semantic vector, and text value has reached the highest accuracy of 84.67%.
computer science, theory & methods, hardware & architecture