Abstract:Keyword extraction plays the same role as the cornerstone in the field of natural language processing. Text classification, information retrieval, abstract generation and text clustering are all based on keyword extraction. This article takes the research of keyword extraction model as the subject. First, it analyzes the principle and limitations of the traditional keyword extraction model TF-IDF when extracting keywords. Secondly, it focuses on the problem of ignoring context and word polysemy in the keyword extraction model. To improve, introduce the concept of context vector, construct a chain-extensible structure for polysemous words, and propose a new keyword extraction method of TF-IDF that combines context and semantic classification. The specific research contents are as follows: TF-IDF extracts keywords based on multiple texts. The target keywords are words that appear frequently in the current text, but appear in other articles that are significantly lower in frequency than the current article. This method takes into account the characteristic content of the article and makes the article distinct. However, this method ignores the influence of the context of the article and the problem of word polysemy. Obviously, the expression of the thought of the article will not only be affected by the context of the words, but also by the semantics of the words. In order to solve the above problems, this paper proposes a TF-IDF keyword extraction method that combines context and semantic classification. First, the text constructs a linked list expandable structure to solve the problem of polysemy; then, the influence of context is introduced. Before inserting a new word into the extended structure, the semantic similarity is calculated through the context vector. Finally, iterate over the entire expandable structure, sum the word frequency with semantic similarity under each semantic, and then sort the results to obtain TopN words as the text keywords of the corresponding article. In this paper, the Sogou news data set is used as the input data of the model, and the accuracy of the experimental results is significantly better than that of TF-IDF.

An Improved TF-IDF Approach for Text Classification

Improving Short Text Classification Through Better Feature Space Selection

Improving short text classification using public search engines

Chinese text classification method using FastText and term frequency-inverse document frequency optimization

A Technique For Improving The Performance Of Naive Bayes Text Classification

An Improved Method of Feature Selection Based on Concept Attributes in Text Classification

Feature Selection Method on Imbalanced Text

An improved supervised term weighting scheme for text representation and classification

Using modified term frequency to improve term weighting for text classification

An Effective Concept Extraction Method for Improving Text Classification Performance

Improving Text Categorization Using the Importance of Words in Different Categories

Several alternative term weighting methods for text representation and classification

Modified DFS-based term weighting scheme for text classification

Dimensionality Reduction With Category Information Fusion And Non-Negative Matrix Factorization For Text Categorization

TF-IDF Keyword Extraction Method Combining Context and Semantic Classification

Adapting Feature Selection Algorithms for the Classification of Chinese Texts

Exploiting Category Information and Document Information to Improve Term Weighting for Text Categorization

Supervised Contrastive Learning with Term Weighting for Improving Chinese Text Classification

Short Text Classification Based on Strong Feature Thesaurus

Research on dynamic self-adaptive term weighting for multi-class text classification algorithm