Abstract:Keyword extraction plays the same role as the cornerstone in the field of natural language processing. Text classification, information retrieval, abstract generation and text clustering are all based on keyword extraction. This article takes the research of keyword extraction model as the subject. First, it analyzes the principle and limitations of the traditional keyword extraction model TF-IDF when extracting keywords. Secondly, it focuses on the problem of ignoring context and word polysemy in the keyword extraction model. To improve, introduce the concept of context vector, construct a chain-extensible structure for polysemous words, and propose a new keyword extraction method of TF-IDF that combines context and semantic classification. The specific research contents are as follows: TF-IDF extracts keywords based on multiple texts. The target keywords are words that appear frequently in the current text, but appear in other articles that are significantly lower in frequency than the current article. This method takes into account the characteristic content of the article and makes the article distinct. However, this method ignores the influence of the context of the article and the problem of word polysemy. Obviously, the expression of the thought of the article will not only be affected by the context of the words, but also by the semantics of the words. In order to solve the above problems, this paper proposes a TF-IDF keyword extraction method that combines context and semantic classification. First, the text constructs a linked list expandable structure to solve the problem of polysemy; then, the influence of context is introduced. Before inserting a new word into the extended structure, the semantic similarity is calculated through the context vector. Finally, iterate over the entire expandable structure, sum the word frequency with semantic similarity under each semantic, and then sort the results to obtain TopN words as the text keywords of the corresponding article. In this paper, the Sogou news data set is used as the input data of the model, and the accuracy of the experimental results is significantly better than that of TF-IDF.

Bert-Based Text Keyword Extraction

Improving short text classification using public search engines

Exploring Simultaneous Keyword and Key Sentence Extraction

Exploring simultaneous keyword and key sentence extraction: improve graph-based ranking using wikipedia.

Multifeature Fusion Keyword Extraction Algorithm Based on TextRank

Automatic Abstraction of Long Chinese Patent Texts Based on P-Bertsum Model

Impact analysis of keyword extraction using contextual word embedding

Keyword Extraction Based on Tf/idf for Chinese News Document

TF-IDF Keyword Extraction Method Combining Context and Semantic Classification

A Patent Keyword Extraction Method Based on Corpus Classification

Enhancing keyphrase extraction from academic articles with their reference information

News keyword extraction algorithm based on semantic clustering and word graph model

Keyword Extraction Approach Based on Probabilistic-Entropy, Graph, and Neural Network Methods

Extraction of temporal information from social media messages using the BERT model

Chinese Keyword Extraction Algorithm Based on Neighbour Words

Automatic Keyword Extraction Based on Phrase Network

Patent Keyword Extraction Algorithm Based on Distributed Representation for Patent Classification

Using pretraining and text mining methods to automatically extract the chemical scientific data

Keyword extraction using support vector machine

FRAKE: Fusional Real-time Automatic Keyword Extraction

Automatic Keywords Extraction Based on Co-Occurrence and Semantic Relationships Between Words