Abstract:In the era of big data, text as an information reserve database is very important, in all walks of life. From humanities research to government decision-making, from precision medicine to quantitative finance, from customer management to marketing, massive text, as one of the most important information carriers, plays an important role everywhere. The text data generated in these practical problems of humanities research, financial industry, marketing, and other fields often has obvious domain characteristics, often containing the professional vocabulary and unique language patterns in these fields and often accompanied by a variety of "noise." Dealing with such texts is a great challenge for the current technical conditions, especially for Chinese texts. A clustering algorithm provides a better solution for text big data information processing. Clustering algorithm is the main body of cluster analysis, K-means algorithm with its implementation principle is simple, low time complexity is widely used in the field of cluster analysis, but its K value needs to be preset, initial clustering center random selection into local optimal solution, other clustering algorithm, such as mean drift clustering, K-means clustering in mining text big data. In view of the problems of the above algorithm, this paper first extracts and analyzes the text big data and then does experiments with the clustering algorithm. Experimental conclusion: by analyzing large-scale text data limited to large-scale and simple data set, the traditional K-means algorithm has low efficiency and reduced accuracy, and the K-means algorithm is susceptible to the influence of initial center and abnormal data. According to the above problems, the K-means cluster analysis algorithm for data sets with large data volumes is analyzed and improved to improve its execution efficiency and accuracy on data sets with large data volume set. Mean shift clustering can be regarded as making many random centers move towards the direction of maximum density gradually, that is, moving their mean centroid continuously according to the probability density of data and finally obtaining multiple maximum density centers. It can also be said that mean shift clustering is a kernel density estimation algorithm.

A Lda-Based Algorithm For Length-Aware Text Clustering

CLDA: Feature Selection for Text Categorization Based on Constrained LDA

Transferring Topical Knowledge from Auxiliary Long Texts for Short Text Clustering

LDA Meets Word2Vec

Text Clustering as Classification with LLMs

LDA Meets Word2Vec: A Novel Model for Academic Abstract Clustering.

A Practical Algorithm for Solving the Sparseness Problem of Short Text Clustering

Short Text Classification Using Latent Dirichlet Allocation

VRCA: A Clustering Algorithm for Massive Amount of Texts

X-DMM: Fast and Scalable Model Based Text Clustering

Word2Cluster: A New Multi-Label Text Clustering Algorithm with an Adaptive Clusters Number

Research and Application of Clustering Algorithm for Text Big Data

Sentiment word co-occurrence and knowledge pair feature extraction based LDA short text clustering algorithm

Optimizing Text Clustering Efficiency through Flexible Latent Dirichlet Allocation Method: Exploring the Impact of Data Features and Threshold Modification

A Text Clustering Algorithm to Detect Basic Level Categories in Texts

A Document Ensemble Clustering Approach Via Dimensionality Reduction

Research on Neural Network Clustering Algorithm for Short Text

Clust-LDA: Joint Model for Text Mining and Author Group Inference

A Hierarchical Text Clustering Algorithm with Cognitive Situation Dimensions.

Feature Dimension Reduction Short Text Clustering Combined with Semantic and Statistics

Towards Semantically Sensitive Text Clustering: a Feature Space Modeling Technology Based on Dimension Extension.