Abstract:Text clustering techniques were usually used to structure the text documents into topic related groups which can facilitate users to get a comprehensive understanding on corpus or results from information retrieval system. Most of existing text clustering algorithm which derived from traditional formatted data clustering heavily rely on term analysis methods and adopted Vector Space Model (VSM) as their document representation. But because of the essential characteristic underlying text such as high dimensionality features vector space, the problem of sparseness has a strong impact on the clustering algorithm. So feature reduction is an important preprocess step for improving the efficiency and accuracy of clustering algorithm by removing redundant and irrelevant terms from corpus. Even the clustering is considered as an unsupervised learning method, but in text, there is still some priori knowledge we can use from NLP analysis based approach. In this paper, we propose a semantic analysis based feature reduction method which used in Chinese text clustering. Our method bases on a dedicated Part-of-Speech tags selection and synonyms consolidation and can reduce the feature space of documents more effectively compared with traditional feature reduction method tfidf and stopwords removal, meanwhile it preserves or sometimes even improves the accuracy of clustering algorithm. In our experiment, we tested our feature reduction method using bisecting k-means algorithm which was proved be efficient in text clustering. The results show that our method can reduce the feature space significantly, and meanwhile have a better clustering accuracy in terms of the purity.

Semantic scoring based on small-world phenomenon for feature selection in text mining

Semantic Annotation with RescoredESA

Improving Short Text Classification Through Better Feature Space Selection

A comprehensive unsupervised feature selection method of two-stage strategy

A Discriminative and Semantic Feature Selection Method for Text Categorization

A Method of Feature Selection Based on Word2Vec in Text Categorization

Aggressive Dimensionality Reduction With Reinforcement Local Feature Selection For Text Categorization

Semantic Text Mining with Linked Data

Combining Lexical and Semantic Features for Short Text Classification.

A Semantic Term Weighting Scheme for Text Categorization.

Feature Dimension Reduction Short Text Clustering Combined with Semantic and Statistics

Using Category-Based Semantic Field For Text Categorization

Semantic Feature Reduction in Chinese Document Clustering

Searching Semantic Web Documents Based on RDF Sentences

Short Text Classification Based on Strong Feature Thesaurus

Exploring Semantic Concept Using Local Invariant Features

An Effective Feature Selection Method For Text Categorization

Exploiting Word Cluster Information for Unsupervised Feature Selection

Efficient Method for Feature Selection in Text Classification

An Efficient Feature Selection Method Using Named Entity Recognition for Chinese Text Categorization

An Evaluation on Feature Selection for Text Clustering