Abstract:From the perspective of corpus measure,which includes three indicators:the clustering density,the complexity and definition of category,this paper studied the influence of three representative Chinese word segmentation methods,including IC-TCLAS,Ik Analyzer and 2-gram,on the performance of text classification under the implicit probabilistic topic model LDA.Mo-reover,the applicability of different Chinese word segmentation methods in different types of texts such as Web and academic documents and its cause were analyzed qualitatively and quantitatively.Experiments show that three indexes can effectively in-dicate the influence of word segmentation method on the classification of texts:Ik Analyzer and ICTCLAS segmentation method are more influenced respectively by the complexity of the category and the clustering density of the category,for 2-gram,the in-fluences of three indexes are similar,so it has good adaptability for different corpus.For corpus of academic literature,2-gram has better performance,F1 values are above 80%.And the corpus of Web pages is more adaptive to different word segmentation methods.This paper provides a reference for the selection of appropriate Chinese word segmentation method in classification system based on LDA model for different types of texts such as Web pages and academic literature by means of corpus measure instead of by experiments only.

The Research of Bibliographic Information Classification Method Based on the Composite Weighted LDA Model

CLDA: Feature Selection for Text Categorization Based on Constrained LDA

Automatic Coefficient Selection in Weighted Maximum Margin Criterion

Research on Deep Web Classification Based on Domain Feature Text

A Novel Feature Selection Approach and Feature Weight Adjustment Technique in Text Classification

A LDA Topic Model Based Collection Selection Method for Distributed Information Retrieval

Research on Automatic Classification for Chinese Bibliography Based on Machine Learning

Medical Aided Diagnosis Using Electronic Medical Records Based On Lda And Word Vector Model

Ontology-Based Feature Weighting For Biomedical Literature Classification

A combined weight method in automatic classification of chinese text

New Feature Selection and Weighting Methods Based on Category Information.

Study on influences of different Chinese word segmentation methods to text automatic classification based on LDA model

A Novel Term Weighting Scheme for Automated Text Categorization

Medical Record Semantic Analysis Based on Weighted LDA

A Method of Automatic Annotation for Medical Record Text Based on Latent Dirichlet Allocation

A New Feature Weighting Method Based on Probability Distribution in Imbalanced Text Classification

Short text classification based on LDA topic model

LSA & LDA topic modeling classification: comparison study on e-books

A Weighted Voting Based Automatic Term Recognition Method

A Comparative Study on Feature Weight in Text Categorization

Web Text Classification based on LDA Model