Abstract:A great deal of information included in Chinese tex t is invaluable asset for further text mining, but the difference between Chinese and the western language s imposes restrictions on further utilization of Ch inese text. No distinction indication between words by us ing spaces is one of the major differences between Chinese, also some other Asian languages, such as J ap nese, Thai, etc., and Western languages. Chinese segmentation and features extraction is essential i Chinese natural language processing because it is a precondition for further Chinese text information r etrieval and knowledge discovery. Maximum matching and frequency statistics (MMFS) segmentation method based on length descending and string frequency statistics is an effective segmentation and extract ion method for Chinese words and phrases, but there are still some shorter words and phrases included in th e longer ones extracted by MMFS can’t be obtained. In order to solve this problem, this paper presents a novel Chinese hierarchy feature extraction method combined MMFS with iterative learning algorithm. Th is method can extract hierarchy feature according t o morphology with no need for lexicon support, no nee d for acquiring the probability between words in advance and no need for Chinese character index. Ex perimental results confirm the efficiency of this statistical method in extracting Chinese hierarchy feature. This method is also beneficial to feature extraction for other Asian languages similar to Chi nese.

Chinese Lexical Analysis Using Hierarchical Hidden Markov Model

Chinese Text Chunking Using Lexicalized HMMs

Chinese Lexical Analysis Based On Hybrid Mmsm Model

HMM and CRF Based Hybrid Model for Chinese Lexical Analysis

HHMM-based Chinese lexical analyzer ICTCLAS

A lexicalized second-order-HMM for ambiguity resolution in Chinese segmentation and POS tagging

Research on Chinese Lexical Analysis System by Fusing Multiple Knowledge Sources

Segmentation and Tagging Ambiguity for Chinese Using HMM

An approach based on iterative learning algorithm for chinese text hierarchy feature extraction without lexicon

Automatic Recognition of Chinese Unknown Word for Single-Character and Affix Models

Experimental Study of Hidden Markov Model Based Part-of-speech Tagging for Chinese Texts

Chinese Named Entity Identification Using Cascaded Hidden Markov Model

A Joint Segmenting And Labeling Approach For Chinese Lexical Analysis

Automatic Segmentation of Hierarchy Feature Without Lexicon for Chinese Text Based on Iterative Learning

A unified framework for text analysis in chinese TTS

Chinese lexical analysis based on hidden semi-crf

Parsing Chinese Based on Lexicalized Model

Hybrid Chinese Text Chunking

Improved Viterbi Algorithm-Based HMM2 for Chinese Words Segmentation

Automatic speech segmentation for Chinese speech database based on HMM

A Study of Chinese Lexical Analysis Based on Discriminative Models