Clustering of Chinese Sentences Using the SMM Model

Tiansang Du,Xinying Xu,Liang Chen,Baobao Chang

DOI: https://doi.org/10.1109/nlpke.2007.4368076

2007-01-01

Abstract:The purpose of this article is to research the clustering method based on statistical model, then deal with the Chinese sentence clustering problem on bilingual lexicographical platform. In the view of cooccurrence data, we develop the Sentence Cluster Model as a multidimensional SMM, and get the solution of parameter estimation by EM algorithm. Based on this model, we represent three methods for sentence clustering, and use Rand index to evaluate our method through experiments on corpus with comparison to the k-means algorithm. We mainly discuss the result on aspect of word sense distinction, part-of-speech distinction and window size choosing.