Applying Spectral Clustering for Chinese Word Sense Induction.

Zhengyan He,Yang Song,Houfeng Wang

2010-01-01

Abstract:Sense Induction is the process of identifying the word sense given its context, often treated as a clustering task. This paper explores the use of spectral cluster method which incorporates word features and n-gram features to determine which cluster the word belongs to, each cluster represents one sense in the given document set.

What problem does this paper attempt to address?

Inducing Word Senses for Cross-lingual Document Clustering

Guoyu Tang,Yunqing Xia,Erik Cambria,Peng Jin

DOI: https://doi.org/10.1109/cis.2013.93

2013-01-01

Abstract:Cross-lingual document clustering is the task of automatically organizing a large collection of cross-lingual documents into a few groups according to their content or topic. It is well known that language barrier and translation ambiguity are two challenging issues for cross-lingual document representation. To address such issues, we propose to represent cross-lingual documents through statistical word senses, which are learned from a parallel corpus by means of a novel cross-lingual word sense induction model. Furthermore, a sense clustering method is adopted to discover semantic relation of word senses, which are used to represent cross-lingual documents through a sense-based vector space model. Evaluation on a benchmarking dataset shows that the proposed model outperforms two state-of-the-art models in cross-lingual document clustering.
Soochow University: Description and Analysis of the Chinese Word Sense Induction System for CLP2010.

Hua Xu,Bing Liu,Longhua Qian,Guodong Zhou

2010-01-01

Abstract:Recent studies on word sense induction (WSI) mainly concentrate on European languages, Chinese word sense induction is becoming popular as it presents a new challenge to WSI. In this paper, we propose a feature-based approach using the spectral clustering algorithm to this problem. We also compare various clustering algorithms and similarity metrics. Experimental results show that our system achieves promising performance in F-score.
Chinese Word Sense Induction with Basic Clustering Algorithms.

Yuxiang Jia,Shiwen Yu,Zhengyan Chen

2010-01-01

Abstract:Word Sense Induction (WSI) is an important topic in natural langage processing area. For the bakeoff task Chinese Word Sense Induction (CWSI), this paper proposes two systems using basic clustering algorithms, k-means and agglomerative clustering. Experimental results show that k-means achieves a better performance. Based only on the data provided by the task organizers, the two systems get FScores of 0.7812 and 0.7651 respectively.
New Word Identification in Social Network Text Based on Time Series Information

Meng Wang,Lanfen Lin,Feng Wang

DOI: https://doi.org/10.1109/cscwd.2014.6846904

2014-01-01

Abstract:Different from the languages widely used in western countries such as English or French, there are no spaces between words in Chinese language, and a segmentation of the texts is necessary before other superior processes. New word identification is an important problem in the segmentation process, especially when the segmentation targets are social network texts which have more abbreviated words or other non-standard representations. Several methods have been proposed to detect Chinese new words. Most of these methods take the corpus as a static set and they don't consider the time domain information. Different from these studies, we regard our social network corpus as a text series spreading along the time line and design a new kind of features named dynamic features which can reflect the temporal variety of the string's statistical features. The experimental results on the dataset crawled from the biggest microblogging application in China show that this method can significantly improve the effect of Chinese new word identification.
Document Representation with Statistical Word Senses in Cross-Lingual Document Clustering

Guoyu Tang,Yunqing Xia,Erik Cambria,Peng Jin,Thomas Fang Zheng

DOI: https://doi.org/10.1142/s021800141559003x

IF: 1.261

2015-01-01

International Journal of Pattern Recognition and Artificial Intelligence

Abstract:Cross-lingual document clustering is the task of automatically organizing a large collection of multi-lingual documents into a few clusters, depending on their content or topic. It is well known that language barrier and translation ambiguity are two challenging issues for cross-lingual document representation. To this end, we propose to represent cross-lingual documents through statistical word senses, which are automatically discovered from a parallel corpus through a novel cross-lingual word sense induction model and a sense clustering method. In particular, the former consists in a sense-based vector space model and the latter leverages on a sense-based latent Dirichlet allocation. Evaluation on the benchmarking datasets shows that the proposed models outperform two state-of-the-art methods for cross-lingual document clustering.
Document Clustering Based on Word Sense Cluster

TANG Guoyu,XIA Yunqing,ZHANG Min,ZHENG Fang

DOI: https://doi.org/10.3969/j.issn.1003-0077.2013.03.015

2013-01-01

Abstract:Document representation is the key part in document clustering.In this paper,we aim at improving document representation in document clustering.Synonymy and polysemy are two challenging issues in document representation.Inspired by the observation that synonymy and polysemy are mainly related to word sense,we present a novel model,referred to as Sense Cluster Model(SCM),to address both issues by representing documents with word sense clusters.In SCM,word sense clusters are first constructed from the development dataset by 1) the word sense induction to automatically discover different senses of each word from raw text;and 2) the word sense clusteringto recognize identical or similar words.Then the probability distribution over word sense clusters is generated to represent every document after word sense disambiguation.The experiments conducted on benchmarking data show that the SCM model outperforms both baseline and the classic topic model,LDA,in the task of document clustering.
Word Clustering for Collocation-Based Word Sense Disambiguation

Peng Jin,Xu Sun,Yunfang Wu,Shiwen Yu

DOI: https://doi.org/10.1007/978-3-540-70939-8_24

2007-01-01

Abstract:The main disadvantage of collocation-based word sense disambiguation is that the recall is low, with relatively high precision. How to improve the recall without decrease the precision? In this paper, we investigate a word-class approach to extend the collocation list which is constructed from the manually sense-tagged corpus. But the word classes are obtained from a larger scale corpus which is not sense tagged. The experiment results have shown that the F-measure is improved to 71% compared to 54% of the baseline system where the word-class is not considered, although the precision decreases slightly. Further study discovers the relationship between the F-measure and the number of word-class trained from the various sizes of corpus.
Coarse-Grained Word Sense Disambiguation Using Features Described in the Lexicon

WU Yun-fang,JIN Peng,GUO Tao

DOI: https://doi.org/10.3969/j.issn.1003-0077.2007.02.001

2007-01-01

Abstract:This paper presents a simple but effective feature-based approach to Chinese word sense disambiguation using the distributional features available from the Grammatical Knowledge-base of Contemporary Chinese. The test data is the sense-tagged corpus of People's Daily.A Nave Bayes classifier is also tried as a comparable statistical method.The feature-based approach achieves precision of 90%,which is comparable to the NB classifier.The striking advantages of the feature-based approach are 1) It is not influenced by the data size,and 2) It can disambiguate some specific words with precision of 100%.The features appropriate for different parts of speech in Chinese WSD are also discussed.This paper demonstrates that sense features described in the lexicon are worth including in WSD.
Induction of Semantic Classes Based on Coordinate Patterns

Likun Qiu,Yunfang Wu,Jing Shi,Yanqiu Shao,Zhiyi Long

DOI: https://doi.org/10.1109/WI-IAT.2011.66

2011-01-01

Abstract:Many NLP and IR applications require semantic classification knowledge of words. However, manually constructing semantic classes is a time-consuming and labor-intensive task. In this paper, we present an algorithm for induction of Chinese semantic classes from natural language text based on coordinate patterns. First, several coordinate patterns are proposed to harvest high-quality coordinate instance. Second, an iterative clustering process is used to cluster words into semantic classes. The clustering process mainly used coordinate relation between words. Experiment results show that the proposed approach performs relatively well and achieves 53.2% in terms of precision. Finally, a thesaurus containing about 15000 Chinese words is generated automatically.
To Word Senses and Beyond: Inducing Concepts with Contextualized Language Models

Bastien Liétard,Pascal Denis,Mikaella Keller

2024-06-29

Abstract:Polysemy and synonymy are two crucial interrelated facets of lexical ambiguity. While both phenomena have been studied extensively in NLP, leading to dedicated systems, they are often been considered independently. While many tasks dealing with polysemy (e.g. Word Sense Disambiguiation or Induction) highlight the role of a word's senses, the study of synonymy is rooted in the study of concepts, i.e. meaning shared across the lexicon. In this paper, we introduce Concept Induction, the unsupervised task of learning a soft clustering among words that defines a set of concepts directly from data. This task generalizes that of Word Sense Induction. We propose a bi-level approach to Concept Induction that leverages both a local lemma-centric view and a global cross-lexicon perspective to induce concepts. We evaluate the obtained clustering on SemCor's annotated data and obtain good performances (BCubed F1 above 0.60). We find that the local and the global levels are mutually beneficial to induce concepts and also senses in our setting. Finally, we create static embeddings representing our induced concepts and use them on the Word-in-Context task, obtaining competitive performances with the State-of-the-Art.

Computation and Language
Research of Term Semantic Hierarchy Induction for Domain-specific Chinese Text Information Processing

Ji Peipei,Yan Xiaoyan,Cen Yonghua,Wang Lingyan

DOI: https://doi.org/10.11925/infotech.1003-3513.2010.09.07

2010-01-01

Abstract:Term semantic relationship is a key step of Chinese text information processing.Through researches on some existing methods at home and abroad,a process of term semantic hierarchy induction is proposed,which uses multiple clustering method to get the whole hierarchy,and combine with comprehensive similarity caculation to get the label of classes.Finally,some experiments are done to verify its rationality.
Computing Word Classes Using Spectral Clustering

Effi Levi,Saggy Herman,Ari Rappoport

DOI: https://doi.org/10.48550/arXiv.1808.05374

2018-08-16

Abstract:Clustering a lexicon of words is a well-studied problem in natural language processing (NLP). Word clusters are used to deal with sparse data in statistical language processing, as well as features for solving various NLP tasks (text categorization, question answering, named entity recognition and others). Spectral clustering is a widely used technique in the field of image processing and speech recognition. However, it has scarcely been explored in the context of NLP; specifically, the method used in this (Meila and Shi, 2001) has never been used to cluster a general word lexicon. We apply spectral clustering to a lexicon of words, evaluating the resulting clusters by using them as features for solving two classical NLP tasks: semantic role labeling and dependency parsing. We compare performance with Brown clustering, a widely-used technique for word clustering, as well as with other clustering methods. We show that spectral clusters produce similar results to Brown clusters, and outperform other clustering methods. In addition, we quantify the overlap between spectral and Brown clusters, showing that each model captures some information which is uncaptured by the other.

Computation and Language
Graph-based Clustering for Detecting Semantic Change Across Time and Languages

Xianghe Ma,Michael Strube,Wei Zhao

2024-02-02

Abstract:Despite the predominance of contextualized embeddings in NLP, approaches to detect semantic change relying on these embeddings and clustering methods underperform simpler counterparts based on static word embeddings. This stems from the poor quality of the clustering methods to produce sense clusters -- which struggle to capture word senses, especially those with low frequency. This issue hinders the next step in examining how changes in word senses in one language influence another. To address this issue, we propose a graph-based clustering approach to capture nuanced changes in both high- and low-frequency word senses across time and languages, including the acquisition and loss of these senses over time. Our experimental results show that our approach substantially surpasses previous approaches in the SemEval2020 binary classification task across four languages. Moreover, we showcase the ability of our approach as a versatile visualization tool to detect semantic changes in both intra-language and inter-language setups. We make our code and data publicly available.

Computation and Language
Word sense induction using word embeddings and community detection in complex networks

Edilson A. Corrêa Jr.,Diego R. Amancio

DOI: https://doi.org/10.1016/j.physa.2019.02.032

2018-03-23

Abstract:Word Sense Induction (WSI) is the ability to automatically induce word senses from corpora. The WSI task was first proposed to overcome the limitations of manually annotated corpus that are required in word sense disambiguation systems. Even though several works have been proposed to induce word senses, existing systems are still very limited in the sense that they make use of structured, domain-specific knowledge sources. In this paper, we devise a method that leverages recent findings in word embeddings research to generate context embeddings, which are embeddings containing information about the semantical context of a word. In order to induce senses, we modeled the set of ambiguous words as a complex network. In the generated network, two instances (nodes) are connected if the respective context embeddings are similar. Upon using well-established community detection methods to cluster the obtained context embeddings, we found that the proposed method yields excellent performance for the WSI task. Our method outperformed competing algorithms and baselines, in a completely unsupervised manner and without the need of any additional structured knowledge source.

Computation and Language,Social and Information Networks
Combining Neural Language Models for WordSense Induction

Nikolay Arefyev,Boris Sheludko,Tatiana Aleksashina

DOI: https://doi.org/10.1007/978-3-030-37334-4_10

2020-06-24

Abstract:Word sense induction (WSI) is the problem of grouping occurrences of an ambiguous word according to the expressed sense of this word. Recently a new approach to this task was proposed, which generates possible substitutes for the ambiguous word in a particular context using neural language models, and then clusters sparse bag-of-words vectors built from these substitutes. In this work, we apply this approach to the Russian language and improve it in two ways. First, we propose methods of combining left and right contexts, resulting in better substitutes generated. Second, instead of fixed number of clusters for all ambiguous words we propose a technique for selecting individual number of clusters for each word. Our approach established new state-of-the-art level, improving current best results of WSI for the Russian language on two RUSSE 2018 datasets by a large margin.

Computation and Language
Researches on Word Sense Discrimination of Chinese Adjective

朱虹,刘扬,俞士汶

DOI: https://doi.org/10.3969/j.issn.1003-0077.2009.06.003

2009-01-01

Abstract:Lexical knowledge acquisition is the bottleneck for many tasks like word sense disambiguation,lexical knowledge base construction et al.This paper introduces an automatic word sense discrimination method for Chinese mid-high-frequency adjectives.We employ the EM algorithm and exploit the features of Chinese character,contextual bag-of-words and host-attribute pair instead of the more unreliable syntactic information.We further optimize the morphology selection by utilizing HowNet in our work.The experimental results show that word sense discrimination results are different from Chinese lexicons and could be used for lexicon modification and expansion even for other type of Chinese words.
Inducing Word Sense with Automatically Learned Hidden Concepts.

Baobao Chang,Wenzhe Pei,Miaohong Chen

2014-01-01

Abstract:Word Sense Induction (WSI) aims to automatically induce meanings of a polysemous word from unlabeled corpora. In this paper, we first propose a novel Bayesian parametric model to WSI. Unlike previous work, our research introduces a layer of hidden concepts and view senses as mixtures of concepts. We believe that concepts generalize the contexts, allowing the model to measure the sense similarity at a more general level. The Zipf’s law of meaning is used as a way of pre-setting the sense number for the parametric model. We further extend the parametric model to non-parametric model which not only simplifies the problem of model selection but also brings improved performance. We test our model on the benchmark datasets released by Semeval-2010 and Semeval-2007. The test results show that our model outperforms state-of-theart systems.
A Study in Dictionary-Based All-word Word Sense Disambiguation for Pre-Qin Chinese

ZHANG Yingjie,LI Bin,CHEN Jiajun,CHEN Xiaohe

DOI: https://doi.org/10.3969/j.issn.1003-0077.2012.03.012

2012-01-01

Abstract:Word Sense Disambiguation(WSD) is a basic task of Natural Language Processing,including the processing of ancient Chinese documents.In this paper we focuse on the specific field of analyzing pre-Qin ancient Chinese documents.Considering the shortage of training data and semantic resources,we employe a semi-supervised machine learning method to perform all-word WSD of Zuo Zhuan and use Chinese Dictionary v2.0 as the knowledge resource.We randomly selecte 22 words of different frequency and sense number to evaluate the proposed method.On the selected words,our method achieves an average accuracy of 67%,which is significant higher than the baseline method of selecting the most frequent sense.This method is promising for sense tagging of ancient Chinese documents when there is no training data available.It also provides a raw sense tagging result for human correction,enriching traditional dictionaries which usually suffer from insufficient word sense entries.
Unsupervised Word Sense Discrimination Improves Construction of the Wordnets.

Hong Zhu,Yang Liu

DOI: https://doi.org/10.1109/wi-iat.2009.282

2009-01-01

Abstract:This paper reports an unsupervised approach we adopted for Wordnet construction. We combine ways of CBC word sense discrimination and automatic mapping measures to remove redundant senses of adjectives in Chinese Concept Dictionary. We optimize the similarity measure of adjective concepts and put forward two novel strategies to automatically evaluate the model's performance. The experimental results show a rather fine conformance to the work based on human judgments.
Chinese word sense disambiguation using hownet

Yuntao Zhang,Ling Gong,Yongcheng Wang

DOI: https://doi.org/10.1007/11539087_123

2005-01-01

Abstract:Word sense disambiguation plays an important role in natural language processing, such as information retrieval, text summarization, machine translation etc. This paper proposes a corpus-based Chinese word sense disambiguation approach using HowNet. The method is based on the co-occurrence frequency between the relatives (such as synonym, antonymy, meronymy) of target word and each word in the context. Further, domains have been used to characterize the senses of polysemous word. To our knowledge, this is the first time a Chinese word sense disambiguation method using domain knowledge is reported. The accuracy is 73.2% at present. The experimental result shows that the method is very promising for Chinese word sense disambiguation.

Applying Spectral Clustering for Chinese Word Sense Induction.

Inducing Word Senses for Cross-lingual Document Clustering

Soochow University: Description and Analysis of the Chinese Word Sense Induction System for CLP2010.

Chinese Word Sense Induction with Basic Clustering Algorithms.

New Word Identification in Social Network Text Based on Time Series Information

Document Representation with Statistical Word Senses in Cross-Lingual Document Clustering

Document Clustering Based on Word Sense Cluster

Word Clustering for Collocation-Based Word Sense Disambiguation

Coarse-Grained Word Sense Disambiguation Using Features Described in the Lexicon

Induction of Semantic Classes Based on Coordinate Patterns

To Word Senses and Beyond: Inducing Concepts with Contextualized Language Models

Research of Term Semantic Hierarchy Induction for Domain-specific Chinese Text Information Processing

Computing Word Classes Using Spectral Clustering

Graph-based Clustering for Detecting Semantic Change Across Time and Languages

Word sense induction using word embeddings and community detection in complex networks

Combining Neural Language Models for WordSense Induction

Researches on Word Sense Discrimination of Chinese Adjective

Inducing Word Sense with Automatically Learned Hidden Concepts.

A Study in Dictionary-Based All-word Word Sense Disambiguation for Pre-Qin Chinese

Unsupervised Word Sense Discrimination Improves Construction of the Wordnets.

Chinese word sense disambiguation using hownet