A New Word Mining Method Based on Fast-text Model

Luozheng Li,Peipei Song,Dan Zhang,Dongyan Zhao
DOI: https://doi.org/10.1142/s2717554522500011
2021-01-01
International Journal of Asian Language Processing
Abstract:It is a challenging task to automatically excavate new words from huge amounts of domain-specific corpus. This paper proposed a new phrase mining method based on a fast-text model. With existing terminology and iterative computing framework, this method can extract high-quality domain terms from a given domain-specific corpus. Experiments on geographic corpus show that mining results of this method are more accurate compared with that of the traditional unsupervised mining method, with an accuracy of 68.6% in the top 500 candidates. Meanwhile, the scoring system is more reasonable and effective. In general, this new word mining method can effectively utilize external knowledge and save costs of manual labeling. It is more practical and important to the construction and expansion of the glossary.
What problem does this paper attempt to address?