A Semantics-enhanced Topic Modelling Technique: Semantic-LDA

Dakshi Kapugama Geeganage,Yue Xu,Yuefeng Li
DOI: https://doi.org/10.1145/3639409
IF: 4.157
2024-02-12
ACM Transactions on Knowledge Discovery from Data
Abstract:Topic modelling is a beneficial technique used to discover latent topics in text collections. But to correctly understand the text content and generate a meaningful topic list, semantics are important. By ignoring semantics, that is, not attempting to grasp the meaning of the words, most of the existing topic modelling approaches can generate some meaningless topic words. Even existing semantic-based approaches usually interpret the meanings of words without considering the context and related words. In this article, we introduce a semantic-based topic model called semantic-LDA that captures the semantics of words in a text collection using concepts from an external ontology. A new method is introduced to identify and quantify the concept–word relationships based on matching words from the input text collection with concepts from an ontology without using pre-calculated values from the ontology that quantify the relationships between the words and concepts. These pre-calculated values may not reflect the actual relationships between words and concepts for the input collection, because they are derived from datasets used to build the ontology rather than from the input collection itself. Instead, quantifying the relationship based on the word distribution in the input collection is more realistic and beneficial in the semantic capture process. Furthermore, an ambiguity handling mechanism is introduced to interpret the unmatched words, that is, words for which there are no matching concepts in the ontology. Thus, this article makes a significant contribution by introducing a semantic-based topic model that calculates the word–concept relationships directly from the input text collection. The proposed semantic-based topic model and an enhanced version with the disambiguation mechanism were evaluated against a set of state-of-the-art systems, and our approaches outperformed the baseline systems in both topic quality and information filtering evaluations.
computer science, information systems, software engineering
What problem does this paper attempt to address?
### Problems Addressed by the Paper The paper primarily addresses the issue of semantic understanding in the process of topic modeling. Specifically: 1. **Importance of Semantic Capture**: - Existing topic modeling methods mostly rely on statistical features (such as word frequency and co-occurrence), and these methods may overlook semantic information when understanding the text content, leading to generated topic words that are not meaningful enough. 2. **Limitations of Existing Methods**: - Current semantic-enhanced topic models typically use word-concept relationships from external knowledge bases directly, but these predefined relationships may not be applicable to specific document collections, thus failing to accurately reflect the actual word-concept relationships. 3. **Proposed New Method**: - The paper introduces a new semantic-enhanced topic model (Semantic-LDA), which captures the semantics of words by extracting word-concept relationships from the input document collection and generates the topic model based on this. - A new method is proposed to quantify the relationship between words and concepts, based on the distribution of words in the document collection rather than using predefined relationships from external knowledge bases. - A method for handling unmatched words is introduced, where if a word cannot be matched with any concept in the knowledge base, the meaning of these words is interpreted through the concepts of related or similar words. In summary, the paper aims to improve the semantic accuracy in the topic modeling process by combining external knowledge bases with the word distribution in document collections.