Subgraph Retrieval Enhanced by Graph-Text Alignment for Commonsense Question Answering

Boci Peng,Yongchao Liu,Xiaohe Bo,Sheng Tian,Baokun Wang,Chuntao Hong,Yan Zhang

DOI: https://doi.org/10.1007/978-3-031-70365-2_3

2024-11-11

Abstract:Commonsense question answering is a crucial task that requires machines to employ reasoning according to commonsense. Previous studies predominantly employ an extracting-and-modeling paradigm to harness the information in KG, which first extracts relevant subgraphs based on pre-defined rules and then proceeds to design various strategies aiming to improve the representations and fusion of the extracted structural knowledge. Despite their effectiveness, there are still two challenges. On one hand, subgraphs extracted by rule-based methods may have the potential to overlook critical nodes and result in uncontrollable subgraph size. On the other hand, the misalignment between graph and text modalities undermines the effectiveness of knowledge fusion, ultimately impacting the task performance. To deal with the problems above, we propose a novel framework: \textbf{S}ubgraph R\textbf{E}trieval Enhanced by Gra\textbf{P}h-\textbf{T}ext \textbf{A}lignment, named \textbf{SEPTA}. Firstly, we transform the knowledge graph into a database of subgraph vectors and propose a BFS-style subgraph sampling strategy to avoid information loss, leveraging the analogy between BFS and the message-passing mechanism. In addition, we propose a bidirectional contrastive learning approach for graph-text alignment, which effectively enhances both subgraph retrieval and knowledge fusion. Finally, all the retrieved information is combined for reasoning in the prediction module. Extensive experiments on five datasets demonstrate the effectiveness and robustness of our framework.

Machine Learning,Artificial Intelligence,Computation and Language,Social and Information Networks

What problem does this paper attempt to address?

The problems that this paper attempts to solve are the two main challenges faced in the Commonsense Question Answering (CSQA) task:

1. **Quality issues in sub - graph extraction**: Existing rule - based methods may overlook key nodes when extracting relevant sub - graphs from the Knowledge Graph (KG), resulting in an uncontrollable sub - graph size. As the number of hops increases, the number of nodes expands dramatically, increasing the burden on the model. Meanwhile, some important entities may not be learned, leading to information loss.

2. **Alignment issues between graph and text modalities**: The misalignment between the graph encoder and the text encoder will weaken the effect of knowledge fusion and ultimately affect task performance. Especially in the case of limited data, this misalignment will be more obvious.

To solve these problems, the authors propose a new framework - **Subgraph Retrieval Enhanced by Graph - Text Alignment (SEPTA)**. Specifically, SEPTA mainly solves the problems in the following ways:

- **Construction of sub - graph vector database**: Convert the knowledge graph into a sub - graph vector database, avoiding the problem of sub - graph quality degradation caused by simple string or semantic matching. Through a breadth - first search (BFS) - style sampling strategy, ensure the integrity of neighbor information for each node and reduce information loss.

- **Bidirectional contrastive learning method**: Propose a bidirectional contrastive learning method to align the semantic spaces of graphs and texts, effectively improving the accuracy of sub - graph retrieval and the effect of knowledge fusion. High - quality semantically equivalent training pairs are constructed through graph - to - text methods without the need for an additional corpus, making it easy to train.

- **Query enhancement strategy**: In order to better adapt to the alignment process, a query enhancement strategy is proposed. By retrieving triples related to the question in the knowledge graph and connecting them to the question - answer pairs, the retrieval accuracy is improved.

Through these improvements, the experimental results of SEPTA on multiple datasets show that this framework not only outperforms existing methods in performance but also performs well in weakly - supervised settings.