Abstract:Commonsense question answering has demonstrated considerable potential across various applications like assistants and social robots. Although fully fine-tuned pre-trained Language Models(LM) have achieved remarkable performance in commonsense reasoning, their tendency to excessively prioritize textual information hampers the precise transfer of structural knowledge and undermines interpretability. Some studies have explored combining LMs with Knowledge Graphs(KGs) by coarsely fusing the two modalities to perform Graph Neural Network(GNN)-based reasoning that lacks a profound interaction between heterogeneous modalities. In this paper, we propose a novel Graph-based Structure-Aware Prompt Learning Model for commonsense reasoning, named G-SAP, aiming to maintain a balance between heterogeneous knowledge and enhance the cross-modal interaction within the LM+GNNs model. In particular, an evidence graph is constructed by integrating multiple knowledge sources, i.e. ConceptNet, Wikipedia, and Cambridge Dictionary to boost the performance. Afterward, a structure-aware frozen PLM is employed to fully incorporate the structured and textual information from the evidence graph, where the generation of prompts is driven by graph entities and relations. Finally, a heterogeneous message-passing reasoning module is used to facilitate deep interaction of knowledge between the LM and graph-based networks. Empirical validation, conducted through extensive experiments on three benchmark datasets, demonstrates the notable performance of the proposed model. The results reveal a significant advancement over the existing models, especially, with 6.12% improvement over the SoTA LM+GNNs model on the OpenbookQA dataset.

Unsupervised Deep Structured Semantic Models for Commonsense Reasoning.

A Hybrid Neural Network Model for Commonsense Reasoning

UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations

Combing Context and Commonsense Knowledge Through Neural Networks for Solving Winograd Schema Problems.

CommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Models

Commonsense Generative Model for Chinese Automatic Knowledge Graph Construction

Open-ended Commonsense Reasoning with Unrestricted Answer Scope

Evaluating Commonsense in Pre-trained Language Models

A Semantic-based Method for Unsupervised Commonsense Question Answering

Alleviating the Knowledge-Language Inconsistency: A Study for Deep Commonsense Knowledge

Commonsense Reasoning for Conversational AI: A Survey of the State of the Art

What Really is Commonsense Knowledge?

Natural Language Processing with Commonsense Knowledge: A Survey

Commonsense Knowledge Enhanced Embeddings for Solving Pronoun Disambiguation Problems in Winograd Schema Challenge

A Graph-Guided Reasoning Approach for Open-ended Commonsense Question Answering

MORE: Multi-mOdal REtrieval Augmented Generative Commonsense Reasoning

Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering

Things not Written in Text: Exploring Spatial Commonsense from Visual Signals

Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning

Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles

G-SAP: Graph-based Structure-Aware Prompt Learning over Heterogeneous Knowledge for Commonsense Reasoning