Abstract:Multimodal knowledge graph (MKG) reasoning has attracted significant attention since impressive performance has been achieved by adding multimodal auxiliary information (i.e., texts and images) to the entities of traditional KGs. However, existing studies heavily rely on path-based methods for learning structural modality, failing to capture the complex structural interactions among multimodal entities beyond the reasoning path. In addition, existing studies have largely ignored the dynamic impact of different multimodal features on different decision facts for reasoning, which utilize asymmetric coattention to independently learn the static interplay between different modalities without dynamically joining the reasoning process. We propose a novel Dynamic Structure-aware representation learning method, namely DySarl, to overcome this problem and significantly improve the MKG reasoning performance. Specifically, we devise a dual-space multihop structural learning module in DySarl, aggregating the multihop structural features of multimodal entities via a novel message-passing mechanism. It integrates the message paradigms in Euclidean and hyperbolic spaces, effectively preserving the neighborhood information beyond the limited multimodal query paths. Furthermore, DySarl has an interactive symmetric attention module to explicitly learn the dynamic impacts of unimodal attention senders and multimodal attention targets on decision facts through a newly designed symmetric attention component and fact-specific gated attention unit, equipping DySarl with the dynamic associations between the multimodal feature learning and later reasoning. Extensive experiments show that DySarl achieves significantly improved reasoning performance on two public MKG datasets compared with that of the state-of-the-art baselines. Source codes are available at https://github.com/HUSTNLP-codes/DySarl.

DMON: A Simple yet Effective Approach for Argument Structure Learning

Probing Structural Knowledge from Pre-trained Language Model for Argumentation Relation Classification

A Structure Self-Aware Model for Discourse Parsing on Multi-Party Dialogues

Multi-view Hierarchical Graph Neural Network for Argumentation Mining

A Neural Transition-based Model for Argumentation Mining

A Joint Framework for Argumentative Text Analysis Incorporating Domain Knowledge

Neural reality of argument structure constructions

Argument Structure Mining in Scientific Articles: A Comparative Analysis

Unsupervised Mutual Learning of Dialogue Discourse Parsing and Topic Segmentation

AutoAM: An End-To-End Neural Model for Automatic and Universal Argument Mining

Aspect-Based Argument Mining

Analysis of Argument Structure Constructions in a Deep Recurrent Language Model

DySarl: Dynamic Structure-Aware Representation Learning for Multimodal Knowledge Graph Reasoning

AMOSL: Adaptive Modality-wise Structure Learning in Multi-view Graph Neural Networks For Enhanced Unified Representation

Struct-X: Enhancing Large Language Models Reasoning with Structured Data

Multi-Task Learning Improves Performance In Deep Argument Mining Models

Deep Learning for Abstract Argumentation Semantics

MACT: Model-Agnostic Cross-Lingual Training for Discourse Representation Structure Parsing

Hierarchical Neural Network: Integrate Divide-and-Conquer and Unified Approach for Argument Unit Recognition and Classification

Active Learning for Argument Mining: A Practical Approach

Unsupervised Deep Structured Semantic Models for Commonsense Reasoning.