Abstract:<p>Knowledge graphs (KGs) can provide rich, structured information for recommendation systems as well as increase accuracy and perform explicit reasoning. Deep reinforcement learning (RL) has also sparked great interest in personalized recommendations. The combination of the two holds promise in carrying out interpretable causal inference procedures and improving the performance of graph-structured recommendation. However, most KG -based recommendation focus on rich semantic relationships between entities in a heterogeneous knowledge graph, and thus fail to fully make use of the image information corresponding to an entity. In order to address these issues, we proposed a novel Multi-modal Knowledge-aware Reinforcement Learning Network (MKRLN), which couples recommendation and interpretability by providing actual paths in multi-modal KG (MKG). The MKRLN can generate path representation by composing the structural and visual information of entities, and infers the underlying rational of agent-MKG interactions by leveraging the sequential dependencies within a path from the MKG. In addition, as KGs have too many attributes and entities, their combination with RL leads to too many action spaces and states in the reinforcement learning space, which complicates the search of action spaces. Furthermore, in order to solve this problem, we proposed a new hierarchical attention-path, which makes users focus their attention on the items they are interested in. This reduces the relations and entities in the KGs, which in turn reduces the action space and state in RL, shortens the path to the target entity, and improves the accuracy of recommendation. Our model has explicit explanation ability in knowledge and images. Finally, we extensively evaluated our model on several large-scale real-world benchmark datasets, and it yielded favorable results compared with state-of-the-art methods.</p>

Multi-modal Knowledge-aware Hierarchical Attention Network for Explainable Medical Question Answering.

MKGN: A Multi-Dimensional Knowledge Enhanced Graph Network for Multi-Hop Question and Answering

MHKD-MVQA: Multimodal Hierarchical Knowledge Distillation for Medical Visual Question Answering.

A Multi-granularity Fusion Neural Network Model for Medical Question Classification

Hierarchical deep multi-modal network for medical visual question answering

Multi-Scale Attentive Interaction Networks for Chinese Medical Question Answer Selection

Knowledge-enhanced Hierarchical Attention for Community Question Answering with Multi-task and Adaptive Learning.

Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering

A knowledge graph based question answering method for medical domain

Interpretable Hierarchical Attention Network for Medical Condition Identification

Multi-level Relational Reasoning and Attention Network for Medical Visual Question Answering

Medical knowledge-based network for Patient-oriented Visual Question Answering

Extracting Medical Knowledge from Crowdsourced Question Answering Website

BertHANK: Hierarchical Attention Networks with Enhanced Knowledge and Pre-Trained Model for Answer Selection

Discovering Multimodal Hierarchical Structures with Graph Neural Networks for Multi-modal and Multi-hop Question Answering.

Multi-modal Knowledge-aware Reinforcement Learning Network for Explainable Recommendation

Query and Attention Augmentation for Knowledge-Based Explainable Reasoning

Knowledge enhanced attention aggregation network for medicine recommendation

MedExQA: Medical Question Answering Benchmark with Multiple Explanations

Path-Aware Cross-Attention Network for Question Answering

HHH: An Online Medical Chatbot System based on Knowledge Graph and Hierarchical Bi-Directional Attention