Abstract:Reinforcement learning (RL) agents are often designed specifically for a particular problem and they generally have uninterpretable working processes. Statistical methods-based agent algorithms can be improved in terms of generalizability and interpretability using symbolic Artificial Intelligence (AI) tools such as logic programming. In this study, we present a model-free RL architecture that is supported with explicit relational representations of the environmental objects. For the first time, we use the PrediNet network architecture in a dynamic decision-making problem rather than image-based tasks, and Multi-Head Dot-Product Attention Network (MHDPA) as a baseline for performance comparisons. We tested two networks in two environments ---i.e., the baseline Box-World environment and our novel environment, Relational-Grid-World (RGW). With the procedurally generated RGW environment, which is complex in terms of visual perceptions and combinatorial selections, it is easy to measure the relational representation performance of the RL agents. The experiments were carried out using different configurations of the environment so that the presented module and the environment were compared with the baselines. We reached similar policy optimization performance results with the PrediNet architecture and MHDPA; additionally, we achieved to extract the propositional representation explicitly ---which makes the agent's statistical policy logic more interpretable and tractable. This flexibility in the agent's policy provides convenience for designing non-task-specific agent architectures. The main contributions of this study are two-fold ---an RL agent that can explicitly perform relational reasoning, and a new environment that measures the relational reasoning capabilities of RL agents.

Towards Relational Multi-Agent Reinforcement Learning Via Inductive Logic Programming

The Multi-Agent System Based on Reinforcement Learning

Incorporating Relational Background Knowledge into Reinforcement Learning via Differentiable Inductive Logic Programming

Deep Inductive Logic Programming meets Reinforcement Learning

Guiding Multi-agent Multi-task Reinforcement Learning by a Hierarchical Framework with Logical Reward Shaping

Cooperative Learning of Multi-Agent Systems Via Reinforcement Learning

Relational-Grid-World: A Novel Relational Reasoning Environment and An Agent Model for Relational Information Extraction

Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation

A new multi-agent reinforcement learning approach

Hybrid Information-driven Multi-agent Reinforcement Learning

Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration

Attentional Policies for Cross-Context Multi-Agent Reinforcement Learning

Relational inductive bias for physical construction in humans and machines

Agent Learning in Relational Domains Based on Logical Mdps with Negation

Intention Propagation for Multi-agent Reinforcement Learning

A Novel Multi-Agent Reinforcement Learning Approach

Hierarchical relationship modeling in multi-agent reinforcement learning for mixed cooperative–competitive environments

Relational Q-Functionals: Multi-Agent Learning to Recover from Unforeseen Robot Malfunctions in Continuous Action Domains

A Two-Layered Multi-Agent Reinforcement Learning Model and Algorithm

LMRL: a Multi-Agent Reinforcement Learning Model and Algorithm

Stabilizing Multi-Agent Deep Reinforcement Learning by Implicitly Estimating Other Agents’ Behaviors