Abstract:The integration of reasoning, learning, and decision-making is key to build more general artificial intelligence systems. As a step in this direction, we propose a novel neural-logic architecture, called differentiable logic machine (DLM), that can solve both inductive logic programming (ILP) and reinforcement learning (RL) problems, where the solution can be interpreted as a first-order logic program. Our proposition includes several innovations. Firstly, our architecture defines a restricted but expressive continuous relaxation of the space of first-order logic programs by assigning weights to predicates instead of rules, in contrast to most previous neural-logic approaches. Secondly, with this differentiable architecture, we propose several (supervised and RL) training procedures, based on gradient descent, which can recover a fully-interpretable solution (i.e., logic formula). Thirdly, to accelerate RL training, we also design a novel critic architecture that enables actor-critic algorithms. Fourthly, to solve hard problems, we propose an incremental training procedure that can learn a logic program progressively. Compared to state-of-the-art (SOTA) differentiable ILP methods, DLM successfully solves all the considered ILP problems with a higher percentage of successful seeds (up to 3.5$\times$). On RL problems, without requiring an interpretable solution, DLM outperforms other non-interpretable neural-logic RL approaches in terms of rewards (up to 3.9%). When enforcing interpretability, DLM can solve harder RL problems (e.g., Sorting, Path) Moreover, we show that deep logic programs can be learned via incremental supervised training. In addition to this excellent performance, DLM can scale well in terms of memory and computational time, especially during the testing phase where it can deal with much more constants ($>$2$\times$) than SOTA.

Deep Inductive Logic Programming meets Reinforcement Learning

Incorporating Relational Background Knowledge into Reinforcement Learning via Differentiable Inductive Logic Programming

Deep Reinforcement Learning for NLP.

Learning explanatory logical rules in non-linear domains: a neuro-symbolic approach

Off-Policy Differentiable Logic Reinforcement Learning

Differentiable Logic Machines

Deep Reinforcement Learning with Temporal Logics

Towards Relational Multi-Agent Reinforcement Learning Via Inductive Logic Programming

Learning First-Order Rules with Differentiable Logic Program Semantics

A Differentiable First-Order Rule Learner for Inductive Logic Programming

A Framework for Following Temporal Logic Instructions with Unknown Causal Dependencies

Differentiable Logic Policy for Interpretable Deep Reinforcement Learning: A Study From an Optimization Perspective

Differentiable Inductive Logic Programming in High-Dimensional Space

Internal Logical Induction for Pixel-Symbolic Reinforcement Learning

Neuro-Symbolic Hierarchical Rule Induction

A Survey on Inductive Logic Programming

Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision.

Learning Symbolic Rules for Interpretable Deep Reinforcement Learning

Learning Onto-Relational Rules with Inductive Logic Programming

Differentiable Logic Programming for Probabilistic Reasoning

Reinforcement Learning with Markov Logic Networks