Abstract:Profiting from the advance of deep convolutional networks, current state-of-the-art video action recognition models have achieved remarkable progress. Nevertheless, most of existing models suffer from low interpretability of the predicted actions. Inspired by the observation that temporally-configured human-object interactions often serve as a key indicator of many actions, this work crafts an action reasoning framework that performs Markov Logic Network (MLN) based probabilistic logical inference. Crucially, we propose to encode an action by first-order logical rules that correspond to the temporal changes of visual relationships in videos. The main contributions of this work are two-fold: 1) Different from existing black-box models, the proposed model simultaneously implements the localization of temporal boundaries and the recognition of action categories by grounding the logical rules of MLN in videos. The weight associated with each such rule further provides an estimate of confidence. These collectively make our model more explainable and robust. 2) Instead of using hand-crafted logical rules in conventional MLN, we develop a data-driven instantiation of the MLN. In specific, a hybrid learning scheme is proposed. It combines MLN's weight learning and reinforcement learning, using the former's results as a self-critic for guiding the latter's training. Additionally, by treating actions as logical predicates, the proposed framework can also be integrated with deep models for further performance boost. Comprehensive experiments on two complex video action datasets (Charades & CAD-120) clearly demonstrate the effectiveness and explainability of our proposed method.

Reinforcement Learning with Markov Logic Networks

Numerical Markov Logic Network: A Scalable Probabilistic Framework for Hybrid Knowledge Inference

Survey of Markov Logic Networks

Application of Markov Logic Network in Transfer Learning

Complex Video Action Reasoning Via Learnable Markov Logic Network

Domain Aware Markov Logic Networks

Data Conflict Resolution with Markov Logic Networks

Incremental Learning of Affordances using Markov Logic Networks

Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses

Markov logic networks based link prediction

Markov logic networks technology application for domain knowledge learning

Learning Logic Rules for the Tower of Knowledge Using Markov Logic Networks

Probabilistic Logic Neural Networks for Reasoning

Agent Learning in Relational Domains Based on Logical Mdps with Negation

Efficient Probabilistic Logic Reasoning with Graph Neural Networks

Deep Inductive Logic Programming meets Reinforcement Learning

RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs

Efficient Inference of Markov Logic Network for Link Prediction

MCLA: Research on Cumulative Learning of Markov Logic Network

A Hybrid Driving Decision-Making System Integrating Markov Logic Networks and Connectionist AI

LMRL: a Multi-Agent Reinforcement Learning Model and Algorithm