Abstract:Profiting from the advance of deep convolutional networks, current state-of-the-art video action recognition models have achieved remarkable progress. Nevertheless, most of existing models suffer from low interpretability of the predicted actions. Inspired by the observation that temporally-configured human-object interactions often serve as a key indicator of many actions, this work crafts an action reasoning framework that performs Markov Logic Network (MLN) based probabilistic logical inference. Crucially, we propose to encode an action by first-order logical rules that correspond to the temporal changes of visual relationships in videos. The main contributions of this work are two-fold: 1) Different from existing black-box models, the proposed model simultaneously implements the localization of temporal boundaries and the recognition of action categories by grounding the logical rules of MLN in videos. The weight associated with each such rule further provides an estimate of confidence. These collectively make our model more explainable and robust. 2) Instead of using hand-crafted logical rules in conventional MLN, we develop a data-driven instantiation of the MLN. In specific, a hybrid learning scheme is proposed. It combines MLN's weight learning and reinforcement learning, using the former's results as a self-critic for guiding the latter's training. Additionally, by treating actions as logical predicates, the proposed framework can also be integrated with deep models for further performance boost. Comprehensive experiments on two complex video action datasets (Charades & CAD-120) clearly demonstrate the effectiveness and explainability of our proposed method.

Application of Markov Logic Network in Transfer Learning

Markov logic networks technology application for domain knowledge learning

Numerical Markov Logic Network: A Scalable Probabilistic Framework for Hybrid Knowledge Inference

Reinforcement Learning with Markov Logic Networks

Sentiment Classification Technology Based on Markov Logic Networks

Markov Logic Networks with its application in De-duplication

Markov logic networks based link prediction

Survey of Markov Logic Networks

Domain Aware Markov Logic Networks

MCLA: Research on Cumulative Learning of Markov Logic Network

Learning Logic Rules for the Tower of Knowledge Using Markov Logic Networks

Data Conflict Resolution with Markov Logic Networks

Efficient Inference of Markov Logic Network for Link Prediction

Complex Video Action Reasoning Via Learnable Markov Logic Network

Incremental Learning of Affordances using Markov Logic Networks

Probabilistic Logic Neural Networks for Reasoning

Novel approach to human activity recognition based on improved Markov logic networks

Transferring knowledge from human-demonstration trajectories to reinforcement learning

Traffic Flow and Speed Forecasting Through a Bayesian Deep Multi-Linear Relationship Network.

Generating Markov Logic Networks Rulebase Based on Probabilistic Latent Semantics Analysis

Multi-label Transfer Learning Via Latent Graph Alignment