Abstract:While achieving tremendous success in various fields, existing multi-agentreinforcement learning (MARL) with a black-box neural network architecturemakes decisions in an opaque manner that hinders humans from understanding thelearned knowledge and how input observations influence decisions. Instead,existing interpretable approaches, such as traditional linear models anddecision trees, usually suffer from weak expressivity and low accuracy. Toaddress this apparent dichotomy between performance and interpretability, oursolution, MIXing Recurrent soft decision Trees (MIXRTs), is a novelinterpretable architecture that can represent explicit decision processes viathe root-to-leaf path and reflect each agent's contribution to the team.Specifically, we construct a novel soft decision tree to address partialobservability by leveraging the advances in recurrent neural networks, anddemonstrate which features influence the decision-making process through thetree-based model. Then, based on the value decomposition framework, we linearlyassign credit to each agent by explicitly mixing individual action values toestimate the joint action value using only local observations, providing newinsights into how agents cooperate to accomplish the task. Theoretical analysisshows that MIXRTs guarantees the structural constraint on additivity andmonotonicity in the factorization of joint action values. Evaluations on thechallenging Spread and StarCraft II tasks show that MIXRTs achieves competitiveperformance compared to widely investigated methods and delivers morestraightforward explanations of the decision processes. We explore a promisingpath toward developing learning algorithms with both high performance andinterpretability, potentially shedding light on new interpretable paradigms forMARL.

A Novel Tree-Based Method for Interpretable Reinforcement Learning

CDT: Cascading Decision Trees for Explainable Reinforcement Learning

RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Space

Can Differentiable Decision Trees Enable Interpretable Reward Learning from Human Feedback?

Interpretable Reinforcement Learning for Robotics and Continuous Control

Optimal Interpretability-Performance Trade-off of Classification Trees with Black-Box Reinforcement Learning

Conservative Q-Improvement: Reinforcement Learning for an Interpretable Decision-Tree Policy

Interpretable Modeling of Deep Reinforcement Learning Driven Scheduling

Learning Interpretable, High-Performing Policies for Autonomous Driving

Limits of Actor-Critic Algorithms for Decision Tree Policies Learning in IBMDPs

SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks

Optimizing Interpretable Decision Tree Policies for Reinforcement Learning

Neural-to-Tree Policy Distillation with Policy Improvement Criterion

Effective Interpretable Policy Distillation via Critical Experience Point Identification

TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning

MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning Via Mixing Recurrent Soft Decision Trees

BET: Explaining Deep Reinforcement Learning through The Error-Prone Decisions

Efficient Tree Policy with Attention-Based State Representation for Interactive Recommendation

Achieving efficient interpretability of reinforcement learning via policy distillation and selective input gradient regularization

Interpretable and Editable Programmatic Tree Policies for Reinforcement Learning

Learn Decision Trees with Deep Visual Primitives