Abstract:Black-box artificial intelligence (AI) induction methods such as deep reinforcement learning (DRL) are increasingly being used to find optimal policies for a given control task. Although policies represented using a black-box AI are capable of efficiently executing the underlying control task and achieving optimal closed-loop performance-controlling the agent from the initial time step until the successful termination of an episode, the developed control rules are often complex and neither interpretable nor explainable. In this article, we use a recently proposed nonlinear decision-tree (NLDT) approach to find a hierarchical set of control rules in an attempt to maximize the open-loop performance for approximating and explaining the pretrained black-box DRL (oracle) agent using the labeled state-action dataset. Recent advances in nonlinear optimization approaches using evolutionary computation facilitate finding a hierarchical set of nonlinear control rules as a function of state variables using a computationally fast bilevel optimization procedure at each node of the proposed NLDT. In addition, we propose a reoptimization procedure for enhancing the closed-loop performance of an already derived NLDT. We evaluate our proposed methodologies (open- and closed-loop NLDTs) on different control problems having multiple discrete actions. In all these problems, our proposed approach is able to find relatively simple and interpretable rules involving one to four nonlinear terms per rule, while simultaneously achieving on par closed-loop performance when compared to a trained black-box DRL agent. A postprocessing approach for simplifying the NLDT is also suggested. The obtained results are inspiring as they suggest the replacement of complicated black-box DRL policies involving thousands of parameters (making them noninterpretable) with relatively simple interpretable policies. The results are encouraging and motivating to pursue further applications of proposed approach in solving more complex control tasks.

Interpretable policy derivation for reinforcement learning based on evolutionary feature synthesis

Generalize Robot Learning from Demonstration to Variant Scenarios with Evolutionary Policy Gradient

Learning to Synthesize Programs as Interpretable and Generalizable Policies

Interpretable Policies for Reinforcement Learning by Genetic Programming

Toward Interpretable-AI Policies Using Evolutionary Nonlinear Decision Trees for Discrete-Action Systems

Towards Interpretable-AI Policies Induction using Evolutionary Nonlinear Decision Trees for Discrete Action Systems

Human-Readable Programs as Actors of Reinforcement Learning Agents Using Critic-Moderated Evolution

Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learning

Neural-to-Tree Policy Distillation with Policy Improvement Criterion

Distilling Reinforcement Learning Policies for Interpretable Robot Locomotion: Gradient Boosting Machines and Symbolic Regression

Interpretable and Editable Programmatic Tree Policies for Reinforcement Learning

Three Pathways to Neurosymbolic Reinforcement Learning with Interpretable Model and Policy Networks

Learning Symbolic Rules for Interpretable Deep Reinforcement Learning

Solving Deep Reinforcement Learning Tasks with Evolution Strategies and Linear Policy Networks

Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learning

Achieving efficient interpretability of reinforcement learning via policy distillation and selective input gradient regularization

Effective Interpretable Policy Distillation via Critical Experience Point Identification

Synthesizing Programmatic Policy for Generalization Within Task Domain

ERL-Re$^2$: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representation

Distilling Deep RL Models Into Interpretable Neuro-Fuzzy Systems

Evolution-Guided Policy Gradient in Reinforcement Learning