Abstract:Design time uncertainty poses an important challenge when developing a self-adaptive system. As an example, defining how the system should adapt when facing a new environment state, requires understanding the precise effect of an adaptation, which may not be known at design time. Online reinforcement learning, i.e., employing reinforcement learning (RL) at runtime, is an emerging approach to realizing self-adaptive systems in the presence of design time uncertainty. By using Online RL, the self-adaptive system can learn from actual operational data and leverage feedback only available at runtime. Recently, Deep RL is gaining interest. Deep RL represents learned knowledge as a neural network whereby it can generalize over unseen inputs, as well as handle continuous environment states and adaptation actions. A fundamental problem of Deep RL is that learned knowledge is not explicitly represented. For a human, it is practically impossible to relate the parametrization of the neural network to concrete RL decisions and thus Deep RL essentially appears as a black box. Yet, understanding the decisions made by Deep RL is key to (1) increasing trust, and (2) facilitating debugging. Such debugging is especially relevant for self-adaptive systems, because the reward function, which quantifies the feedback to the RL algorithm, must be defined by developers. The reward function must be explicitly defined by developers, thus introducing a potential for human error. To explain Deep RL for self-adaptive systems, we enhance and combine two existing explainable RL techniques from the machine learning literature. The combined technique, XRL-DINE, overcomes the respective limitations of the individual techniques. We present a proof-of-concept implementation of XRL-DINE, as well as qualitative and quantitative results of applying XRL-DINE to a self-adaptive system exemplar.

Resilience-based explainable reinforcement learning in chemical process safety

A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges

Hierarchical Framework for Interpretable and Probabilistic Model-Based Safe Reinforcement Learning

Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Intervention

Distill2Explain: Differentiable decision trees for explainable reinforcement learning in energy application controllers

Explainable Reinforcement Learning via Model Transforms

Safe reinforcement learning under temporal logic with reward design and quantum action selection

Causal State Distillation for Explainable Reinforcement Learning

Explainable AI in Deep Reinforcement Learning Models for Power System Emergency Control

SAFE-RL: Saliency-Aware Counterfactual Explainer for Deep Reinforcement Learning Policies

Explainable Reinforcement Learning: Basic Problems Exploration and Method Survey

A User Study on Explainable Online Reinforcement Learning for Adaptive Systems

Demystifying Reinforcement Learning in Production Scheduling via Explainable AI

Explainable and Safe Reinforcement Learning for Autonomous Air Mobility

BET: Explaining Deep Reinforcement Learning through The Error-Prone Decisions

REACT: Revealing Evolutionary Action Consequence Trajectories for Interpretable Reinforcement Learning

Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systems

ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning

Experiential Explanations for Reinforcement Learning

Think Smart, Act SMARL! Analyzing Probabilistic Logic Driven Safety in Multi-Agent Reinforcement Learning