Abstract:Deep reinforcement learning (DRL) has demonstrated significant potential in industrial manufacturing domains such as workshop scheduling and energy system management. However, due to the model's inherent uncertainty, rigorous validation is requisite for its application in real-world tasks. Specific tests may reveal inadequacies in the performance of pre-trained DRL models, while the "black-box" nature of DRL poses a challenge for testing model behavior. We propose a novel performance improvement framework based on probabilistic automata, which aims to proactively identify and correct critical vulnerabilities of DRL systems, so that the performance of DRL models in real tasks can be improved with minimal model modifications. First, a probabilistic automaton is constructed from the historical trajectory of the DRL system by abstracting the state to generate probabilistic decision-making units (PDMUs), and a reverse breadth-first search (BFS) method is used to identify the key PDMU-action pairs that have the greatest impact on adverse outcomes. This process relies only on the state-action sequence and final result of each trajectory. Then, under the key PDMU, we search for the new action that has the greatest impact on favorable results. Finally, the key PDMU, undesirable action and new action are encapsulated as monitors to guide the DRL system to obtain more favorable results through real-time monitoring and correction mechanisms. Evaluations in two standard reinforcement learning environments and three actual job scheduling scenarios confirmed the effectiveness of the method, providing certain guarantees for the deployment of DRL models in real-world applications.

Data-driven active corrective control in power systems: an interpretable deep reinforcement learning approach

Deep Reinforcement Learning Based Approach for Active Power Security Correction Control of Power System

Exploring the Vulnerability of Deep Reinforcement Learning-based Emergency Control for Low Carbon Power Systems

Deep Reinforcement Learning for Power System Applications: An Overview

Adaptive Power System Emergency Control using Deep Reinforcement Learning

Explainable AI in Deep Reinforcement Learning Models for Power System Emergency Control

Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control

Implementing Deep Reinforcement Learning-Based Grid Voltage Control in Real-World Power Systems: Challenges and Insights

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

A multi-hierarchical interpretable method for DRL-based dispatching control in power systems

Review of Deep Reinforcement Learning and Its Application in Modern Renewable Power System Control

Application and Development of Reinforcement Learning Theory in Power Systems

A prior knowledge-embedded reinforcement learning method for real-time active power corrective control in complex power systems

Active power correction strategies based on deep reinforcement learning-part I: A simulation-driven solution for robustness

Probabilistic Automata-Based Method for Enhancing Performance of Deep Reinforcement Learning Systems

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Safe Reinforcement Learning for Power System Control: A Review

A Deep Reinforcement Learning Approach to Improve the Learning Performance in Process Control

A Deep Reinforcement Learning Framework for Automatic Operation Control of Power System Considering Extreme Weather Events

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

Fault Identification in Power Network Based on Deep Reinforcement Learning