Abstract:As power systems become more complex with the continuous integration of intelligent distributed energy resources (DERs), new risks and uncertainties arise. Consequently, to enhance system resiliency, it is essential to account for various uncertain events when implementing the optimization problem for the energy management system (EMS). This paper presents a preventive EMS considering the probability of failure (PoF) of each system component across different scenarios. A conditional-value-at-risk (CVaR)-based framework is proposed to integrate the uncertainties of the distribution network. Loads are classified into critical, semi-critical, and non-critical categories to prioritize essential loads during generation resource shortages. A proximal policy optimization (PPO)-based reinforcement learning (RL) agent is used to solve the formulated problem and generate the control decisions. The proposed framework is evaluated on a notional MVDC ship system and a modified IEEE 30-bus system, where the results demonstrate that the PPO agent can successfully optimize the objective function while maintaining the network and operational constraints. For validation, the RL-based method is benchmarked against a traditional optimization approach, further highlighting its effectiveness and robustness. This comparison shows that RL agents can offer more resiliency against future uncertain events compared to the traditional solution methods due to their adaptability and learning capacity.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in the distribution system, with the continuous integration of intelligent distributed energy resources (DERs), new risks and uncertainties keep emerging. In order to enhance the resilience of the system, it is necessary to consider various uncertain events when implementing the optimization problems of the energy management system (EMS). Specifically, the paper aims to preventively manage the energy of the distribution system by introducing a deep - reinforcement - learning - based method to deal with the impact of uncertain events. ### Main Problems and Objectives 1. **Enhancing System Resilience**: Considering that the power system is becoming more and more complex and is facing an increasing number of uncertain events (such as cyber - attacks, failures, and extreme weather conditions), it is necessary to improve the system's resistance to these events. 2. **Optimizing Energy Management**: A preventive EMS framework is proposed, which takes into account the probability of failure (PoF) of each system component and generates control decisions according to different scenarios. 3. **Dealing with Uncertainty and Non - linearity**: Traditional optimization methods have high computational complexity when dealing with uncertainty and non - linearity in complex power systems, and are unable to adapt to online solutions or learn from new conditions. Therefore, the paper adopts the deep - reinforcement - learning (DRL) algorithm to solve these problems. ### Specific Measures - **Introducing the CVaR Framework**: Use the conditional value - at - risk (CVaR) framework to incorporate the impact of uncertain events into the optimization problem, so as to more effectively deal with high - impact and low - probability events. - **Classifying Load Priorities**: Divide loads into three categories: critical, semi - critical, and non - critical, to ensure that important loads are given priority when resources are scarce. - **Using the PPO Algorithm**: Adopt the proximal policy optimization (PPO) algorithm to train the reinforcement - learning agent to generate control decisions while ensuring that network and operational constraints are met. ### Experimental Verification The paper conducted experiments on the hypothesized MVDC ship system and the modified IEEE 30 - node system. The results show that the PPO agent can successfully optimize the objective function and exhibit higher resilience and adaptability in the face of future uncertain events. ### Summary The main contribution of the paper is to propose a new preventive EMS framework. This framework not only takes into account future uncertain events, but also significantly reduces the computational complexity through the CVaR framework and the PPO algorithm, and improves the system's resilience and real - time response ability.

Preventive Energy Management for Distribution Systems Under Uncertain Events: A Deep Reinforcement Learning Approach

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

Hybrid Multi-Agent Reinforcement Learning for Electric Vehicle Resilience Control Towards a Low-Carbon Transition

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

Optimal Electric Vehicle Charging Strategy with Markov Decision Process and Reinforcement Learning Technique

Proximal policy optimization through a deep reinforcement learning framework for remedial action schemes of VSC-HVDC

DDPG-based Reactive Power Optimization Strategy for Active Distribution Network

Robust Deep Reinforcement Learning for Volt-VAR Optimization in Active Distribution System under Uncertainty

Safe Reinforcement Learning for Emergency LoadShedding of Power Systems

Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control

Optimal Planning of Hybrid Energy Storage Systems using Curtailed Renewable Energy through Deep Reinforcement Learning

Multi-agent Deep Reinforcement Learning for Resilience-Driven Routing and Scheduling of Mobile Energy Storage Systems

Deep Reinforcement Learning Based Optimal Energy Management of Multi-energy Microgrids with Uncertainties

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Attention and masking embedded ensemble reinforcement learning for smart energy optimization and risk evaluation under uncertainties

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

Safe Imitation Learning-based Optimal Energy Storage Systems Dispatch in Distribution Networks

Adaptive Power System Emergency Control using Deep Reinforcement Learning

Deep reinforcement learning approach to estimate the energy-mix proportion for secure operation of converter-dominated power system

Safety-Aware Reinforcement Learning for Electric Vehicle Charging Station Management in Distribution Network

Interior-point policy optimization based multi-agent deep reinforcement learning method for secure home energy management under various uncertainties