Abstract:Aiming at a cleaner future power system, many regimes in the world have proposed their ambitious decarbonizing plan, with increasing penetration of renewable energy sources (RES) playing an alternative role to conventional energy. As a result, power system tends to have less backup capacity and operate near their designed limit, thus exacerbating system vulnerability against extreme events. Under this reality, resilient control for the multi-energy micro-grid is facing the following challenges, which are: 1) the effect from the stochastic uncertainties of RES; 2) the need for a model-free and fast-reacting control scheme under extreme events; and 3) efficient exploration and robust performance with limited extreme events data. To deal with the aforementioned challenges, this paper proposes a novel Bayesian Deep Reinforcement Learning-based resilient control approach for multi-energy micro-grid. In particular, the proposed approach replaces the deterministic network in traditional Reinforcement Learning with a Bayesian probabilistic network in order to obtain an approximation of the value function distribution, which effectively solves the Q-value overestimation issue. Compared with the naive Deep Deterministic Policy Gradient (DDPG) method and optimization method, the effectiveness and importance of employing the Bayesian Reinforcement Learning approach are investigated and illustrated across different operating scenarios. Case studies have shown that by using the Monte Carlo posterior mean of the Bayesian value function distribution instead of a deterministic estimation, the proposed Bayesian Deep Deterministic Policy Gradient (BDDPG) method achieves a near-optimum policy in a more stable process, which verifies the robustness and the practicability of the proposed approach.

Chance Constrained MDP Formulation and Bayesian Advantage Policy Optimization for Stochastic Dynamic Optimal Power Flow

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

Chance Constrained Nonlinear Model Predictive Control

Chance Constrained Optimal Power Flow: Risk-Aware Network Control under Uncertainty

Safe deep reinforcement learning for real-time AC optimal power flow: A near-optimal solution

A Safe DRL Method for Fast Solution of Real-Time Optimal Power Flow

Synchronization-Aware and Algorithm-Efficient Chance Constrained Optimal Power Flow

Alternating Direction Method of Multipliers for Solving Joint Chance Constrained Optimal Power Flow under Uncertainties

Robust preventive and corrective security-constrained OPF for worst contingencies with the adoption of VPP: A safe reinforcement learning approach

Constrained Reinforcement Learning for Predictive Control in Real-Time Stochastic Dynamic Optimal Power Flow

A Hybrid Dynamic Optimization Approach for Stability Constrained Optimal Power Flow

Multi-agent VPP Coordinated Control Optimization and Risk Analysis Based on the Interactive Algorithm

Data-driven Decision Making with Probabilistic Guarantees (Part 2): Applications of Chance-constrained Optimization in Power Systems

Chance-constrained optimal dispatch of integrated energy systems based on data-driven sparse polynomial chaos expansion

Deep Sigma Point Processes-Assisted Chance-Constrained Power System Transient Stability Preventive Control

Unifying Chance-Constrained and Robust Optimal Power Flow for Resilient Network Operations

Tractable Convex Approximations for Distributionally Robust Joint Chance-Constrained Optimal Power Flow Under Uncertainty

Chance-Constrained Optimization-Based Unbalanced Optimal Power Flow for Radial Distribution Networks

Optimal Load Ensemble Control in Chance-Constrained Optimal Power Flow

Stochastic Optimal Power Flow Based on Data-Driven Distributionally Robust Optimization

Solving Mission-Wide Chance-Constrained Optimal Control Using Dynamic Programming