Abstract:The rapid urbanization and increasing use of distributed renewable energy resources have imposed a significant burden on power networks. Smart buildings equipped with artificial intelligence technology can play a pivotal role in energy management, ultimately enhancing energy efficiency and voltage quality. However, ensuring voltage stability within large-scale smart building systems presents challenges due to the coexistence of diverse energy sources and the fluctuating nature of renewable energy. This paper proposes a safe multi-energy management framework achieved by online decentralized execution and centralized training for large scale smart buildings in distribution networks. The energy management problem is formulated as a safety-augmented Markov decision process, presenting intractability for dynamic programming due to its extensive continuous state space. To solve this issue and improve the convergence speed and training process stability, a safety-augmented constrained multi-agent reinforcement learning algorithm based on reward extrapolation is proposed. In this algorithm, hazard values are introduced to enhance non-safe multi-agent reinforcement learning algorithms and meet safety constraints. A novel reward network is designed by imitating expert underlying intentions to ensure the rationality of the reward function for multi-objective tasks. Additionally, the loss function for estimating the Q-network is redesigned during training process to guarantee effective convergence. Theoretical analysis is conducted to provide the convergence guarantee. Numerical case studies based on actual data are performed to validate the effectiveness and scalability of our approach, showing that smart buildings can achieve superior energy management performance while ensuring voltage safety for distribution networks. The source code of the proposed algorithm will be available at https://github.com/SYiyun/CMARL-EX.

Reinforcement Learning with Enhanced Safety for Optimal Dispatch of Distributed Energy Resources in Active Distribution Networks

Multi-Agent Reinforcement Learning-based Distributed Economic Dispatch Considering Network attacks and Uncertain Costs

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

RL-ADN: A High-Performance Deep Reinforcement Learning Environment for Optimal Energy Storage Systems Dispatch in Active Distribution Networks

Energy Management Based on Safe Multi-Agent Reinforcement Learning for Smart Buildings in Distribution Networks

Safe Imitation Learning-based Optimal Energy Storage Systems Dispatch in Distribution Networks

Coordinated Dispatch of Energy Storage Systems in the Active Distribution Network: A Complementary Reinforcement Learning and Optimization Approach

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

RL2: Reinforce Large Language Model to Assist Safe Reinforcement Learning for Energy Management of Active Distribution Networks

Safety-Aware Reinforcement Learning for Electric Vehicle Charging Station Management in Distribution Network

Online Operational Decision-making for Integrated Electric-Gas Systems with Safe Reinforcement Learning

Contingency-constrained economic dispatch with safe reinforcement learning

A Critical Review of Safe Reinforcement Learning Techniques in Smart Grid Applications

Hierarchical deep reinforcement learning for self-adaptive economic dispatch

Distributed Multiagent Reinforcement Learning With Action Networks for Dynamic Economic Dispatch

Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices

Multiagent-Based Reinforcement Learning for Optimal Reactive Power Dispatch.

Optimal Bi-Level Bidding and Dispatching Strategy between Active Distribution Network and Virtual Alliances Using Distributed Robust Multi-Agent Deep Reinforcement Learning

Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks

Optimal dispatch of integrated energy system based on deep reinforcement learning