Abstract:The rapid urbanization and increasing use of distributed renewable energy resources have imposed a significant burden on power networks. Smart buildings equipped with artificial intelligence technology can play a pivotal role in energy management, ultimately enhancing energy efficiency and voltage quality. However, ensuring voltage stability within large-scale smart building systems presents challenges due to the coexistence of diverse energy sources and the fluctuating nature of renewable energy. This paper proposes a safe multi-energy management framework achieved by online decentralized execution and centralized training for large scale smart buildings in distribution networks. The energy management problem is formulated as a safety-augmented Markov decision process, presenting intractability for dynamic programming due to its extensive continuous state space. To solve this issue and improve the convergence speed and training process stability, a safety-augmented constrained multi-agent reinforcement learning algorithm based on reward extrapolation is proposed. In this algorithm, hazard values are introduced to enhance non-safe multi-agent reinforcement learning algorithms and meet safety constraints. A novel reward network is designed by imitating expert underlying intentions to ensure the rationality of the reward function for multi-objective tasks. Additionally, the loss function for estimating the Q-network is redesigned during training process to guarantee effective convergence. Theoretical analysis is conducted to provide the convergence guarantee. Numerical case studies based on actual data are performed to validate the effectiveness and scalability of our approach, showing that smart buildings can achieve superior energy management performance while ensuring voltage safety for distribution networks. The source code of the proposed algorithm will be available at https://github.com/SYiyun/CMARL-EX.

Optimal Energy Management of Energy Internet: A Distributed Actor-Critic Reinforcement Learning Method

Optimal Energy Management Strategies for Energy Internet Via Deep Reinforcement Learning Approach

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Energy Management Based on Safe Multi-Agent Reinforcement Learning for Smart Buildings in Distribution Networks

Soft actor-critic –based multi-objective optimized energy conversion and management strategy for integrated energy systems with renewable energy

Optimal Energy Operation Strategy for We-Energy of Energy Internet Based on Hybrid Reinforcement Learning With Human-in-the-Loop

An integrated approach for multi-objective optimisation and MCDM of energy internet under uncertainty

Energy Management for Energy Internet: A Combination of Game Theory and Big Data-Based Renewable Power Forecasting

Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

Online and Energy-Efficient Task-Processing for Distributed Edge Networks

Optimal dispatch of integrated energy system based on deep reinforcement learning

Distributed Consensus Based and Network Economic Control of Energy Internet Management

Deep Reinforcement Learning for Scenario-Based Robust Economic Dispatch Strategy in Internet of Energy

Distributed Reactive Power Optimization for Energy Internet via Multiagent Deep Reinforcement Learning With Graph Attention Networks

Distributed Deep Reinforcement Learning for Renewable Energy Accommodation Assessment With Communication Uncertainty in Internet of Energy

Distributed flexible resource regulation strategy for residential communities based on deep reinforcement learning

Stochastic Optimal Control for Energy Internet: A Bottom-Up Energy Management Approach

Distributed Management of Fluctuating Energy Resources in Dynamic Networked Systems

Multi-agent deep reinforcement learning based distributed control architecture for interconnected multi-energy microgrid energy management and optimization

An improved Soft Actor-Critic strategy for optimal energy management