Abstract:The rapid urbanization and increasing use of distributed renewable energy resources have imposed a significant burden on power networks. Smart buildings equipped with artificial intelligence technology can play a pivotal role in energy management, ultimately enhancing energy efficiency and voltage quality. However, ensuring voltage stability within large-scale smart building systems presents challenges due to the coexistence of diverse energy sources and the fluctuating nature of renewable energy. This paper proposes a safe multi-energy management framework achieved by online decentralized execution and centralized training for large scale smart buildings in distribution networks. The energy management problem is formulated as a safety-augmented Markov decision process, presenting intractability for dynamic programming due to its extensive continuous state space. To solve this issue and improve the convergence speed and training process stability, a safety-augmented constrained multi-agent reinforcement learning algorithm based on reward extrapolation is proposed. In this algorithm, hazard values are introduced to enhance non-safe multi-agent reinforcement learning algorithms and meet safety constraints. A novel reward network is designed by imitating expert underlying intentions to ensure the rationality of the reward function for multi-objective tasks. Additionally, the loss function for estimating the Q-network is redesigned during training process to guarantee effective convergence. Theoretical analysis is conducted to provide the convergence guarantee. Numerical case studies based on actual data are performed to validate the effectiveness and scalability of our approach, showing that smart buildings can achieve superior energy management performance while ensuring voltage safety for distribution networks. The source code of the proposed algorithm will be available at https://github.com/SYiyun/CMARL-EX.

Multi-agent hierarchical reinforcement learning for energy management

Multi-agent reinforcement learning dealing with hybrid action spaces: A case study for off-grid oriented renewable building energy system

Multi-Agent Reinforcement Learning for Power Grid Topology Optimization

Multi-Agent Reinforcement Learning for Power System Operation and Control

Model-Free Real-Time Autonomous Energy Management for a Residential Multi-Carrier Energy System: A Deep Reinforcement Learning Approach

Energy Management Based on Safe Multi-Agent Reinforcement Learning for Smart Buildings in Distribution Networks

Multi-Agent Reinforcement Learning for Energy Networks: Computational Challenges, Progress and Open Problems

Flexible Power Load Control Based on Hierarchical Multi-Agent Reinforcement Learning

Multi-agent deep reinforcement learning based distributed control architecture for interconnected multi-energy microgrid energy management and optimization

Multi-agent deep reinforcement learning-based optimal energy management for grid-connected multiple energy carrier microgrids

A Multi-Agent Deep Reinforcement Learning Approach Enabled Distributed Energy Management Schedule for the Coordinate Control of Multi-Energy Hub with Gas, Electricity, and Freshwater

Towards Pareto-optimal energy management in integrated energy systems: A multi-agent and multi-objective deep reinforcement learning approach

Multi-agent Deep Reinforcement Learning for Resilience-Driven Routing and Scheduling of Mobile Energy Storage Systems

Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

Hierarchical Reinforcement Learning for Power Network Topology Control

Multi-energy Management of Interconnected Multi-microgrid System Using Multi-agent Deep Reinforcement Learning

Distributed Power Control for Large Energy Harvesting Networks: A Multi-Agent Deep Reinforcement Learning Approach

Multiagent Reinforcement Learning with Learning Automata for Microgrid Energy Management and Decision Optimization

Multistep Multiagent Reinforcement Learning for Optimal Energy Schedule Strategy of Charging Stations in Smart Grid.

Continuous Multiagent Control Using Collective Behavior Entropy for Large-Scale Home Energy Management

Reinforcement Learning for Efficient Design and Control Co-optimisation of Energy Systems