Abstract:The rapid urbanization and increasing use of distributed renewable energy resources have imposed a significant burden on power networks. Smart buildings equipped with artificial intelligence technology can play a pivotal role in energy management, ultimately enhancing energy efficiency and voltage quality. However, ensuring voltage stability within large-scale smart building systems presents challenges due to the coexistence of diverse energy sources and the fluctuating nature of renewable energy. This paper proposes a safe multi-energy management framework achieved by online decentralized execution and centralized training for large scale smart buildings in distribution networks. The energy management problem is formulated as a safety-augmented Markov decision process, presenting intractability for dynamic programming due to its extensive continuous state space. To solve this issue and improve the convergence speed and training process stability, a safety-augmented constrained multi-agent reinforcement learning algorithm based on reward extrapolation is proposed. In this algorithm, hazard values are introduced to enhance non-safe multi-agent reinforcement learning algorithms and meet safety constraints. A novel reward network is designed by imitating expert underlying intentions to ensure the rationality of the reward function for multi-objective tasks. Additionally, the loss function for estimating the Q-network is redesigned during training process to guarantee effective convergence. Theoretical analysis is conducted to provide the convergence guarantee. Numerical case studies based on actual data are performed to validate the effectiveness and scalability of our approach, showing that smart buildings can achieve superior energy management performance while ensuring voltage safety for distribution networks. The source code of the proposed algorithm will be available at https://github.com/SYiyun/CMARL-EX.

Energy Management of Intelligent Building Based on Deep Reinforced Learning

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

A Novel Two-stage Reinforcement Learning Framework for Sustainable Building Energy Management Systems

Operational Optimization for Off-Grid Renewable Building Energy System Using Deep Reinforcement Learning

Autonomous Household Energy Management Using Deep Reinforcement Learning

Model-Free Real-Time Autonomous Energy Management for a Residential Multi-Carrier Energy System: A Deep Reinforcement Learning Approach

Energy Management Based on Safe Multi-Agent Reinforcement Learning for Smart Buildings in Distribution Networks

Deep reinforcement learning based home energy management system with devices operational dependencies

Deep Reinforcement Learning for Real-Time Energy Management in Smart Home

Coordinated energy management for integrated energy system incorporating multiple flexibility measures of supply and demand sides: A deep reinforcement learning approach

Deep Reinforcement Learning for Smart Building Energy Management: A Survey.

Optimal Energy Management Strategies for Energy Internet Via Deep Reinforcement Learning Approach

On-line Building Energy Optimization using Deep Reinforcement Learning

Optimization strategy based on deep reinforcement learning for home energy management

Deep Reinforcement Learning for Smart Home Energy Management

A Review of Deep Reinforcement Learning for Smart Building Energy Management

A Real-time Demand Response Strategy of Home Energy Management by Using Distributed Deep Reinforcement Learning.

A Multi-Agent Deep Constrained Q-Learning Method for Smart Building Energy Management Under Uncertainties

Mixed Deep Reinforcement Learning Considering Discrete-continuous Hybrid Action Space for Smart Home Energy Management

Distributed flexible resource regulation strategy for residential communities based on deep reinforcement learning

Exploring Deep Reinforcement Learning for Holistic Smart Building Control