Abstract:The rapid urbanization and increasing use of distributed renewable energy resources have imposed a significant burden on power networks. Smart buildings equipped with artificial intelligence technology can play a pivotal role in energy management, ultimately enhancing energy efficiency and voltage quality. However, ensuring voltage stability within large-scale smart building systems presents challenges due to the coexistence of diverse energy sources and the fluctuating nature of renewable energy. This paper proposes a safe multi-energy management framework achieved by online decentralized execution and centralized training for large scale smart buildings in distribution networks. The energy management problem is formulated as a safety-augmented Markov decision process, presenting intractability for dynamic programming due to its extensive continuous state space. To solve this issue and improve the convergence speed and training process stability, a safety-augmented constrained multi-agent reinforcement learning algorithm based on reward extrapolation is proposed. In this algorithm, hazard values are introduced to enhance non-safe multi-agent reinforcement learning algorithms and meet safety constraints. A novel reward network is designed by imitating expert underlying intentions to ensure the rationality of the reward function for multi-objective tasks. Additionally, the loss function for estimating the Q-network is redesigned during training process to guarantee effective convergence. Theoretical analysis is conducted to provide the convergence guarantee. Numerical case studies based on actual data are performed to validate the effectiveness and scalability of our approach, showing that smart buildings can achieve superior energy management performance while ensuring voltage safety for distribution networks. The source code of the proposed algorithm will be available at https://github.com/SYiyun/CMARL-EX.

A Community Energy Management Method Based on Explainable Reinforcement Learning

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

MF^2: Model-free Reinforcement Learning for Modeling-Free Building HVAC Control with Data-Driven Environment Construction in a Residential Building

Heating ventilation air‐conditioner system for multi‐regional commercial buildings based on deep reinforcement learning

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

Modelling building HVAC control strategies using a deep reinforcement learning approach

A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control

Energy Optimization for HVAC Systems in Multi-VAV Open Offices: A Deep Reinforcement Learning Approach

Toward Smart Multizone HVAC Control by Combining Context-Aware System and Deep Reinforcement Learning.

Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning

Deep Reinforcement Learning for Smart Home Energy Management

Energy Management Based on Safe Multi-Agent Reinforcement Learning for Smart Buildings in Distribution Networks

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings

Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control

Enhancing the Performance of Multi-Agent Reinforcement Learning for Controlling HVAC Systems

Occupant-centric HVAC and window control: A reinforcement learning model for enhancing indoor thermal comfort and energy efficiency

MBRL-MC: An HVAC Control Approach via Combining Model-based Deep Reinforcement Learning and Model Predictive Control