Abstract:Commercial office buildings contribute 17 percent of Carbon Emissions in the US, according to the US Energy Information Administration (EIA), and improving their efficiency will reduce their environmental burden and operating cost. A major contributor of energy consumption in these buildings are the Heating, Ventilation, and Air Conditioning (HVAC) devices. HVAC devices form a complex and interconnected thermodynamic system with the building and outside weather conditions, and current setpoint control policies are not fully optimized for minimizing energy use and carbon emission. Given a suitable training environment, a Reinforcement Learning (RL) agent is able to improve upon these policies, but training such a model, especially in a way that scales to thousands of buildings, presents many practical challenges. Most existing work on applying RL to this important task either makes use of proprietary data, or focuses on expensive and proprietary simulations that may not be grounded in the real world. We present the Smart Buildings Control Suite, the first open source interactive HVAC control dataset extracted from live sensor measurements of devices in real office buildings. The dataset consists of two components: six years of real-world historical data from three buildings, for offline RL, and a lightweight interactive simulator for each of these buildings, calibrated using the historical data, for online and model-based RL. For ease of use, our RL environments are all compatible with the OpenAI gym environment standard. We also demonstrate a novel method of calibrating the simulator, as well as baseline results on training an RL agent on the simulator, predicting real-world data, and training an RL agent directly from data. We believe this benchmark will accelerate progress and collaboration on building optimization and environmental sustainability research.

Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings

Multi-agent reinforcement learning dealing with hybrid action spaces: A case study for off-grid oriented renewable building energy system

Grid-Interactive Multi-Zone Building Control Using Reinforcement Learning with Global-Local Policy Search

Reinforcement learning building control approach harnessing imitation learning

Two-Stage Reinforcement Learning Policy Search for Grid-Interactive Building Control

GridLearn: Multiagent reinforcement learning for grid-aware building energy management

CityLearn: Standardizing Research in Multi-Agent Reinforcement Learning for Demand Response and Urban Energy Management

Active Reinforcement Learning for Robust Building Control

Multi-agent Deep Reinforcement Learning-Based Coordination Control for Grid-Aware Multi-Buildings

Applications in CityLearn Gym Environment for Multi-Objective Control Benchmarking in Grid-Interactive Buildings and Districts

MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearn

Operational Optimization for Off-Grid Renewable Building Energy System Using Deep Reinforcement Learning

Challenges of real-world reinforcement learning: definitions, benchmarks and analysis

Real-World Implementation of Reinforcement Learning Based Energy Coordination for a Cluster of Households

Real-World Data and Calibrated Simulation Suite for Offline Training of Reinforcement Learning Agents to Optimize Energy and Emission in Buildings for Environmental Sustainability

Optimal Management of Grid-Interactive Efficient Buildings via Safe Reinforcement Learning

Comparative analysis of grid-interactive building control algorithms: From model-based to learning-based approaches

Experimental evaluation of offline reinforcement learning for HVAC control in buildings

Flexible Power Load Control Based on Hierarchical Multi-Agent Reinforcement Learning

On Training Effective Reinforcement Learning Agents for Real-time Power Grid Operation and Control

Simulation Studies on Deep Reinforcement Learning for Building Control with Human Interaction