Abstract:In recent years, advanced control strategies based on Deep Reinforcement Learning (DRL) proved to be effective in optimizing the management of integrated energy systems in buildings, reducing energy costs and improving indoor comfort conditions when compared to traditional reactive controllers. However, the scalability and implementation of DRL controllers are still limited since they require a considerable amount of time before converging to a near-optimal solution. This issue is currently addressed in literature through the offline pre-training of the DRL agent. However this solution results in two main critical issues: (1) the need to develop a building surrogate model to perform the training task, and (2) the need to perform a fine-tuning process over several training episodes to obtain a near-optimal control policy. In this context, this paper introduces an Online Transfer Learning (OTL) strategy that exploits two knowledge-sharing techniques, weight-initialization and imitation learning, to transfer a DRL control policy from a source office building to various target buildings in a simulation environment coupling EnergyPlus and Python. A DRL controller based on discrete Soft Actor–Critic (SAC) is trained on the source building to manage the operation of a cooling system consisting of a chiller and a thermal storage. Several target buildings are defined to benchmark the performance of the OTL strategy with that of a Rule-Based Controller (RBC) and two DRL-based control strategies, deployed in offline and online fashion. The strategy adopted for OTL emulates the real world implementation with a simulation process by implementing the transferred DRL agent for a single episode in the target buildings. Target buildings have the same geometrical features and are served by the same energy system as the source building, but differ in terms of weather conditions, electricity price schedules, occupancy patterns, and building envelope efficiency levels. The results show that the OTL strategy can reduce the cumulated sum of temperature violations on average by 50% and 80% respectively when compared to RBC and online DRL while enhancing the energy system operation with electricity cost savings ranging between 20% and 40%. The OTL agent performs slightly worse than the offline DRL controller but it does not require any modeling effort and can be implemented directly on target buildings emulating a real-world implementation.

An improved transfer learning strategy for short-term cross-building energy prediction using data incremental

Transferability investigation of a Sim2Real deep transfer learning framework for cross-building energy prediction

Implementation of a Long Short-Term Memory Transfer Learning (LSTM-TL)-Based Data-Driven Model for Building Energy Demand Forecasting

A reinforcing transfer learning approach to predict buildings energy performance

Forecasting building energy demand and on-site power generation for residential buildings using long and short-term memory method with transfer learning

Performance assessment of cross office building energy prediction in the same region using the domain adversarial transfer learning strategy

Transfer learning and source domain restructuring-based BiLSTM approach for building energy consumption prediction

Cloud-edge collaboration based transferring prediction of building energy consumption

A multi-source transfer learning model based on LSTM and domain adaptation for building energy prediction

Deep learning and transfer learning models of energy consumption forecasting for a building with poor information data

A general multi-source ensemble transfer learning framework integrate of LSTM-DANN and similarity metric for building energy prediction

Transfer Learning in Deep Learning Models for Building Load Forecasting: Case of Limited Data

An innovative heterogeneous transfer learning framework to enhance the scalability of deep reinforcement learning controllers in buildings with integrated energy systems

Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings

A hybrid deep learning-based method for short-term building energy load prediction combined with an interpretation process

A Building Energy Consumption Prediction Method Based on Integration of a Deep Neural Network and Transfer Reinforcement Learning

Forecasting District-Scale Energy Dynamics Through Integrating Building Network and Long Short-Term Memory Learning Algorithm

Comparative study on deep transfer learning strategies for cross-system and cross-operation-condition building energy systems fault diagnosis

A novel deep generative modeling-based data augmentation strategy for improving short-term building energy predictions

Enhanced Multi-Horizon Occupancy Prediction in Smart Buildings using Cascaded Bi-LSTM Models with Integrated Features

CBLSTM-AE: A Hybrid Deep Learning Framework for Predicting Energy Consumption