Abstract:The development of distributed renewable energy resources and smart energy management are efficient approaches to decarbonizing building energy systems. Reinforcement learning (RL) is a data-driven control algorithm that trains a large amount of data to learn control policy. However, this learning process generally presents low learning efficiency using real-world stochastic data. To address this challenge, this study proposes a model-based RL approach to optimize the operation of existing zero-energy houses considering PV generation consumption and energy costs. The model-based approach takes advantage of the inner understanding of the system dynamics; this knowledge improves the learning efficiency. A reward function is designed considering the physical constraints of battery storage, photovoltaic (PV) production feed-in profit, and energy cost. Measured data of a zero-energy house are used to train and test the proposed RL agent control, including Q-learning, deep Q network (DQN), and deep deterministic policy gradient (DDPG) agents. The results show that the proposed RL agents can achieve fast convergence during the training process. In comparison with the rule-based strategy, test cases verify the cost-effectiveness performances of proposed RL approaches in scheduling operations of the hybrid energy system under different scenarios. The comparative analysis of test periods shows that the DQN agent presents better energy cost-saving performances than Q-learning while the Q-learning agent presents more flexible action control of the battery with the fluctuation of real-time electricity prices. The DDPG algorithm can achieve the highest PV self-consumption ratio, 49.4%, and the self-sufficiency ratio reaches 36.7%. The DDPG algorithm outperforms rule-based operation by 7.2% for energy cost during test periods.

A Hybrid Approach for Home Energy Management with Imitation Learning and Online Optimization

Optimal residential load scheduling in smart grid: A comprehensive approach

Interior-point policy optimization based multi-agent deep reinforcement learning method for secure home energy management under various uncertainties

Online Optimal Power Scheduling of a Microgrid via Imitation Learning

Optimization strategy based on deep reinforcement learning for home energy management

A data-driven DRL-based home energy management system optimization framework considering uncertain household parameters

Deep Reinforcement Learning for Real-Time Energy Management in Smart Home

Reinforcement Learning-Based Energy Management of Smart Home with Rooftop Solar Photovoltaic System, Energy Storage System, and Home Appliances

Real-Time Energy Management in Smart Homes Through Deep Reinforcement Learning

A hybrid deep learning-based online energy management scheme for industrial microgrid

Development and Evaluation of an Online Home Energy Management Strategy for Load Coordination in Smart Homes with Renewable Energy Sources

A Multi-Agent Deep Constrained Q-Learning Method for Smart Building Energy Management Under Uncertainties

A hybrid evolutionary and machine learning approach for smart building: Sustainable building energy management design

Deep reinforcement learning based home energy management system with devices operational dependencies

Two-timescale autonomous energy management strategy based on multi-agent deep reinforcement learning approach for residential multicarrier energy system

Comparative Analysis of Reinforcement Learning Approaches for Multi-Objective Optimization in Residential Hybrid Energy Systems

An Online Home Energy Management System Using Q-Learning and Deep Q-Learning

Economical Electric Vehicle Charging Scheduling Via Deep Imitation Learning

Demand side energy management for smart homes using a novel learning technique–economic analysis aspects

Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control