Abstract:A number of electric devices in buildings can be considered as important demand response (DR) resources, for instance, the battery energy storage system (BESS) and the heat, ventilation, and air conditioning (HVAC) systems. The conventional model-based DR methods rely on efficient on-demand computing resources. However, the current buildings suffer from the high cost of computing resources and lack a cost-effective automation system, which becomes the main obstacle to the popularization and implementation of the DR program. Therefore, in this paper, we present a hybrid cloud and edge control strategy for BESS and HVAC based on deep reinforcement learning (DRL). On the cloud infrastructure, the agent learns the control strategy online based on the proposed continuous dueling deep Q-learning (C-DDQN) algorithm, and the learned strategy is distributed to the edge devices for execution. Under this framework, the data-intensive application of cloud computing in real-time DR shows advantages in high processing speed, unlimited data aggregation, fault-tolerant, cost-saving, security, and confidentiality. However, if every controller is trained from the beginning, the cloud resources are wasted to a large extent. Therefore, we propose a transfer deep reinforcement learning methodology to transfer the control strategies between BESS and HVAC units. The transfer learning is realized based on fine-tuning and the proposed Evolving Domain Adaptation Network (EDAN). In case studies, it is verified that the proposed transfer deep reinforcement learning algorithm shows better convergence and learning capability compared with not applying transfer learning technologies. Compared with the conventional model-based method, the proposed methodology speeds up the decision-making time by 105 times.

DPBRE: Demand Prediction and Bidding Responsiveness Enhancement in Smart Grids with Edge and Cloud Computing

A Multi-Timescale Allocation Algorithm of Energy and Power for Demand Response in Smart Grids: A Stackelberg Game Approach

Non-Cooperative Game in Block Bidding Markets Considering Demand Response

A Game Theory-Based Interactive Demand Response for Handling Dynamic Prices in Security-Constrained Electricity Markets

A New Customer-Side Demand Response Strategy in Block Bidding Markets

Edge-End Based Demand Response System of Aggregated Residential Load: Case Study

Data-driven online interactive bidding strategy for demand response

Fast Distributed Demand Response with Spatially- and Temporally-Coupled Constraints in Smart Grid

Price-Based Residential Demand Response Management in Smart Grids: A Reinforcement Learning-Based Approach

Evolutionary Game Based Demand Response Bidding Strategy for End-Users Using Q-Learning and Compound Differential Evolution

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

Demand Response Optimization Strategy of Data Center Considering P2P Transaction

5G Ultradense Cellular-Network-Based Edge Demand Response: Energy Consumption Reduction

Dynamic pricing for load shifting: Reducing electric vehicle charging impacts on the grid through machine learning-based demand response

Online Edge Computing Demand Response Via Deadline-Aware V2G Discharging Auctions

Emergency Demand Response in Edge Computing

When Smart Grid Meets Geo-Distributed Cloud: An Auction Approach To Datacenter Demand Response

An Online Market Mechanism for Edge Emergency Demand Response Via Cloudlet Control

A Hybrid Cloud and Edge Control Strategy for Demand Responses Using Deep Reinforcement Learning and Transfer Learning

Optimal Residential Demand Response in Distribution Networks

Operation of Distributed Battery Considering Demand Response Using Deep Reinforcement Learning in Grid Edge Control