Abstract:With the development of energy Internet and intelligent buildings, the interactions of supply and demand sides of integrated energy system (IES) offer an attractive route for flexible energy management in buildings. However, traditional model-based control methods over-rely on precise mathematical modeling, difficult to flexibly deal with complex and changeable operating environments of IES. Therefore, this work proposes a coordinated operation optimization framework based on the deep reinforcement learning (DRL) algorithm for optimal scheduling of IES. Firstly, two supply-side and three demand-side flexibility measures are considered to tap the potential of flexible scheduling, including active adjustment of energy conversion equipment, energy storage, incentive-based demand response, electric vehicles and thermal inertia of building. Secondly, the coordinated optimization is formulated as a partially-observable Markov decision process. The twin delayed deep deterministic policy (TD3) algorithm is employed to solve the optimal energy management problem of IES, aiming at operation cost and user satisfaction. Thirdly, a hierarchical reward shaping (HRS) mechanism is proposed to improve the training performance of DRL, which could evaluate the current and final performance of agent and return the underway reward at each step and final reward. The developed optimization methodology is used for a case study in the building. The results show that the proposed HRS-TD3 algorithm achieves the fastest convergence and has the best economic performance compared with the other baseline algorithms. The operation cost of coordinated optimization is superior to those of the three baseline scenarios and achieves an improvement of 33.1%, 3.5% and 29.8%, respectively.

Energy Management of Integrated Energy System Based on Stackelberg Game and Deep Reinforcement Learning

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Equilibrium Interaction Strategies for Integrated Energy System Incorporating Demand-Side Management Based on Stackelberg Game Approach

A Real-time Demand Response Strategy of Home Energy Management by Using Distributed Deep Reinforcement Learning.

A Real-Time Pricing Scheme for Energy Management in Integrated Energy Systems: A Stackelberg Game Approach

Pricing strategy of regional integrated energy system considering privacy protection based on deep reinforcement learning

Energy Pricing and Management for the Integrated Energy Service Provider: A Stochastic Stackelberg Game Approach

Coordinated energy management for integrated energy system incorporating multiple flexibility measures of supply and demand sides: A deep reinforcement learning approach

A Hybrid Data-Driven Method for Low-Carbon Economic Energy Management Strategy in Electricity-Gas Coupled Energy Systems Based on Transformer Network and Deep Reinforcement Learning

Operation of a commercial district integrated energy system considering dynamic integrated demand response: A Stackelberg game approach

Distributed Collaborative Operation Strategies in Multi-Agent Integrated Energy System Considering Integrated Demand Response Based on Game Theory

Low-carbon Economic Dispatch of Electricity-Heat-Gas Integrated Energy Systems Based on Deep Reinforcement Learning

Optimal Operation of Integrated Energy System Based on Deep Reinforcement Learning

A Stackelberg game-based approach to transaction optimization for distributed integrated energy system

Blockchain-Assisted Secure Energy Trading in Electricity Markets: A Tiny Deep Reinforcement Learning-Based Stackelberg Game Approach

Dynamic Energy Dispatch Strategy for Integrated Energy System Based on Improved Deep Reinforcement Learning

Deep Reinforcement Learning–based Approach for Optimizing Energy Conversion in Integrated Electrical and Heating System with Renewable Energy

Energy Management for a User Interactive Smart Community: A Stackelberg Game Approach

Dynamic energy conversion and management strategy for an integrated electricity and natural gas system with renewable energy: Deep reinforcement learning approach

Dynamic optimization of an integrated energy system with carbon capture and power-to-gas interconnection: A deep reinforcement learning-based scheduling strategy