Abstract:The increasing penetration of intermittent renewable energy sources leads to a higher demand for flexible balancing service in the power system. Consequently, demand-side management techniques, such as utilizing flexible resources from residential electric-heating loads, provide a promising solution. However, due to various uncertainties such as stochastic weather and market prices, it is challenging to develop an economically efficient strategy. Hence, this paper proposes an economic operation optimization approach for residential electric-heating loads to develop optimal bidding and scheduling strategy in the multi-stage market. First, the dynamic characteristics of aggregated electric-heating loads are modeled as the virtual energy storage systems (VESS) to quantify the flexibility potential arising from the combination of building thermal inertia, user tolerance and thermal energy storage (TES). Subsequently, the joint energy and balancing service market is introduced to exploit the operating flexibility of VESS. A multi-agent deep reinforcement learning (MADRL) optimization framework is proposed to address the multi-objective problem at different decision frequency, which involves achieving lower economic costs while maintaining indoor comfort. Finally, we combine the DRL with imitation learning (IL) by introducing the behavior cloning mechanism to mitigate training process instability. It reduces the computational costs and eliminates the need for redundant additional structure. Comparative simulation experiments demonstrate the efficiency of data-driven optimization approach, which provides an adaptive, economic cost-effective and general solution for multi-type aggregators to activate their flexibility in dynamic markets.

Aggregated demand flexibility prediction of residential thermostatically controlled loads and participation in electricity balance markets

Estimation Approach to Aggregated Power and Response Potential of Residential Thermostatically Controlled Loads

Provision of multiple services with controllable loads as multi-area thermal energy storage

Flexibility Management with Virtual Batteries of Thermostatically Controlled Loads: Real-Time Control System and Potential in Spain

A Geometric Approach to Aggregate Flexibility Modeling of Thermostatically Controlled Loads

Reliability Evaluation of Power Systems Considering Integration of Renewable Energy and Aggregated TCLs

Economic operation of an agent-based virtual storage aggregated residential electric-heating loads in multiple electricity markets

A bottom-up quantification of flexibility potential from the thermal energy storage in electric space heating

Modeling Demand Flexibility of RES-based Virtual Power Plants

A Strategy for Contingency Management with Aggregated Thermostatically Controlled Loads in China's Electricity Market

Day-ahead Trading of Aggregated Energy Flexibility - Full Version

Hierarchical Scheduling of Aggregated TCL Flexibility for Transactive Energy in Power Systems.

Multi-Period Flexibility Forecast for Low Voltage Prosumers

A Multi-Battery Model for the Aggregate Flexibility of Heterogeneous Electric Vehicles

Trilevel Scheduling Model Considering Residential Demand Flexibility of Aggregated HVACs and EVs under Distribution LMP

Reach and hold flexibility characterization and trade-off analysis for aggregations of thermostatically controlled loads

Power allocation by load aggregator with heterogeneous loads using weighted projection

Learning a Distributed Control Scheme for Demand Flexibility in Thermostatically Controlled Loads

Aggregate Flexibility of Virtual Power Plants with Temporal Coupling Constraints.

System planning with demand assets in balancing markets

Enhanced Sufficient Battery Model for Aggregate Flexibility of Thermostatically Controlled Loads Considering Coupling Constraints