Abstract:The integration of distributed energy resources (DERs) into wholesale energy markets can greatly enhance grid flexibility, improve market efficiency, and contribute to a more sustainable energy future. As DERs -- such as solar PV panels and energy storage -- proliferate, effective mechanisms are needed to ensure that small prosumers can participate meaningfully in these markets. We study a wholesale market model featuring multiple DER aggregators, each controlling a portfolio of DER resources and bidding into the market on behalf of the DER asset owners. The key of our approach lies in recognizing the repeated nature of market interactions the ability of participants to learn and adapt over time. Specifically, Aggregators repeatedly interact with each other and with other suppliers in the wholesale market, collectively shaping wholesale electricity prices (aka the locational marginal prices (LMPs)). We model this multi-agent interaction using a mean-field game (MFG), which uses market information -- reflecting the average behavior of market participants -- to enable each aggregator to predict long-term LMP trends and make informed decisions. For each aggregator, because they control the DERs within their portfolio under certain contract structures, we employ a mean-field control (MFC) approach (as opposed to a MFG) to learn an optimal policy that maximizes the total rewards of the DERs under their management. We also propose a reinforcement learning (RL)-based method to help each agent learn optimal strategies within the MFG framework, enhancing their ability to adapt to market conditions and uncertainties. Numerical simulations show that LMPs quickly reach a steady state in the hybrid mean-field approach. Furthermore, our results demonstrate that the combination of energy storage and mean-field learning significantly reduces price volatility compared to scenarios without storage.

Aggregate Feasible Region of DERs: Exact Formulation and Approximate Models

Flexibility Resources Aggregation and Disaggregation Method Considering Renewable Consumption

Aggregate Operation Model for Numerous Small-Capacity Distributed Energy Resources Considering Uncertainty

Aggregated Feasible Region of Heterogeneous Demand-Side Flexible Resources - Part I: Theoretical Derivation of the Exact Model

A Projection-Based Approach for Distributed Energy Resources Aggregation

Modeling and Aggregating DER Flexibility Region in VPPs: an Elimination and Projection Approach

Aggregate Temporally Coupled Power Flexibility of DERs Considering Distribution System Security Constraints

Stochastic modeling for the aggregated flexibility of distributed energy resources

Approximating Energy-Regulation Feasible Regions of Virtual Power Plants: A Data-Driven Inverse Optimization Approach

Aggregation Mechanism Between Virtual Power Plants and Distributed Energy Resources

Aggregating Distributed Energy Resources: Efficiency and Market Power

On Efficient Aggregation of Distributed Energy Resources

A Network-Aware Distributed Energy Resource Aggregation Framework for Flexible, Cost-Optimal, and Resilient Operation

Evaluating the Impact of Multiple DER Aggregators on Wholesale Energy Markets: A Hybrid Mean Field Approach

Characterizing Temporal-coupled Feasible Region of Active Distribution Networks.

Impact analysis of DERs on bulk power system stability through the parameterization of aggregated DER_a model for real feeders

Quantification of Market Power Mitigation via Efficient Aggregation of Distributed Energy Resources

An AC-Feasible Linear Model in Distribution Networks with Energy Storage

Competitive DER Aggregation for Participation in Wholesale Markets

Coordination of medium-voltage distribution networks and microgrids based on an aggregate flexibility region approach

Grid-aware aggregation and realtime disaggregation of distributed energy resources in radial networks