Abstract:The objective of the authors' study is to investigate a dual‐layer optimization model for the economic viability and operational stability of distribution system operators and virtual power plants. To address this issue, the authors have introduced a parameter‐sharing deep reinforcement learning algorithm. Through case studies and comparisons with traditional model‐based algorithms and heuristic algorithms, the authors have demonstrated the excellent performance of their proposed method in terms of both efficiency and solution accuracy. The authors' aim is to enhance the benefits and operational efficiency for market participants while providing valuable insights for decision‐makers and researchers in this field. With the increasing integration of distributed energy resources (DERs) into distribution systems, the optimization of system operation has become complex, facing challenges such as inadequate consideration of market participants' benefits, poor computational efficiency, and data privacy concerns. This paper introduces the concept of a virtual power plant (VPP) as a solution for energy integration and management. To strike a balance between operational safety and the interests of market participants, a dual‐layer model is proposed. This model considers the benefits of both Distribution System Operators (DSO) and VPP, while also enhancing the consideration of distribution network constraints. The DSO considers AC optimal power flow and utilizes penalty functions to ensure network security in case of violations. To enhance computational efficiency and privacy, the paper presents the parameter‐sharing twin delayed deep deterministic policy gradient approach. This approach allows multiple intelligent agents to share a neural network model, effectively reducing the computational load. During the training process, only essential data is exchanged among the agents, ensuring the privacy of sensitive information. The effectiveness of the proposed model and the algorithm is validated through a case study on an IEEE 33‐node system.

A Deep Reinforcement Learning Approach to Efficient Distributed Optimization

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

DDPG-based Reactive Power Optimization Strategy for Active Distribution Network

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Decentralized Policy Optimization

Deep Reinforcement Learning for Multiobjective Optimization

Optimal Energy System Scheduling Using A Constraint-Aware Reinforcement Learning Algorithm

Proximal Policy Optimization with Mixed Distributed Training

Deep Reinforcement Learning Based Approach for Optimal Power Flow of Distribution Networks Embedded with Renewable Energy and Storage Devices

DPO: Differential reinforcement learning with application to optimal configuration search

Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning

Scalable Model-based Policy Optimization for Decentralized Networked Systems

A reinforcement learning approach for dynamic multi-objective optimization

A novel reinforcement learning-based method for structure optimization

A Deep Reinforcement Learning Approach to the Flexible Flowshop Scheduling Problem with Makespan Minimization

Meta-Learning-Based Deep Reinforcement Learning for Multiobjective Optimization Problems

Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

Collaborative operation optimization of distribution system and virtual power plants using multi‐agent deep reinforcement learning with parameter‐sharing mechanism

A Data-driven Method for Fast AC Optimal Power Flow Solutions via Deep Reinforcement Learning

Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation via Deep Reinforcement Learning