Abstract:Energy hub (EH) is an independent entity that benefits to the efficiency, flexibility, and reliability of integrated energy systems (IESs). On the other hand, the rapid emerging of electric vehicles (EVs) drives the EV aggregator (EVAGG) as another independent entity to facilitate the electricity exchange with the grid. However, due to privacy consideration for different owners, it is challenging to investigate the optimal coordinated strategies for such interconnected entities only by exchanging the information of electrical energy. Besides, the existence of parameter uncertainties (load demands, EVs' charging behaviors, wind power and photovoltaic generation), continuous decision space, dynamic energy flows, and non-convex multi-objective function is difficult to solve. To this end, this paper proposes a novel model-free multi-agent deep reinforcement learning (MADRL)-based decentralized coordination model to minimize the energy costs of EH entities and maximize profits of EVAGGs. First, a long short-term memory (LSTM) module is used to capture the future trend of uncertainties. Then, the coordination problem is formulated as Markov games and solved by the attention enabled MADRL algorithm, where the EH or EVAGG entity is modeled as an adaptive agent. An attention mechanism makes each agent only focus on state information related to the reward. The proposed MADRL adopts the forms of offline centralized training to learn the optimal coordinated control strategy, and decentralized execution to enable agents' online decisions to only require local measurements. A safety network is employed to cope with equality constraints (demand-supply balance). Simulation results illustrate that the proposed method achieves similar results compared to the traditional model-based method with perfect knowledge of system models, and the computation performance is at least two orders of magnitudes shorter than the traditional method. The testing results of the proposed method are better than those of the Concurrent and other MADRL method, with 10.79%/3.06% lower energy cost and 17.11%/6.82% higher profits of aggregator. Besides, the electric equality constraint of the proposed method is only 0.25 MW averaged per day, which is a small and acceptable violation.

EV Charging Strategy Considering Transformer Lifetime Via Evolutionary Curriculum Learning-Based Multiagent Deep Reinforcement Learning

A Multi-agent Deep Reinforcement Learning-Based Approach for the Optimization of Transformer Life Using Coordinated Electric Vehicles

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

A Cooperative Charging Control Strategy for Electric Vehicles Based on Multiagent Deep Reinforcement Learning

A Multi-Agent Reinforcement Learning-Based Method for Multiple Electric Vehicles Charging Scheduling

A Deep RL-based Algorithm for Coordinated Charging of Electric Vehicles

Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks

Optimal Electric Vehicle Charging Strategy with Markov Decision Process and Reinforcement Learning Technique

Deep Reinforcement Learning Based Charging Scheduling for Household Electric Vehicles in Active Distribution Network

An Efficient Distributed Multi-Agent Reinforcement Learning for EV Charging Network Control

Multi-agent Deep Reinforcement Learning Based Approach for Power Command Response Combining Fast and Normal Charging Piles

Centralized vs. Decentralized Multi-Agent Reinforcement Learning for Enhanced Control of Electric Vehicle Charging Networks

Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability

Multi-Agent Deep Reinforcement Learning Method for EV Charging Station Game

Multi-agent DRL-based Data-Driven Approach for PEVs Charging/discharging Scheduling in Smart Grid.

Multi-objective Optimal Scheduling of Charging Stations Based on Deep Reinforcement Learning

Coordinated voltage regulation of high renewable-penetrated distribution networks: An evolutionary curriculum-based deep reinforcement learning approach

Novel Data-Driven Decentralized Coordination Model for Electric Vehicle Aggregator and Energy Hub Entities in Multi-Energy System Using an Improved Multi-Agent DRL Approach

Research on Multi-objective Charging Scheduling Strategy for Electric Vehicles: A Deep Reinforcement Learning Approach

A Scalable and Coordinated Energy Management for Electric Vehicles Based on Multiagent Reinforcement Learning Method