Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

A Foresight-Seeing and Transferable Optimization Method for Synergic Operation of Multiple Flexible Resources in Active Distribution Network

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

A Multi-Stage Optimization Approach for Active Distribution Network Scheduling Considering Coordinated Electrical Vehicle Charging Strategy

Day-Ahead Optimal Dispatch in Active Distribution Network Based on Deep Reinforcement Learning with Improved Feature Extraction Network

Optimal Operation Strategy for Active Distribution Company With Conservation Voltage Reduction in Competitive Electricity Markets

Optimal Bi-Level Bidding and Dispatching Strategy between Active Distribution Network and Virtual Alliances Using Distributed Robust Multi-Agent Deep Reinforcement Learning

DDPG-based Reactive Power Optimization Strategy for Active Distribution Network

Active and Reactive Power Collaborative Optimization for Active Distribution Networks Considering Bi-Directional V2G Behavior

Multi-Objective Optimization Operation of Multi-Agent Active Distribution Network Based on Analytical Target Cascading Method

Robust Operation of Flexible Distribution Network with Large-Scale EV Charging Loads

Collaborative operation optimization of distribution system and virtual power plants using multi‐agent deep reinforcement learning with parameter‐sharing mechanism

Deep Reinforcement Learning Based Approach for Dynamic Optimal Power Flow in Active Distribution Network

OPTIMAL RECONFIGURATION OF THE ACTIVE DISTRIBUTION NETWORKWITH DISTRIBUTED GENERATION AND ELECTRIC VEHICLE

Coordinated Dispatch of Energy Storage Systems in the Active Distribution Network: A Complementary Reinforcement Learning and Optimization Approach

Multi-Stage Planning of Active Distribution Networks Considering the Co-Optimization of Operation Strategies

Bilevel optimal coordination of active distribution network and charging stations considering EV drivers' willingness

Optimal Scheduling in Active Distribution Network Considering Coordinated Interactions among Source-Network-Load-Storage

A Bi-Level Optimal Operation Model for Small-Scale Active Distribution Networks Considering the Coupling Fluctuation of Spot Electricity Prices and Renewable Energy Sources

A Modified Long Short-Term Memory-Deep Deterministic Policy Gradient-Based Scheduling Method for Active Distribution Networks

Electric Vehicles Management in Distribution Network: A Data-Efficient Bi-level Safe Deep Reinforcement Learning Method

Optimization Scheduling of Distribution Network with Aggregated Flexible Resources