Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Graph Learning-Based Voltage Regulation in Distribution Networks with Multi-Microgrids

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Secure Energy Management of Multi-Energy Microgrid: A Physical-Informed Safe Reinforcement Learning Approach

Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm

Physical-assisted multi-agent graph reinforcement learning enabled fast voltage regulation for PV-rich active distribution network

Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning

Coordinated voltage regulation of high renewable-penetrated distribution networks: An evolutionary curriculum-based deep reinforcement learning approach

Topology-independent End-to-end Learning Model for Improving the Voltage Profile in Microgrids-Integrated Power Distribution Networks

A self-organizing multi-agent system for distributed voltage regulation

Distributed Data-Driven Frequency Control in Networked Microgrids via Voltage Regulation

Multi Agent Safe Graph Reinforcement Learning for PV Inverter s Based Real-Time De centralized Volt/Var Control in Zoned Distribution Networks

Voltage Regulation Algorithms for Multiphase Power Distribution Grids

Intelligent Multi-Microgrid Energy Management Based on Deep Neural Network and Model-Free Reinforcement Learning

Multi-Stage Voltage Support Optimization for Microgrids With Multiple Distributed Generation Units

Deep reinforcement learning based voltage regulation in edge computing paradigm for PV rich distribution networks

Multi-timescale voltage control for distribution system based on multi-agent deep reinforcement learning

Preference based multi-objective reinforcement learning for multi-microgrid system optimization problem in smart grid

Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning

Multi-agent Graph Reinforcement Learning for Decentralized Volt-VAR Control in Power Distribution Systems

Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

GridLearn: Multiagent reinforcement learning for grid-aware building energy management