Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Graph Convolutional Network Based Multi-Objective Meta-Deep Q-Learning for Eco-Routing

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

DQ-GAT: Towards Safe and Efficient Autonomous Driving With Deep Q-Learning and Graph Attention Networks

Deep Learning Based Proactive Multi-Objective Eco-Routing Strategies for Connected and Automated Vehicles

GMIX: Graph-based spatial–temporal multi-agent reinforcement learning for dynamic electric vehicle dispatching system

A Deep Reinforcement Learning Framework for Eco-driving in Connected and Automated Hybrid Electric Vehicles

Multi-Agent Decision-Making Modes in Uncertain Interactive Traffic Scenarios via Graph Convolution-Based Deep Reinforcement Learning

Multi-objective optimization for autonomous driving strategy based on Deep Q Network

Deep reinforcement learning and reward shaping based eco-driving control for automated HEVs among signalized intersections

Multi-objective Eco-Routing Model Development and Evaluation for Battery Electric Vehicles

Collaborated eco-routing optimization for continuous traffic flow based on energy consumption difference of multiple vehicles

Graph cooperation deep reinforcement learning for ecological urban traffic signal control

Eco-driving strategies using reinforcement learning for mixed traffic in the vicinity of signalized intersections

Electric vehicle charging navigation strategy in coupled smart grid and transportation network: A hierarchical reinforcement learning approach

Multi-Objective Eco-Routing for Dynamic Control of Connected & Automated Vehicles

Eco-driving at signalized intersections: a parameterized reinforcement learning approach

A large-scale traffic signal control algorithm based on multi-layer graph deep reinforcement learning

Overcoming driving challenges in complex urban traffic: A multi-objective eco-driving strategy via safety model based reinforcement learning

CCGN: Centralized collaborative graphical transformer multi-agent reinforcement learning for multi-intersection signal free-corridor

Eco-driving for Electric Connected Vehicles at Signalized Intersections: A Parameterized Reinforcement Learning approach

Online longitudinal trajectory planning for connected and autonomous vehicles in mixed traffic flow with deep reinforcement learning approach