Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Multi-agent Reinforcement Learning-based Capacity Planning for On-demand Vehicular Fog Computing

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Many-to-Many Task Offloading in Vehicular Fog Computing: A Multi-Agent Deep Reinforcement Learning Approach

Hierarchical Task Offloading for Vehicular Fog Computing Based on Multi-Agent Deep Reinforcement Learning

Priority-Aware Task Offloading in Vehicular Fog Computing Based on Deep Reinforcement Learning

Cross-Regional Task Offloading with Multi-Agent Reinforcement Learning for Hierarchical Vehicular Fog Computing.

Federated Deep Reinforcement Learning-Based Task Allocation in Vehicular Fog Computing.

Mobile Vehicles as Fog Nodes for Latency Optimization in Smart Cities

Dynamic Many-to-Many Task Offloading in Vehicular Fog Computing: A Multi-Agent DRL Approach

A Scalable Mean-Field MARL Framework for Multi-Objective V2X Resource Allocation

Federated Multi-Agent Deep Reinforcement Learning for Resource Allocation of Vehicle-to-Vehicle Communications

Deep Reinforcement Learning-Based V2V Partial Computation Offloading in Vehicular Fog Computing

Deep Reinforcement Learning for Delay-Optimized Task Offloading in Vehicular Fog Computing

Joint Communication and Computation Resource Allocation in Fog-Based Vehicular Networks

Platoon Leader Selection, User Association and Resource Allocation on a C-V2X based highway: A Reinforcement Learning Approach

DNN Partitioning, Task Offloading, and Resource Allocation in Dynamic Vehicular Networks: A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approach

Joint Access Mode Selection and Spectrum Allocation for Fog Computing Based Vehicular Networks

Mobility Prediction-Based Joint Task Assignment and Resource Allocation in Vehicular Fog Computing.

Fog Computing Model and Efficient Algorithms for Directional Vehicle Mobility in Vehicular Network

A Delay-Optimal Task Scheduling Strategy for Vehicle Edge Computing Based on the Multi-Agent Deep Reinforcement Learning Approach

Fully Distributed Fog Load Balancing with Multi-Agent Reinforcement Learning