Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

A hierarchical reinforcement learning approach for energy‐aware service function chain dynamic deployment in IoT

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Intelligent Orchestrating of IoT Microservices Based on Reinforcement Learning

Toward Scalable and Efficient Hierarchical Deep Reinforcement Learning for 5G RAN Slicing

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Reinforcement Learning for Resource Demand Prediction and Virtual Function Network Migration in Digital Twin Network

Towards an energy-efficient Data Center Network based on deep reinforcement learning

IoT Network with Energy Efficiency for Dynamic Sink via Reinforcement Learning

Hierarchical Deep Reinforcement Learning for Joint Service Caching and Computation Offloading in Mobile Edge-Cloud Computing

Distributed Reinforcement Learning for Age of Information Minimization in Real-Time IoT Systems

Enhancing Smart Grids with Internet of Energy: Deep Reinforcement Learning and Convolutional Neural Network

Dynamic service function chain placement in mobile computing: An asynchronous advantage actor‐critic based approach

Energy‐efficient resource allocation over wireless communication systems through deep reinforcement learning

Real-Time Virtual Machine Scheduling in Industry IoT Network: A Reinforcement Learning Method

Optimal Status Update for Caching Enabled IoT Networks: A Dueling Deep R-Network Approach

DRL-D: Revenue-Aware Online Service Function Chain Deployment Via Deep Reinforcement Learning

Hierarchical Reinforcement Learning Empowered Task Offloading in V2I Networks

Reinforcement Learning based Multi-Access Control and Battery Prediction with Energy Harvesting in IoT Systems

A Deep Reinforcement Learning-Based Caching Strategy for IoT Networks With Transient Data

Collaborative Cloud-Edge Service Cognition Framework for DNN Configuration toward Smart IIoT

Energy-Efficient Ultra-Dense Network with Deep Reinforcement Learning