Abstract:Electric Vehicle (EV) has become a preferable choice in the modern transportation system due to its environmental and energy sustainability. However, in many large cities, EV drivers often fail to find the proper spots for charging, because of the limited charging infrastructures and the spatiotemporally unbalanced charging demands. Indeed, the recent emergence of deep reinforcement learning provides great potential to improve the charging experience from various aspects over a long-term horizon. In this paper, we propose an Imitative Multi-Agent Spatio-Temporal Reinforcement Learning (RlCharge) framework for intelligently recommending public accessible charging stations by jointly considering various long-term spatio-temporal factors. Specifically, by regarding each charging station as an individual agent, we formulate the problem as a multi-objective multi-agent reinforcement learning task. We first develop a multi-agent actor-critic framework with centralized training decentralized execution. Particularly, we propose a tailor-designed centralized attentive critic to coordinate the recommendation between geo-distributed agents, and introduce a delayed access strategy to exploit the knowledge of future charging competition during centralized training. Moreover, to handle the partial observability problem during decentralized execution in the large-scale multi-agent system, we propose the spatio-temporal heterogeneous graph convolution module, including (1) a dynamic graph convolution block to generate real-time representations for observable forthcoming EVs, and (2) a spatial graph convolution block to share the agent observations by message propagation between spatially adjacent agents. After that, to effectively optimize multiple divergent learning objectives, we extend the centralized attentive critic to multi-critics, and develop a dynamic gradient re-weighting strategy to adaptively guide the optimization direction. In addition, we p- opose an adaptive imitation learning scheme to further accelerate and stabilize the policy convergence. Finally, extensive experiments on two real-world datasets demonstrate that RlCharge achieves the best comprehensive performance compared with ten baseline approaches.

Evolution-Assisted Deep Reinforcement Learning for Fast Charging Station Coordinated Operation.

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

Multiagent Deep Reinforcement Learning for Electric Vehicle Fast Charging Station Pricing Game in Electricity-Transportation Nexus

Deep Reinforcement Learning Based Charging Scheduling for Household Electric Vehicles in Active Distribution Network

Multi-Agent Graph Reinforcement Learning Method for Electric Vehicle on-Route Charging Guidance in Coupled Transportation Electrification

Learning to Operate an Electric Vehicle Charging Station Considering Vehicle-grid Integration

Online Prediction-Assisted Safe Reinforcement Learning for Electric Vehicle Charging Station Recommendation in Dynamically Coupled Transportation-Power Systems

Electric vehicle charging scheduling control strategy for the large-scale scenario with non-cooperative game-based multi-agent reinforcement learning

A Deep RL-based Algorithm for Coordinated Charging of Electric Vehicles

RLCharge: Imitative Multi-Agent Spatiotemporal Reinforcement Learning for Electric Vehicle Charging Station Recommendation

Real-time fast charging station recommendation for electric vehicles in coupled power-transportation networks: A graph reinforcement learning method

A deep reinforcement learning-based charging scheduling approach with augmented Lagrangian for electric vehicles

Decentralized collaborative optimal scheduling for EV charging stations based on multi‐agent reinforcement learning

PROLIFIC: Deep Reinforcement Learning for Efficient EV Fleet Scheduling and Charging

Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks

Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability

EV Charging Command Fast Allocation Approach based on Deep Reinforcement Learning with Safety Modules

Electric Vehicle Charging Guidance Strategy Considering “Traffic Network-Charging Station-Driver” Modeling: A Multi-Agent Deep Reinforcement Learning Based Approach

Optimal self-consumption scheduling of highway electric vehicle charging station based on multi-agent deep reinforcement learning