Abstract:Nowadays, most of the taxi drivers have become users of the relocation recommendation service offered by online ride-hailing platforms (e.g., Uber and Didi Chuxing), which could oftentimes lead drivers to places with profitable orders. At the same time, electric taxis (e-taxis) are increasingly adopted and gradually replacing gasoline taxis in today's public transportation systems due to their environmental-friendly nature. Though effective for traditional gasoline taxis, existing relocation recommendation schemes are rather suboptimal for e-taxi drivers' user experience. On one hand, the existing schemes take no account of taxis' refueling decisions, as the refueling durations of gasoline taxis are usually short enough to be ignored. However, the charging duration of the e-taxis spent at charging stations can be as long as hours. Obviously, an e-taxi's battery could be easily depleted by the continuous relocations suggested by existing schemes, and thus will have to be charged for a long time afterwards, making the e-taxi driver miss numerous order-serving opportunities. On the other hand, charging posts are typically sparsely and unevenly distributed across a city. With no consideration of charging opportunities, existing schemes could probably send an e-taxi to an area with no charging post around, even though its battery is running low. To optimize e-taxi drivers' user experience, in this paper, we design a joint charging and relocation recommendation system for e-taxi drivers (CARE). We take the perspective of e-taxi drivers and formulate their decision making as a multi-agent reinforcement learning problem where each e-taxi driver aims to maximize his own cumulative rewards. More specifically, we propose a novel multi-agent mean field hierarchical reinforcement learning (MFHRL) framework. The hierarchical architecture of MFHRL helps-the proposed CARE provide far-sighted charging and relocation recommendations for e-taxi drivers. Besides, we integrate each hierarchical level of MFHRL separately with the mean field approximation to incorporate e-taxis' mutual influences in decision making. We set up a simulator with one of the largest real-world e-taxi datasets in Shenzhen, China, which contains the GPS trajectory data and transaction data of 3848 e-taxis from June 1st to June 30th, 2017, coupled with 165 charging stations including 317 fast charging posts and 1421 slow charging posts. We adopt this simulator to generate 6 dynamic urban environments, which reflect the different real-world scenarios faced by e-taxi drivers. In all of these environments, we conduct extensive experiments to validate that the proposed MFHRL framework greatly outperforms all baselines by significantly increasing the rewards obtained by e-taxi drivers. Besides, we also show that the charging policy learned by MFHRL can effectively reduce the range anxiety of e-taxi drivers, which significantly boosts e-taxi drivers' quality of experience.

Deep Reinforcement Learning for Urban Multi-Taxis Cruising Strategy

Reinforcement Learning For Optimizing Driving Policies On Cruising Taxis Services

Spatial-temporal Pricing for Ride-Sourcing Platform with Reinforcement Learning

Good or Mediocre? A Deep Reinforcement Learning Approach for Taxi Revenue Efficiency Optimization

Optimize Taxi Driving Strategies Based On Reinforcement Learning

AdaBoost-Bagging Deep Inverse Reinforcement Learning for Autonomous Taxi Cruising Route and Speed Planning

Where to go: Agent Guidance with Deep Reinforcement Learning in A City-Scale Online Ride-Hailing Service

Dual-objective Optimization of Taxi Dispatching in Simulated Road Network Based on Reinforcement Learning

Intelligent Cruise Guidance and Vehicle Resource Management With Deep Reinforcement Learning

Supply-Demand-aware Deep Reinforcement Learning for Dynamic Fleet Management

Multi-agent Reinforcement Learning to Unify Order-Matching and Vehicle-Repositioning in Ride-Hailing Services

Optimising Stochastic Routing for Taxi Fleets with Model Enhanced Reinforcement Learning

META: A City-Wide Taxi Repositioning Framework Based on Multi-Agent Reinforcement Learning

Toward Large-Scale Agent Guidance in an Urban Taxi Service

Context-Aware Taxi Dispatching at City-Scale Using Deep Reinforcement Learning

Joint Charging and Relocation Recommendation for E-Taxi Drivers via Multi-Agent Mean Field Hierarchical Reinforcement Learning

A robust deep reinforcement learning approach to driverless taxi dispatching under uncertain demand

Multi-agent reinforcement learning-based dynamic task assignment for vehicles in urban transportation system

Combinatorial Optimization Meets Reinforcement Learning: Effective Taxi Order Dispatching at Large-Scale

Scalable Deep Reinforcement Learning for Ride-Hailing

The Real-Time Dynamic Prediction of Optimal Taxi Cruising Area Based on Deep Learning