Abstract:Public transports, such as subway lines and buses, offer affordable ride-sharing services and reduce the road network traffic. Extracting passengers’ preferences from their public transit choices is important to city planners but technically non-trivial. When traveling by taking public transits, passengers make sequences of transit choices, and their rewards are usually influenced by other passengers’ choices. This process can be modeled as a Markov Game (MG). In this paper, we make the first effort to model travelers’ preferences of making transit choices using MGs. Based on the discovery that passengers usually do not change their policies, we propose novel algorithms to extract reward functions from the observed deterministic equilibrium joint policy of all agents in a general-sum MG to infer travelers’ preferences. First, we assume we have the access to the entire joint policy. We characterize the set of all reward functions for which the given joint policy is a Nash equilibrium policy. In order to remove the degeneracy of the solution, we then attempt to pick reward functions so as to maximize the sum of the deviation between the the observed policy and the sub-optimal policy of each agent. This results in a skillfully solvable linear programming algorithm for the multi-agent inverse reinforcement learning (MA-IRL) problem. Then, we deal with the case where we have access to the equilibrium joint policy through a set of actual trajectories. We propose an iterative algorithm inspired by single-agent apprenticeship learning algorithms and the cyclic coordinate descent approach. We evaluate the proposed algorithms on both a simple Grid Game and a unique real-world dataset (from Shenzhen, China). Results show that when we have access to the full policy, our algorithm can efficiently recover most of the reward structure, especially the interaction of agents. In the case where we only have access to a set of sampled expert trajectories, our algorithm can provide an explanation of the expert trajectories. Measured with respect to the experts’ unknown reward function, the performance of the policy output by our algorithm is close to that of the expert policy.

Unveiling Taxi Drivers' Strategies via cGAIL: Conditional Generative Adversarial Imitation Learning

cGAIL: Conditional Generative Adversarial Imitation Learning—An Application in Taxi Drivers’ Strategy Learning

Learning Decision Making Strategies of Non-experts: A NEXT-GAIL Model for Taxi Drivers

xGAIL: Explainable Generative Adversarial Imitation Learning for Explainable Human Decision Analysis

Exploring Competitiveness of Taxis to Ride-Hailing Services from a Multidimensional Spatio-Temporal Perspective: A Case Study in Beijing, China

Optimize Taxi Driving Strategies Based On Reinforcement Learning

Ibat: Detecting Anomalous Taxi Trajectories From Gps Traces

Real Time Anomalous Trajectory Detection and Analysis

Hunting or Waiting? Discovering Passenger-Finding Strategies from a Large-Scale Real-World Taxi Dataset

Exploring the Social Learning of Taxi Drivers in Latent Vehicle-to-Vehicle Networks.

TrajGAIL: Trajectory Generative Adversarial Imitation Learning for Long-term Decision Analysis

Inferring Passengers' Interactive Choices on Public Transits Via MA-AL: Multi-Agent Apprenticeship Learning.

Deep Reinforcement Learning for Urban Multi-Taxis Cruising Strategy

Understanding Taxi Service Strategies from Taxi GPS Traces

Good or Mediocre? A Deep Reinforcement Learning Approach for Taxi Revenue Efficiency Optimization

Finding A Taxi with Illegal Driver Substitution Activity via Behavior Modelings

Taxi-Based Mobility Demand Formulation and Prediction Using Conditional Generative Adversarial Network-Driven Learning Approaches

Reinforcement Learning For Optimizing Driving Policies On Cruising Taxis Services

Optimal Passenger-Seeking Policies on E-hailing Platforms Using Markov Decision Process and Imitation Learning

BiGAIL: Learning Intention-based Driving Policy from Multi-task Demonstrations

Anomalous Ride-Hailing Driver Detection with Deep Transfer Inverse Reinforcement Learning