Abstract:Public transports, such as subway lines and buses, offer affordable ride-sharing services and reduce the road network traffic. Extracting passengers’ preferences from their public transit choices is important to city planners but technically non-trivial. When traveling by taking public transits, passengers make sequences of transit choices, and their rewards are usually influenced by other passengers’ choices. This process can be modeled as a Markov Game (MG). In this paper, we make the first effort to model travelers’ preferences of making transit choices using MGs. Based on the discovery that passengers usually do not change their policies, we propose novel algorithms to extract reward functions from the observed deterministic equilibrium joint policy of all agents in a general-sum MG to infer travelers’ preferences. First, we assume we have the access to the entire joint policy. We characterize the set of all reward functions for which the given joint policy is a Nash equilibrium policy. In order to remove the degeneracy of the solution, we then attempt to pick reward functions so as to maximize the sum of the deviation between the the observed policy and the sub-optimal policy of each agent. This results in a skillfully solvable linear programming algorithm for the multi-agent inverse reinforcement learning (MA-IRL) problem. Then, we deal with the case where we have access to the equilibrium joint policy through a set of actual trajectories. We propose an iterative algorithm inspired by single-agent apprenticeship learning algorithms and the cyclic coordinate descent approach. We evaluate the proposed algorithms on both a simple Grid Game and a unique real-world dataset (from Shenzhen, China). Results show that when we have access to the full policy, our algorithm can efficiently recover most of the reward structure, especially the interaction of agents. In the case where we only have access to a set of sampled expert trajectories, our algorithm can provide an explanation of the expert trajectories. Measured with respect to the experts’ unknown reward function, the performance of the policy output by our algorithm is close to that of the expert policy.

Multiagent-Based Simulation of Temporal-Spatial Characteristics of Activity-Travel Patterns Using Interactive Reinforcement Learning

Multi-agent Based Simulation of Temporal-spatial Characteristics of Activity-travel Patterns Using Interactive Reinforcement Learning

A multi-agent model of traffic simulation around urban scenic spots: From the perspective of tourist behaviors

Activity-Travel Demand Modeling Based on Multi-Agent Simulation

Enhancing Multi-Agent Based Simulation with Human-Agents Interactive Spatial Behaviour

An Initial Implementation of Multiagent Simulation of Travel Behavior for A Medium-Sized City in China

Multi-Agent Simulation for Multi-Mode Travel Policy to Improve Park and Ride Efficiency

Agent-based modelling with geographically weighted calibration for intra-urban activities simulation using taxi GPS trajectories

Evaluating staggered working hours using a multi-agent-based Q-learning model

Allocating Time and Location Information to Activity-Travel Patterns Through Reinforcement Learning.

Multi-Agent Simulation on Day-To-Day Route Choice Behavior

Population Game-Assisted Multi-Agent Reinforcement Learning Method for Dynamic Multi-Vehicle Route Selection

Agent-based Approach to Model Commuter Behaviour's Day-to-day Dynamics under Pre-Trip Information

Dynamic Route and Departure Time Choice Model Based on Self-Adaptive Reference Point and Reinforcement Learning

Application Of An Agent-Based Modeling To Simulate Destination Choice For Shopping And Recreation

Multi-agent reinforcement learning-based dynamic task assignment for vehicles in urban transportation physical internet

Multi-agent reinforcement learning for traffic congestion on one-way multi-lane highways

Multi-agent reinforcement learning-based dynamic task assignment for vehicles in urban transportation system

Inferring Passengers' Interactive Choices on Public Transits Via MA-AL: Multi-Agent Apprenticeship Learning.

Utilizing Multi Agent Simulation to Explore Cities

A Day-to-Day Route Choice Model Based on Reinforcement Learning