Abstract:Public transports, such as subway lines and buses, offer affordable ride-sharing services and reduce the road network traffic. Extracting passengers’ preferences from their public transit choices is important to city planners but technically non-trivial. When traveling by taking public transits, passengers make sequences of transit choices, and their rewards are usually influenced by other passengers’ choices. This process can be modeled as a Markov Game (MG). In this paper, we make the first effort to model travelers’ preferences of making transit choices using MGs. Based on the discovery that passengers usually do not change their policies, we propose novel algorithms to extract reward functions from the observed deterministic equilibrium joint policy of all agents in a general-sum MG to infer travelers’ preferences. First, we assume we have the access to the entire joint policy. We characterize the set of all reward functions for which the given joint policy is a Nash equilibrium policy. In order to remove the degeneracy of the solution, we then attempt to pick reward functions so as to maximize the sum of the deviation between the the observed policy and the sub-optimal policy of each agent. This results in a skillfully solvable linear programming algorithm for the multi-agent inverse reinforcement learning (MA-IRL) problem. Then, we deal with the case where we have access to the equilibrium joint policy through a set of actual trajectories. We propose an iterative algorithm inspired by single-agent apprenticeship learning algorithms and the cyclic coordinate descent approach. We evaluate the proposed algorithms on both a simple Grid Game and a unique real-world dataset (from Shenzhen, China). Results show that when we have access to the full policy, our algorithm can efficiently recover most of the reward structure, especially the interaction of agents. In the case where we only have access to a set of sampled expert trajectories, our algorithm can provide an explanation of the expert trajectories. Measured with respect to the experts’ unknown reward function, the performance of the policy output by our algorithm is close to that of the expert policy.

Multi-agent Reinforcement Learning-based Passenger Spoofing Attack on Mobility-as-a-Service

All Your GPS Are Belong to Us: Towards Stealthy Manipulation of Road Navigation Systems.

MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning

SoK: Rethinking Sensor Spoofing Attacks Against Robotic Vehicles from a Systematic View.

Laser-Based LiDAR Spoofing: Effects Validation, Capability Quantification, and Countermeasures

Collaborative Attack Sequence Generation Model Based on Multiagent Reinforcement Learning for Intelligent Traffic Signal System

Resiliency of Mobility-as-a-Service Systems to Denial-of-Service Attacks

Multi-Agent Reinforcement Learning for Assessing False-Data Injection Attacks on Transportation Networks

A Survey of AI-Related Cyber Security Risks and Countermeasures in Mobility-as-a-Service

CDA: Covert Deception Attacks in Multi-Agent Resource Scheduling

Defending Against Network Attacks for Secure AI Agent Migration in Vehicular Metaverses

Safe Multi-Agent Reinforcement Learning for Wireless Applications Against Adversarial Communications

Cooperative Perception for Safe Control of Autonomous Vehicles under LiDAR Spoofing Attacks

Adversarial Attacks Against Deep Reinforcement Learning Framework in Internet of Vehicles

Hybrid Trajectory Planning for Connected and Autonomous Vehicle Considering Communication Spoofing Attacks

A Novel Classifier Exploiting Mobility Behaviors for Sybil Detection in Connected Vehicle Systems

Pseudonym Inference in Cooperative Vehicular Traffic Scenarios

Distributed Adaptive Event-Triggered Control of Connected Automated Vehicle Platoon Systems with Spoofing Cyber Attacks

Detecting stealthy cyberattacks on adaptive cruise control vehicles: A machine learning approach

Inferring Passengers' Interactive Choices on Public Transits Via MA-AL: Multi-Agent Apprenticeship Learning.

Camouflage Adversarial Attacks on Multiple Agent Systems