Abstract:Bus holding control is a widely-adopted strategy for maintaining stability and improving the operational efficiency of bus systems. Traditional model-based methods often face challenges with the low accuracy of bus state prediction and passenger demand estimation. In contrast, Reinforcement Learning (RL), as a data-driven approach, has demonstrated great potential in formulating bus holding strategies. RL determines the optimal control strategies in order to maximize the cumulative reward, which reflects the overall control goals. However, translating sparse and delayed control goals in real-world tasks into dense and real-time rewards for RL is challenging, normally requiring extensive manual trial-and-error. In view of this, this study introduces an automatic reward generation paradigm by leveraging the in-context learning and reasoning capabilities of Large Language Models (LLMs). This new paradigm, termed the LLM-enhanced RL, comprises several LLM-based modules: reward initializer, reward modifier, performance analyzer, and reward refiner. These modules cooperate to initialize and iteratively improve the reward function according to the feedback from training and test results for the specified RL-based task. Ineffective reward functions generated by the LLM are filtered out to ensure the stable evolution of the RL agents' performance over iterations. To evaluate the feasibility of the proposed LLM-enhanced RL paradigm, it is applied to various bus holding control scenarios, including a synthetic single-line system and a real-world multi-line system. The results demonstrate the superiority and robustness of the proposed paradigm compared to vanilla RL strategies, the LLM-based controller, and conventional space headway-based feedback control. This study sheds light on the great potential of utilizing LLMs in various smart mobility applications.

LLM4RL: Enhancing Reinforcement Learning with Large Language Models

Large Language Models for Autonomous Driving (LLM4AD): Concept, Benchmark, Simulation, and Real-Vehicle Experiment

Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods

LLM4Drive: A Survey of Large Language Models for Autonomous Driving

Empowering Autonomous Driving with Large Language Models: A Safety Perspective

Prompting Multi-Modal Tokens to Enhance End-to-End Autonomous Driving Imitation Learning with LLMs

Large Language Model-Enhanced Reinforcement Learning for Generic Bus Holding Control Strategies

The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Models

DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving

Generating and Evolving Reward Functions for Highway Driving with Large Language Models

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents

Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles

Receive, Reason, and React: Drive as You Say, With Large Language Models in Autonomous Vehicles

Drive Like a Human: Rethinking Autonomous Driving with Large Language Models

Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving

Reinforcement Learning Problem Solving with Large Language Models

A Survey on Multimodal Large Language Models for Autonomous Driving

Efficient Reinforcement Learning with Large Language Model Priors

From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems