Abstract:As urban centers evolve into smart cities, sustainable mobility emerges as a cornerstone for ensuring environmental integrity and enhancing quality of life. Autonomous vehicles (AVs) play a pivotal role in this transformation, with the potential to significantly improve efficiency and safety, and reduce environmental impacts. This study introduces a novel Multi-Agent Actor–Critic (MA2C) algorithm tailored for multi-AV lane-changing in mixed-traffic scenarios, a critical component of intelligent transportation systems in smart cities. By incorporating a local reward system that values efficiency, safety, and passenger comfort, and a parameter-sharing scheme that encourages inter-agent collaboration, our MA2C algorithm presents a comprehensive approach to urban traffic management. The MA2C algorithm leverages reinforcement learning to optimize lane-changing decisions, ensuring optimal traffic flow and enhancing both environmental sustainability and urban living standards. The actor–critic architecture is refined to minimize variances in urban traffic conditions, enhancing predictability and safety. The study extends to simulating realistic human-driven vehicle (HDV) behavior using the Intelligent Driver Model (IDM) and the model of Minimizing Overall Braking Induced by Lane changes (MOBIL), contributing to more accurate and effective traffic management strategies. Empirical results indicate that the MA2C algorithm outperforms existing state-of-the-art models in managing lane changes, passenger comfort, and inter-vehicle cooperation, essential for the dynamic environment of smart cities. The success of the MA2C algorithm in facilitating seamless interaction between AVs and HDVs holds promise for more fluid urban traffic conditions, reduced congestion, and lower emissions. This research contributes to the growing body of knowledge on autonomous driving within the framework of sustainable smart cities, focusing on the integration of AVs into the urban fabric. It underscores the potential of machine learning and artificial intelligence in developing transportation systems that are not only efficient and safe but also sustainable, supporting the broader goals of creating resilient, adaptive, and environmentally friendly urban spaces.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to solve the traffic management problems in sustainable smart cities through multi - agent reinforcement learning (MARL) technology, especially the lane - changing problem of multiple autonomous vehicles (AVs) in a mixed traffic environment. Specifically, the paper attempts to solve the following key problems: 1. **Improve traffic efficiency**: By optimizing lane - changing decisions, reduce traffic congestion and improve the overall efficiency of urban traffic. 2. **Enhance traffic safety**: Ensure that autonomous vehicles avoid collisions and maintain a safe distance during lane - changing, thus improving road safety. 3. **Improve passenger comfort**: During lane - changing, reduce sudden acceleration and sudden braking to improve the passenger's riding experience. 4. **Reduce environmental impact**: By optimizing driving behavior, reduce emissions and energy consumption to support the sustainable development of the city. ### Main methods and contributions To achieve the above goals, the paper proposes a new algorithm named multi - agent actor - critic (MA2C). The main features and contributions of this algorithm include: 1. **Multi - agent cooperation**: Through the parameter - sharing mechanism, encourage cooperation among multiple autonomous vehicles to jointly optimize the traffic flow. 2. **Local reward system**: Design a local reward system that comprehensively considers efficiency, safety, and passenger comfort to ensure that each decision can balance these factors. 3. **Simulate real - human driving behavior**: Use the intelligent driver model (IDM) and the minimizing overall braking induced by lane changes (MOBIL) model to more accurately simulate human driving behavior and improve the effectiveness of traffic management strategies. 4. **Application of reinforcement learning**: Utilize reinforcement learning technology to enable autonomous vehicles to learn from experience, continuously optimize their decision - making strategies, and adapt to the complex urban traffic environment. ### Experimental verification The paper verifies the effectiveness of the MA2C algorithm through a series of experiments. The experimental results show that the MA2C algorithm is superior to the existing state - of - the - art models in managing lane - changing, passenger comfort, and vehicle - to - vehicle cooperation. Specific experimental methods include: 1. **Longitudinal control**: Use the IDM model to adjust the position and speed of human - driven vehicles, consider ecological driving variables, and promote fuel efficiency and reduce emissions. 2. **Horizontal control**: Use the MOBIL model to adjust lateral movement, consider eco - friendly lane - changing, reduce sudden braking, improve traffic fluency, and reduce environmental impact. 3. **Lane - changing threshold**: Based on the politeness coefficient, influence lane - changing decisions, and consider overall traffic efficiency and emission reduction. ### Conclusion This research demonstrates the potential of multi - agent reinforcement learning in solving traffic management problems in sustainable smart cities by introducing the MA2C algorithm. This method not only improves traffic efficiency and safety but also enhances passenger comfort and reduces environmental impact, providing a new solution for the traffic systems of future smart cities.

Sustainable Smart Cities through Multi-Agent Reinforcement Learning-Based Cooperative Autonomous Vehicles

Developing Smart Lane-changing Strategies for CAVs on Freeways based on MOBIL and Reinforcement Learning

Hybrid Operations of Human Driving Vehicles and Automated Vehicles with Data-Driven Agent-Based Simulation

A Reinforcement Learning Approach to Smart Lane Changes of Self-driving Cars

Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic

Policy-Based Reinforcement Learning for Training Autonomous Driving Agents in Urban Areas With Affordance Learning

Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic

Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approach

Developing an eco-driving strategy in a hybrid traffic network using reinforcement learning

Multi-Agent Reinforcement Learning for Traffic Flow Management of Autonomous Vehicles

A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehicles

Adaptive Road Configurations for Improved Autonomous Vehicle-Pedestrian Interactions using Reinforcement Learning

Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

A Reinforcement Learning-based Adaptive Control Model for Future Street Planning, An Algorithm and A Case Study

Enhancing autonomous vehicle hyperawareness in busy traffic environments: A machine learning approach

Deep Reinforcement Learning-Based Long-Range Autonomous Valet Parking for Smart Cities

Multi-agent reinforcement learning for autonomous vehicles: a survey

Social Coordination and Altruism in Autonomous Driving

Prediction-aware and Reinforcement Learning based Altruistic Cooperative Driving

Towards Socially Responsive Autonomous Vehicles: A Reinforcement Learning Framework with Driving Priors and Coordination Awareness

A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways