From Sim to Real: A Pipeline for Training and Deploying Traffic Smoothing Cruise Controllers
Nathan Lichtlé,Eugene Vinitsky,Matthew Nice,Rahul Bhadani,Matthew Bunting,Fangyu Wu,Benedetto Piccoli,Benjamin Seibold,Daniel B. Work,Jonathan W. Lee,Jonathan Sprinkle,Alexandre M. Bayen
DOI: https://doi.org/10.1109/tro.2024.3463407
IF: 7.8
2024-10-11
IEEE Transactions on Robotics
Abstract:Designing and validating controllers for connected and automated vehicles to enhance traffic flow presents significant challenges, from the complexity of replicating real-world stop-and-go traffic dynamics in simulation, to the intricacies involved in transitioning from simulation to actual deployment. In this work, we present a full pipeline from data collection to controller deployment. Specifically, we collect 772 km of driving data from the I-24 in Tennessee, and use it to build a one-lane simulator, placing simulated vehicles behind real-world trajectories. Using policy-gradient methods with an asymmetric critic, we improve fuel efficiency by over 10% when simulating congested scenarios. Our comprehensive approach includes reinforcement learning for controller training, software verification, hardware validation and setup, and navigating various sim-to-real challenges. Furthermore, we analyze the controller's behavior and wave-smoothing properties, and deploy it on four Toyota Rav4's in a real-world validation experiment on the I-24. Finally, we release the driving dataset (Nice et al., 2021), the simulator and the trained controller (Lichtlé et al., 2022), to enable future benchmarking and controller design.
robotics