Abstract:Predicting the future states of surrounding traffic participants and planning a safe, smooth, and socially compliant trajectory accordingly is crucial for autonomous vehicles. There are two major issues with the current autonomous driving system: the prediction module is often separated from the planning module and the cost function for planning is hard to specify and tune. To tackle these issues, we propose a differentiable integrated prediction-planning framework (DIPP) that can also learn the cost function from data. Specifically, our framework uses a differentiable nonlinear optimizer as the motion planner, which takes as input the predicted trajectories of surrounding agents given by the neural network and optimizes the trajectory for the autonomous vehicle, enabling all operations to be differentiable, including the cost function weights. The proposed framework is trained on a large-scale real-world driving dataset to imitate human driving trajectories in the entire driving scene and validated in both open-loop and closed-loop manners. The open-loop testing results reveal that the proposed method outperforms the baseline methods across a variety of metrics and delivers planning-centric prediction results, allowing the planning module to output trajectories close to those of human drivers. In closed-loop testing, the proposed method outperforms various baseline methods, showing the ability to handle complex urban driving scenarios and robustness against the distributional shift. Importantly, we find that joint training of planning and prediction modules achieves better performance than planning with a separate trained prediction module in both open-loop and closed-loop tests. Moreover, the ablation study indicates that the learnable components in the framework are essential to ensure planning stability and performance.

Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterization

Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectory

Rethinking Imitation-based Planner for Autonomous Driving

Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous Driving

A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning

Integrating Decision-Making Into Differentiable Optimization Guided Learning for End-to-End Planning of Autonomous Vehicles

Learning Hierarchical Behavior and Motion Planning for Autonomous Driving.

Hybrid Imitation-Learning Motion Planner for Urban Driving

Iterative Imitation Policy Improvement for Interactive Autonomous Driving

Evaluation of MPC-based Imitation Learning for Human-like Autonomous Driving

Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving

PILOT: Efficient Planning by Imitation Learning and Optimisation for Safe Autonomous Driving

PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving

End-to-end Driving via Conditional Imitation Learning

Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving

Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving

Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios

Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety

Yaw-Guided Imitation Learning for Autonomous Driving in Urban Environments

EasyChauffeur: A Baseline Advancing Simplicity and Efficiency on Waymax

Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learning