Abstract:Predicting the future states of surrounding traffic participants and planning a safe, smooth, and socially compliant trajectory accordingly is crucial for autonomous vehicles. There are two major issues with the current autonomous driving system: the prediction module is often separated from the planning module and the cost function for planning is hard to specify and tune. To tackle these issues, we propose a differentiable integrated prediction-planning framework (DIPP) that can also learn the cost function from data. Specifically, our framework uses a differentiable nonlinear optimizer as the motion planner, which takes as input the predicted trajectories of surrounding agents given by the neural network and optimizes the trajectory for the autonomous vehicle, enabling all operations to be differentiable, including the cost function weights. The proposed framework is trained on a large-scale real-world driving dataset to imitate human driving trajectories in the entire driving scene and validated in both open-loop and closed-loop manners. The open-loop testing results reveal that the proposed method outperforms the baseline methods across a variety of metrics and delivers planning-centric prediction results, allowing the planning module to output trajectories close to those of human drivers. In closed-loop testing, the proposed method outperforms various baseline methods, showing the ability to handle complex urban driving scenarios and robustness against the distributional shift. Importantly, we find that joint training of planning and prediction modules achieves better performance than planning with a separate trained prediction module in both open-loop and closed-loop tests. Moreover, the ablation study indicates that the learnable components in the framework are essential to ensure planning stability and performance.

Safe Real-World Autonomous Driving by Learning to Predict and Plan with a Mixture of Experts

Towards Safe Motion Planning for Autonomous Driving in Highway

Motion Planning for Autonomous Driving with Real Traffic Data Validation

Planning by Simulation: Motion Planning with Learning-based Parallel Scenario Prediction for Autonomous Driving

Self-Aware Trajectory Prediction for Safe Autonomous Driving

An Integrating Comprehensive Trajectory Prediction with Risk Potential Field Method for Autonomous Driving

Enhance Planning with Physics-informed Safety Controller for End-to-end Autonomous Driving

Long-Tail Prediction Uncertainty Aware Trajectory Planning for Self-driving Vehicles.

Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous Driving

Trajectory Planning and Safety Assessment of Autonomous Vehicles Based on Motion Prediction and Model Predictive Control.

Safe Trajectory Planning Using Reinforcement Learning for Self Driving

Hybrid Imitation-Learning Motion Planner for Urban Driving

Risk-Aware Neural Navigation From BEV Input for Interactive Driving

Predictive Trajectory Planning for On-Road Autonomous Vehicles Based on a Spatiotemporal Risk Field

Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving

Hierarchical Learned Risk-Aware Planning Framework for Human Driving Modeling

PLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous driving

ObserveNet Control: A Vision-Dynamics Learning Approach to Predictive Control in Autonomous Vehicles

A Learning-Based Model Predictive Trajectory Planning Controller for Automated Driving in Unstructured Dynamic Environments

Intention-based and Risk-Aware Trajectory Prediction for Autonomous Driving in Complex Traffic Scenarios

Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehicles