Abstract:Predicting the future states of surrounding traffic participants and planning a safe, smooth, and socially compliant trajectory accordingly is crucial for autonomous vehicles. There are two major issues with the current autonomous driving system: the prediction module is often separated from the planning module and the cost function for planning is hard to specify and tune. To tackle these issues, we propose a differentiable integrated prediction-planning framework (DIPP) that can also learn the cost function from data. Specifically, our framework uses a differentiable nonlinear optimizer as the motion planner, which takes as input the predicted trajectories of surrounding agents given by the neural network and optimizes the trajectory for the autonomous vehicle, enabling all operations to be differentiable, including the cost function weights. The proposed framework is trained on a large-scale real-world driving dataset to imitate human driving trajectories in the entire driving scene and validated in both open-loop and closed-loop manners. The open-loop testing results reveal that the proposed method outperforms the baseline methods across a variety of metrics and delivers planning-centric prediction results, allowing the planning module to output trajectories close to those of human drivers. In closed-loop testing, the proposed method outperforms various baseline methods, showing the ability to handle complex urban driving scenarios and robustness against the distributional shift. Importantly, we find that joint training of planning and prediction modules achieves better performance than planning with a separate trained prediction module in both open-loop and closed-loop tests. Moreover, the ablation study indicates that the learnable components in the framework are essential to ensure planning stability and performance.

Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations

QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Driving

Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehicles

End to End Autonomous Driving Via Occupancy and Motion Flow

Hybrid Imitation-Learning Motion Planner for Urban Driving

Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learning

Perceive, Attend, and Drive: Learning Spatial Attention for Safe Self-Driving

Safe Real-World Autonomous Driving by Learning to Predict and Plan with a Mixture of Experts

Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving

Planning by Simulation: Motion Planning with Learning-based Parallel Scenario Prediction for Autonomous Driving

Enhance Planning with Physics-informed Safety Controller for End-to-end Autonomous Driving

Towards Safe Motion Planning for Autonomous Driving in Highway

Integrating Decision-Making Into Differentiable Optimization Guided Learning for End-to-End Planning of Autonomous Vehicles

End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving

Safe Output Feedback Motion Planning from Images via Learned Perception Modules and Contraction Theory

Attention-based Highway Safety Planner for Autonomous Driving Via Deep Reinforcement Learning

Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving

Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving

Behavior and Interaction-aware Motion Planning for Autonomous Driving Vehicles based on Hierarchical Intention and Motion Prediction

Vision-Based Uncertainty-Aware Motion Planning based on Probabilistic Semantic Segmentation