Abstract:For autonomous vehicles to proactively plan safe trajectories and make informed decisions, they must be able to predict the future occupancy states of the local environment. However, common issues with occupancy prediction include predictions where moving objects vanish or become blurred, particularly at longer time horizons. We propose an environment prediction framework that incorporates environment semantics for future occupancy prediction. Our method first semantically segments the environment and uses this information along with the occupancy information to predict the spatiotemporal evolution of the environment. We validate our approach on the real-world Waymo Open Dataset. Compared to baseline methods, our model has higher prediction accuracy and is capable of maintaining moving object appearances in the predictions for longer prediction time horizons.

What problem does this paper attempt to address?

This paper addresses the problem of environmental prediction in autonomous driving. Current methods for environment prediction face challenges such as moving object disappearance or blurring, especially over long prediction time ranges. The paper proposes an environment prediction framework that combines semantic information to predict future spatiotemporal occupancy grids, aiming to improve the safety trajectory planning and decision-making capabilities of autonomous vehicles. The method involves first performing semantic segmentation on the environment, and then using this information along with occupancy information to predict the spatiotemporal evolution of the environment. The proposed method is validated on the Waymo Open Dataset and shows higher prediction accuracy compared to baseline methods, particularly in long-term predictions by better tracking the appearance of moving objects. The main contribution of the paper is the introduction of Semantic Grid Maps (SMGMs), which incorporate semantic information into the occupancy prediction framework. By internally predicting future environmental semantics and passing this information to the occupancy prediction module, the accuracy of the predictions is improved. Compared to a binary architecture that only distinguishes between static and dynamic objects, this approach is able to better learn different motion models of different categories of dynamic objects such as vehicles, cyclists, and pedestrians. In the experimental section, the paper uses the Waymo Open Dataset for quantitative and qualitative evaluations, demonstrating that the proposed model outperforms baseline models based solely on occupancy information or environmental dynamics in terms of prediction accuracy, image similarity, and dynamic occupancy probability error. Additionally, the study explores the influence of semantic categories on prediction results.

Predicting Future Spatiotemporal Occupancy Grids with Semantics for Autonomous Driving

Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving

Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling

Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps

Flow-guided Motion Prediction with Semantics and Dynamic Occupancy Grid Maps

StopNet: Scalable Trajectory and Occupancy Prediction for Urban Autonomous Driving

Real-Time 3D Occupancy Prediction via Geometric-Semantic Disentanglement

Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving

AdaptiveOcc: Adaptive Octree-based Network for Multi-Camera 3D Semantic Occupancy Prediction in Autonomous Driving

AdaOcc: Adaptive-Resolution Occupancy Prediction

Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving

Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning

Hybrid-Prediction Integrated Planning for Autonomous Driving

LOPR: Latent Occupancy PRediction using Generative Models

Towards Safe and Reliable Autonomous Driving: Dynamic Occupancy Set Prediction

UnO: Unsupervised Occupancy Fields for Perception and Forecasting

Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehicles

Learning-based 3D Occupancy Prediction for Autonomous Navigation in Occluded Environments

StreamingFlow: Streaming Occupancy Forecasting with Asynchronous Multi-modal Data Streams via Neural Ordinary Differential Equation

Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting

Occupancy Map Prediction Using Generative and Fully Convolutional Networks for Vehicle Navigation