Abstract:As a novel 3D scene representation, semantic occupancy has gained much attention in autonomous driving. However, existing occupancy prediction methods mainly focus on designing better occupancy representations, such as tri-perspective view or neural radiance fields, while ignoring the advantages of using long-temporal information. In this paper, we propose a radar-camera multi-modal temporal enhanced occupancy prediction network, dubbed TEOcc. Our method is inspired by the success of utilizing temporal information in 3D object detection. Specifically, we introduce a temporal enhancement branch to learn temporal occupancy prediction. In this branch, we randomly discard the t-k input frame of the multi-view camera and predict its 3D occupancy by long-term and short-term temporal decoders separately with the information from other adjacent frames and multi-modal inputs. Besides, to reduce computational costs and incorporate multi-modal inputs, we specially designed 3D convolutional layers for long-term and short-term temporal decoders. Furthermore, since the lightweight occupancy prediction head is a dense classification head, we propose to use a shared occupancy prediction head for the temporal enhancement and main branches. It is worth noting that the temporal enhancement branch is only performed during training and is discarded during inference. Experiment results demonstrate that TEOcc achieves state-of-the-art occupancy prediction on nuScenes benchmarks. In addition, the proposed temporal enhancement branch is a plug-and-play module that can be easily integrated into existing occupancy prediction methods to improve the performance of occupancy prediction. The code and models will be released at <a class="link-external link-https" href="https://github.com/VDIGPKU/TEOcc" rel="external noopener nofollow">this https URL</a>.

Learning-based 3D Occupancy Prediction for Autonomous Navigation in Occluded Environments

Occupancy Map Prediction Using Generative and Fully Convolutional Networks for Vehicle Navigation

Learning Occupancy for Monocular 3D Object Detection

SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction

The Foreseeable Future: Self-Supervised Learning to Predict Dynamic Scenes for Indoor Navigation

OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments

Real-Time 3D Occupancy Prediction via Geometric-Semantic Disentanglement

Learning Occupancy Priors of Human Motion from Semantic Maps of Urban Environments

Online Diffusion-Based 3D Occupancy Prediction at the Frontier with Probabilistic Map Reconciliation

Fully Sparse 3D Occupancy Prediction

Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving

EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understanding

AdaptiveOcc: Adaptive Octree-based Network for Multi-Camera 3D Semantic Occupancy Prediction in Autonomous Driving

Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction

OPUS: Occupancy Prediction Using a Sparse Set

Occlusion-Aware Crowd Navigation Using People as Sensors

SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving

3Dopformer: 3D Occupancy Perception from Multi-Camera Images with Directional and Distance Enhancement

TEOcc: Radar-camera Multi-modal Occupancy Prediction via Temporal Enhancement