Abstract:Occupancy prediction plays a pivotal role in autonomous driving (AD) due to the fine-grained geometric perception and general object recognition capabilities. However, existing methods often incur high computational costs, which contradicts the real-time demands of AD. To this end, we first evaluate the speed and memory usage of most public available methods, aiming to redirect the focus from solely prioritizing accuracy to also considering efficiency. We then identify a core challenge in achieving both fast and accurate performance: \textbf{the strong coupling between geometry and semantic}. To address this issue, 1) we propose a Geometric-Semantic Dual-Branch Network (GSDBN) with a hybrid BEV-Voxel representation. In the BEV branch, a BEV-level temporal fusion module and a U-Net encoder is introduced to extract dense semantic features. In the voxel branch, a large-kernel re-parameterized 3D convolution is proposed to refine sparse 3D geometry and reduce computation. Moreover, we propose a novel BEV-Voxel lifting module that projects BEV features into voxel space for feature fusion of the two branches. In addition to the network design, 2) we also propose a Geometric-Semantic Decoupled Learning (GSDL) strategy. This strategy initially learns semantics with accurate geometry using ground-truth depth, and then gradually mixes predicted depth to adapt the model to the predicted geometry. Extensive experiments on the widely-used Occ3D-nuScenes benchmark demonstrate the superiority of our method, which achieves a 39.4 mIoU with 20.0 FPS. This result is $\sim 3 \times$ faster and +1.9 mIoU higher compared to FB-OCC, the winner of CVPR2023 3D Occupancy Prediction Challenge. Our code will be made open-source.

PVP: Polar Representation Boost for 3D Semantic Occupancy Prediction

Predicting Polarization Beyond Semantics for Wearable Robotics.

VPOcc: Exploiting Vanishing Point for Monocular 3D Semantic Occupancy Prediction

PARTNER: Level up the Polar Representation for LiDAR 3D Object Detection

PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Prediction

PVO: Panoptic Visual Odometry.

ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction

OPUS: Occupancy Prediction Using a Sparse Set

PolarBEVDet: Exploring Polar Representation for Multi-View 3D Object Detection in Bird's-Eye-View

Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction

Polar Parametrization for Vision-based Surround-View 3D Detection

ProtoOcc: Accurate, Efficient 3D Occupancy Prediction Using Dual Branch Encoder-Prototype Query Decoder

PolarFormer: Multi-camera 3D Object Detection with Polar Transformer

SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Prediction

Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimation

Real-Time 3D Occupancy Prediction via Geometric-Semantic Disentanglement

OccupancyDETR: Using DETR for Mixed Dense-sparse 3D Occupancy Prediction

OccRWKV: Rethinking Efficient 3D Semantic Occupancy Prediction with Linear Complexity

FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective View

MR-Occ: Efficient Camera-LiDAR 3D Semantic Occupancy Prediction Using Hierarchical Multi-Resolution Voxel Representation

SiamPolar: Semi-supervised Realtime Video Object Segmentation with Polar Representation