Abstract:In this paper, we propose a new paradigm, named Historical Object Prediction (HoP) for multi-view 3D detection to leverage temporal information more effectively. The HoP approach is straightforward: given the current timestamp t, we generate a pseudo Bird's-Eye View (BEV) feature of timestamp t-k from its adjacent frames and utilize this feature to predict the object set at timestamp t-k. Our approach is motivated by the observation that enforcing the detector to capture both the spatial location and temporal motion of objects occurring at historical timestamps can lead to more accurate BEV feature learning. First, we elaborately design short-term and long-term temporal decoders, which can generate the pseudo BEV feature for timestamp t-k without the involvement of its corresponding camera images. Second, an additional object decoder is flexibly attached to predict the object targets using the generated pseudo BEV feature. Note that we only perform HoP during training, thus the proposed method does not introduce extra overheads during inference. As a plug-and-play approach, HoP can be easily incorporated into state-of-the-art BEV detection frameworks, including BEVFormer and BEVDet series. Furthermore, the auxiliary HoP approach is complementary to prevalent temporal modeling methods, leading to significant performance gains. Extensive experiments are conducted to evaluate the effectiveness of the proposed HoP on the nuScenes dataset. We choose the representative methods, including BEVFormer and BEVDet4D-Depth to evaluate our method. Surprisingly, HoP achieves 68.5% NDS and 62.4% mAP with ViT-L on nuScenes test, outperforming all the 3D object detectors on the leaderboard. Codes will be available at https://github.com/Sense-X/HoP.

BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection

BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereo

BEVFusion4D: Learning LiDAR-Camera Fusion Under Bird's-Eye-View via Cross-Modality Guidance and Temporal Aggregation

OCBEV: Object-Centric BEV Transformer for Multi-View 3D Object Detection

A Streamlined Framework for Bev-Based 3d Object Detection with Prior Masking

BEVStereo++: Accurate Depth Estimation in Multi-view 3D Object Detection via Dynamic Temporal Stereo

SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videos

BEVUDA: Multi-geometric Space Alignments for Domain Adaptive BEV 3D Object Detection

BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving

Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Prediction

BEVStereo: Enhancing Depth Estimation in Multi-View 3D Object Detection with Temporal Stereo

TSC-BEV: Temporal-Spatial Feature Consistency 3D Object Detection

Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detection

BEVHeight++: Toward Robust Visual Centric 3D Object Detection

BEVDepth: Acquisition of Reliable Depth for Multi-View 3D Object Detection

RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection

BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection