Abstract:Facing the significant challenge of 3D object detection in complex weather conditions and road environments, existing algorithms based on single-frame point cloud data struggle to achieve desirable results. These methods typically focus on spatial relationships within a single frame, overlooking the semantic correlations and spatiotemporal continuity between consecutive frames. This leads to discontinuities and abrupt changes in the detection outcomes. To address this issue, this paper proposes a multi-frame 3D object detection algorithm based on a deformable spatiotemporal Transformer. Specifically, a deformable cross-scale Transformer module is devised, incorporating a multi-scale offset mechanism that non-uniformly samples features at different scales, enhancing the spatial information aggregation capability of the output features. Simultaneously, to address the issue of feature misalignment during multi-frame feature fusion, a deformable cross-frame Transformer module is proposed. This module incorporates independently learnable offset parameters for different frame features, enabling the model to adaptively correlate dynamic features across multiple frames and improve the temporal information utilization of the model. A proposal-aware sampling algorithm is introduced to significantly increase the foreground point recall, further optimizing the efficiency of feature extraction. The obtained multi-scale and multi-frame voxel features are subjected to an adaptive fusion weight extraction module, referred to as the proposed mixed voxel set extraction module. This module allows the model to adaptively obtain mixed features containing both spatial and temporal information. The effectiveness of the proposed algorithm is validated on the KITTI, nuScenes, and self-collected urban datasets. The proposed algorithm achieves an average precision improvement of 2.1% over the latest multi-frame-based algorithms.

Precise Spatial Transformation Mechanism for Small-Size-Aware Roadside 3D Object Detection on Traffic Surveillance Cameras

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection

HeightFormer: A Semantic Alignment Monocular 3D Object Detection Method from Roadside Perspective

BEVHeight++: Toward Robust Visual Centric 3D Object Detection

Roadside Monocular 3D Detection via 2D Detection Prompting

Towards Scenario Generalization for Vision-based Roadside 3D Object Detection

SGV3D:Towards Scenario Generalization for Vision-based Roadside 3D Object Detection

DS-Trans: A 3D Object Detection Method Based on a Deformable Spatiotemporal Transformer for Autonomous Vehicles

Towards Efficient 3D Object Detection in Bird's-Eye-View Space for Autonomous Driving: A Convolutional-Only Approach

Real-Time And Robust 3D Object Detection with Roadside LiDARs

Automatic Background Construction and Object Detection Based on Roadside LiDAR

Spatial Information Enhancement with Multi-Scale Feature Aggregation for Long-Range Object and Small Reflective Area Object Detection from Point Cloud

BEVSpread: Spread Voxel Pooling for Bird's-Eye-View Representation in Vision-based Roadside 3D Object Detection

3D Extended Object Tracking by Fusing Roadside Sparse Radar Point Clouds and Pixel Keypoints

Real-Time 3D Object Detection on Crowded Pedestrians

BEVSpread: Spread Voxel Pooling for Bird’s-Eye-View Representation in Vision-based Roadside 3D Object Detection

Real-time 3D Traffic Cone Detection for Autonomous Driving

MOSE: Boosting Vision-based Roadside 3D Object Detection with Scene Cues

3D Object Detection for Point Cloud in Virtual Driving Environment