Abstract:Facing the significant challenge of 3D object detection in complex weather conditions and road environments, existing algorithms based on single-frame point cloud data struggle to achieve desirable results. These methods typically focus on spatial relationships within a single frame, overlooking the semantic correlations and spatiotemporal continuity between consecutive frames. This leads to discontinuities and abrupt changes in the detection outcomes. To address this issue, this paper proposes a multi-frame 3D object detection algorithm based on a deformable spatiotemporal Transformer. Specifically, a deformable cross-scale Transformer module is devised, incorporating a multi-scale offset mechanism that non-uniformly samples features at different scales, enhancing the spatial information aggregation capability of the output features. Simultaneously, to address the issue of feature misalignment during multi-frame feature fusion, a deformable cross-frame Transformer module is proposed. This module incorporates independently learnable offset parameters for different frame features, enabling the model to adaptively correlate dynamic features across multiple frames and improve the temporal information utilization of the model. A proposal-aware sampling algorithm is introduced to significantly increase the foreground point recall, further optimizing the efficiency of feature extraction. The obtained multi-scale and multi-frame voxel features are subjected to an adaptive fusion weight extraction module, referred to as the proposed mixed voxel set extraction module. This module allows the model to adaptively obtain mixed features containing both spatial and temporal information. The effectiveness of the proposed algorithm is validated on the KITTI, nuScenes, and self-collected urban datasets. The proposed algorithm achieves an average precision improvement of 2.1% over the latest multi-frame-based algorithms.

Anchor-Based Transformer for Temporal LiDAR 3D Object Detection

SEFormer: Structure Embedding Transformer for 3D Object Detection

Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection in Autonomous Driving

Dynamic graph transformer for 3D object detection

AnchorPoint: Query Design for Transformer-Based 3D Object Detection and Tracking

Multi-Scale Spatial Transformer Network for LiDAR-Camera 3D Object Detection.

NSAW: An Efficient and Accurate Transformer for Vehicle LiDAR Object Detection.

Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Clouds

KPTr: Key Point Transformer for LiDAR-based 3D Object Detection

Long-short Range Adaptive Transformer with Dynamic Sampling for 3D Object Detection

3D point cloud object detection algorithm based on Transformer

CenterFormer: Center-based Transformer for 3D Object Detection

SPV-SSD: An Anchor-Free 3D Single-Stage Detector with Supervised-PointRendering and Visibility Representation

DS-Trans: A 3D Object Detection Method Based on a Deformable Spatiotemporal Transformer for Autonomous Vehicles

LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attention

Li3DeTr: A LiDAR based 3D Detection Transformer

OcTr: Octree-based Transformer for 3D Object Detection

An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Clouds

STFormer3D: Spatio-Temporal Transformer Based 3D Object Detection for Intelligent Driving.

HCT-Det: a Hybrid CNN-transformer Architecture for 3D Object Detection from Point Clouds

DVST: Deformable Voxel Set Transformer for 3D Object Detection from Point Clouds