Abstract:Facing the significant challenge of 3D object detection in complex weather conditions and road environments, existing algorithms based on single-frame point cloud data struggle to achieve desirable results. These methods typically focus on spatial relationships within a single frame, overlooking the semantic correlations and spatiotemporal continuity between consecutive frames. This leads to discontinuities and abrupt changes in the detection outcomes. To address this issue, this paper proposes a multi-frame 3D object detection algorithm based on a deformable spatiotemporal Transformer. Specifically, a deformable cross-scale Transformer module is devised, incorporating a multi-scale offset mechanism that non-uniformly samples features at different scales, enhancing the spatial information aggregation capability of the output features. Simultaneously, to address the issue of feature misalignment during multi-frame feature fusion, a deformable cross-frame Transformer module is proposed. This module incorporates independently learnable offset parameters for different frame features, enabling the model to adaptively correlate dynamic features across multiple frames and improve the temporal information utilization of the model. A proposal-aware sampling algorithm is introduced to significantly increase the foreground point recall, further optimizing the efficiency of feature extraction. The obtained multi-scale and multi-frame voxel features are subjected to an adaptive fusion weight extraction module, referred to as the proposed mixed voxel set extraction module. This module allows the model to adaptively obtain mixed features containing both spatial and temporal information. The effectiveness of the proposed algorithm is validated on the KITTI, nuScenes, and self-collected urban datasets. The proposed algorithm achieves an average precision improvement of 2.1% over the latest multi-frame-based algorithms.

Deformable 3-D Model Based Vehicle Matching with Weighted Hausdorff and EDA in Traffic Surveillance

Regularization Based Iterative Point Match Weighting for Accurate Rigid Transformation Estimation

Rethinking Human Pose Estimation for Autonomous Driving with 3D Event Representations.

From Time to Space: Automatic Annotation of Unmarked Traffic Scene Based on Trajectory Data.

Vision-Based Moving Objects Detection with Background Modeling

Vehicle Detection Based on Deep Dual-Vehicle Deformable Part Models

Multiple-Kernel Based Vehicle Tracking Using 3D Deformable Model and Camera Self-Calibration

Research on Recognition and Classification of Moving Objects in Mixed Traffic Based on Video Detection

Application of Image Processing and Three-Dimensional Data Reconstruction Algorithm Based on Traffic Video in Vehicle Component Detection

Image Guidance Based 3D Vehicle Detection in Traffic Scene.

3D Extended Object Tracking by Fusing Roadside Sparse Radar Point Clouds and Pixel Keypoints

Weighted Deformable Part Model for Robust Human Detection

Double-pedestrian deformable part model and its application in pedestrian detection

An Efficient Wide-Range Pseudo-3D Vehicle Detection Using A Single Camera

Anomaly Detection in Traffic Scenes via Spatial-aware Motion Reconstruction

Joint 3-D Shape Estimation and Landmark Localization from Monocular Cameras of Intelligent Vehicles

Self-supervised Learning of Implicit Shape Representation with Dense Correspondence for Deformable Objects

High Efficient Moving Object Extraction and Classification in Traffic Video Surveillance

Automatic Outer Contour Detection and Quantification of Vehicles Using Monocular Vision

DS-Trans: A 3D Object Detection Method Based on a Deformable Spatiotemporal Transformer for Autonomous Vehicles

Vehicle Detection by Fusing Part Model Learning and Semantic Scene Information for Complex Urban Surveillance