Abstract:Facing the significant challenge of 3D object detection in complex weather conditions and road environments, existing algorithms based on single-frame point cloud data struggle to achieve desirable results. These methods typically focus on spatial relationships within a single frame, overlooking the semantic correlations and spatiotemporal continuity between consecutive frames. This leads to discontinuities and abrupt changes in the detection outcomes. To address this issue, this paper proposes a multi-frame 3D object detection algorithm based on a deformable spatiotemporal Transformer. Specifically, a deformable cross-scale Transformer module is devised, incorporating a multi-scale offset mechanism that non-uniformly samples features at different scales, enhancing the spatial information aggregation capability of the output features. Simultaneously, to address the issue of feature misalignment during multi-frame feature fusion, a deformable cross-frame Transformer module is proposed. This module incorporates independently learnable offset parameters for different frame features, enabling the model to adaptively correlate dynamic features across multiple frames and improve the temporal information utilization of the model. A proposal-aware sampling algorithm is introduced to significantly increase the foreground point recall, further optimizing the efficiency of feature extraction. The obtained multi-scale and multi-frame voxel features are subjected to an adaptive fusion weight extraction module, referred to as the proposed mixed voxel set extraction module. This module allows the model to adaptively obtain mixed features containing both spatial and temporal information. The effectiveness of the proposed algorithm is validated on the KITTI, nuScenes, and self-collected urban datasets. The proposed algorithm achieves an average precision improvement of 2.1% over the latest multi-frame-based algorithms.

Improved Deformable Part Model for Object Detection Based on Scale Invariant Feature Transform

An Empirical Study of Deformable Part Model with Fast Feature Pyramid

Deformable Part-Based Model Transfer for Object Detection.

An improved object detection algorithm based on multi-scaled and deformable convolutional neural networks

Active Deformable Part Models

DSN: A New Deformable Subnetwork for Object Detection

Object Tracking with Embedded Deformable Parts in Dynamic Conditional Random Fields

Enhanced Deformable Part Model for Pedestrian Detection Via Joint State Inference

Assemble New Object Detector With Transferred Auxiliary Filters And Distinct Target Filters

A DPM Based Approach to Joint Object Detection and Sub-category Recognition

DS-Trans: A 3D Object Detection Method Based on a Deformable Spatiotemporal Transformer for Autonomous Vehicles

Balanced Mixture of Deformable Part Models with Automatic Part Configurations.

FFD: Fast Feature Detector

Deformable Feature Alignment and Refinement for Moving Infrared Dim-small Target Detection

Deformable DETR: Deformable Transformers for End-to-End Object Detection

Robust Object Tracking Combining Color and Scale Invariant Features

Towards Better Object Detection in Scale Variation with Adaptive Feature Selection

Object Detection with Discriminatively Trained Part Based Models

Small Object Detection by DETR via Information Augmentation and Adaptive Feature Fusion

An effective method for small objects detection based on MDFFAM and LKSPP

Multiple Instance Ffeature for Robust Part-Based Object Detection