Abstract:Point cloud-based 3D object tracking is an important task in autonomous driving. Though great advances regarding Siamese-based 3D tracking have been made recently, it remains challenging to learn the correlation between the template and search branches effectively with the sparse LIDAR point cloud data. Instead of performing correlation of the two branches at just one point in the network, in this paper, we present a multi-correlation Siamese Transformer network that has multiple stages and carries out feature correlation at the end of each stage based on sparse pillars. More specifically, in each stage, self-attention is first applied to each branch separately to capture the non-local context information. Then, cross-attention is used to inject the template information into the search area. This strategy allows the feature learning of the search area to be aware of the template while keeping the individual characteristics of the template intact. To enable the network to easily preserve the information learned at different stages and ease the optimization, for the search area, we densely connect the initial input sparse pillars and the output of each stage to all subsequent stages and the target localization network, which converts pillars to bird's eye view (BEV) feature maps and predicts the state of the target with a small densely connected convolution network. Deep supervision is added to each stage to further boost the performance as well. The proposed algorithm is evaluated on the popular KITTI, nuScenes, and Waymo datasets, and the experimental results show that our method achieves promising performance compared with the state-of-the-art. Ablation study that shows the effectiveness of each component is provided as well. Code is available at <a class="link-external link-https" href="https://github.com/liangp/MCSTN-3DSOT" rel="external noopener nofollow">this https URL</a>.

MLVSNet: Multi-level Voting Siamese Network for 3D Visual Tracking

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

PointSiamRCNN: Target-aware Voxel-based Siamese Tracker for Point Clouds

Beyond Traditional Driving Scenes: A Robotic-Centric Paradigm for 2D+3D Human Tracking Using Siamese Transformer Network

GLT-T++: Global-Local Transformer for 3D Siamese Tracking with Ranking Loss

GLT-T: Global-Local Transformer Voting for 3D Single Object Tracking in Point Clouds

TLPG-Tracker: Joint Learning of Target Localization and Proposal Generation for Visual Tracking.

Multi-Correlation Siamese Transformer Network with Dense Connection for 3D Single Object Tracking

The Multi-task Fully Convolutional Siamese Network with Correlation Filter Layer for Real-Time Visual Tracking

OS-DS tracker: Orientation-variant Siamese 3D tracking with Detection based Sampling

Divert More Attention to Vision-Language Tracking

3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds

Integrating Scaling Strategy and Central Guided Voting for 3D Point Cloud Object Tracking

Learning Deep Lucas-Kanade Siamese Network for Visual Tracking

MVCTrack: Boosting 3D Point Cloud Tracking via Multimodal-Guided Virtual Cues

Siamese Residual Network for Efficient Visual Tracking

3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds.

Distractor-aware Siamese Networks for Visual Object Tracking

MLGT: multi-local guided tracker for visual object tracking

Multi-Level Structure-Enhanced Network for 3D Single Object Tracking in Sparse Point Clouds

Local to Global Tracker: A Siamese Network for Long-term Tracking