Abstract:LiDAR-based 3D single object tracking is a challenging issue in robotics and autonomous driving. Currently, existing approaches usually suffer from the problem that objects at long distance often have very sparse or partially-occluded point clouds, which makes the features extracted by the model ambiguous. Ambiguous features will make it hard to locate the target object and finally lead to bad tracking results. To solve this problem, we utilize the powerful Transformer architecture and propose a Point-Track-Transformer (PTT) module for point cloud-based 3D single object tracking task. Specifically, PTT module generates fine-tuned attention features by computing attention weights, which guides the tracker focusing on the important features of the target and improves the tracking ability in complex scenarios. To evaluate our PTT module, we embed PTT into the dominant method and construct a novel 3D SOT tracker named PTT-Net. In PTT-Net, we embed PTT into the voting stage and proposal generation stage, respectively. PTT module in the voting stage could model the interactions among point patches, which learns context-dependent features. Meanwhile, PTT module in the proposal generation stage could capture the contextual information between object and background. We evaluate our PTT-Net on KITTI and NuScenes datasets. Experimental results demonstrate the effectiveness of PTT module and the superiority of PTT-Net, which surpasses the baseline by a noticeable margin, $\sim$10% in the Car category. Meanwhile, our method also has a significant performance improvement in sparse scenarios. In general, the combination of transformer and tracking pipeline enables our PTT-Net to achieve state-of-the-art performance on both two datasets. Additionally, PTT-Net could run in real-time at 40FPS on NVIDIA 1080Ti GPU. Our code is open-sourced for the research community at https://github.com/shanjiayao/PTT.

VTT: Long-term Visual Tracking with Transformers

Exploiting Temporal Coherence for Self-Supervised Visual Tracking by Using Vision Transformer

GTT: Visual Tracking with Gaussion Transformer

MTAtrack: Multilevel Transformer Attention for Visual Tracking

Global Tracking Transformers

Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking

ViTT: Vision Transformer Tracker

AViTMP: A Tracking-Specific Transformer for Single-Branch Visual Tracking

Learning Spatio-Temporal Transformer for Visual Tracking

Visual tracking using transformer with a combination of convolution and attention

Propagating Prior Information with Transformer for Robust Visual Object Tracking

RTSformer: A Robust Toroidal Transformer With Spatiotemporal Features for Visual Tracking

Temporal Relation Transformer for Robust Visual Tracking with Dual-Memory Learning

Compact Transformer Tracker with Correlative Masked Modeling.

CVTrack: Combined Convolutional Neural Network and Vision Transformer Fusion Model for Visual Tracking

Real-Time 3D Single Object Tracking With Transformer

SslTransT: Self-supervised pre-training visual object tracking with Transformers

Target-aware transformer tracking with hard occlusion instance generation

SparseTT: Visual Tracking with Sparse Transformers

GTTrack: Gaussian Transformer Tracker for Visual Tracking.

Transformer-based visual object tracking via fine-coarse concatenated attention and cross concatenated MLP