Abstract:LiDAR-based 3D single object tracking is a challenging issue in robotics and autonomous driving. Currently, existing approaches usually suffer from the problem that objects at long distance often have very sparse or partially-occluded point clouds, which makes the features extracted by the model ambiguous. Ambiguous features will make it hard to locate the target object and finally lead to bad tracking results. To solve this problem, we utilize the powerful Transformer architecture and propose a Point-Track-Transformer (PTT) module for point cloud-based 3D single object tracking task. Specifically, PTT module generates fine-tuned attention features by computing attention weights, which guides the tracker focusing on the important features of the target and improves the tracking ability in complex scenarios. To evaluate our PTT module, we embed PTT into the dominant method and construct a novel 3D SOT tracker named PTT-Net. In PTT-Net, we embed PTT into the voting stage and proposal generation stage, respectively. PTT module in the voting stage could model the interactions among point patches, which learns context-dependent features. Meanwhile, PTT module in the proposal generation stage could capture the contextual information between object and background. We evaluate our PTT-Net on KITTI and NuScenes datasets. Experimental results demonstrate the effectiveness of PTT module and the superiority of PTT-Net, which surpasses the baseline by a noticeable margin, $\sim$10% in the Car category. Meanwhile, our method also has a significant performance improvement in sparse scenarios. In general, the combination of transformer and tracking pipeline enables our PTT-Net to achieve state-of-the-art performance on both two datasets. Additionally, PTT-Net could run in real-time at 40FPS on NVIDIA 1080Ti GPU. Our code is open-sourced for the research community at https://github.com/shanjiayao/PTT.

P2T: Part-to-Target Tracking Via Deep Regression Learning

Exploit Spatiotemporal Contextual Information for 3D Single Object Tracking Via Memory Networks

Fast-Tracker 2.0: Improving Autonomy of Aerial Tracking with Active Vision and Human Location Regression.

Deep Relative Tracking.

TLPG-Tracker: Joint Learning of Target Localization and Proposal Generation for Visual Tracking.

Target Transformed Regression for Accurate Tracking

Target-Aware Deep Tracking

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Part-Based Visual Tracking with Online Latent Structural Learning

Part-Based Robust Tracking Using Online Latent Structured Learning

Robust Visual Tracking Method via Deep Learning

Target-Aware Tracking with Long-term Context Attention

CAT: Corner Aided Tracking With Deep Regression Network

SMART: Joint Sampling and Regression for Visual Tracking.

UnifiedTT: Visual Tracking with unified Transformer

DeepPCT: Single Object Tracking in Dynamic Point Cloud Sequences

Memory network for tracking with deep regression

Real-Time 3D Single Object Tracking With Transformer

Adaptive Part Mining for Robust Visual Tracking.

Partial Occlusion Handling for Visual Tracking Via Robust Part Matching

Multi-Person Articulated Tracking With Spatial and Temporal Embeddings