Abstract:LiDAR-based 3D single object tracking is a challenging issue in robotics and autonomous driving. Currently, existing approaches usually suffer from the problem that objects at long distance often have very sparse or partially-occluded point clouds, which makes the features extracted by the model ambiguous. Ambiguous features will make it hard to locate the target object and finally lead to bad tracking results. To solve this problem, we utilize the powerful Transformer architecture and propose a Point-Track-Transformer (PTT) module for point cloud-based 3D single object tracking task. Specifically, PTT module generates fine-tuned attention features by computing attention weights, which guides the tracker focusing on the important features of the target and improves the tracking ability in complex scenarios. To evaluate our PTT module, we embed PTT into the dominant method and construct a novel 3D SOT tracker named PTT-Net. In PTT-Net, we embed PTT into the voting stage and proposal generation stage, respectively. PTT module in the voting stage could model the interactions among point patches, which learns context-dependent features. Meanwhile, PTT module in the proposal generation stage could capture the contextual information between object and background. We evaluate our PTT-Net on KITTI and NuScenes datasets. Experimental results demonstrate the effectiveness of PTT module and the superiority of PTT-Net, which surpasses the baseline by a noticeable margin, $\sim$10% in the Car category. Meanwhile, our method also has a significant performance improvement in sparse scenarios. In general, the combination of transformer and tracking pipeline enables our PTT-Net to achieve state-of-the-art performance on both two datasets. Additionally, PTT-Net could run in real-time at 40FPS on NVIDIA 1080Ti GPU. Our code is open-sourced for the research community at https://github.com/shanjiayao/PTT.

Accurate 3D Single Object Tracker With Local-to-Global Feature Refinement

PointSiamRCNN: Target-aware Voxel-based Siamese Tracker for Point Clouds

Object tracking with 3D LIDAR via multi-task sparse learning

Exploit Spatiotemporal Contextual Information for 3D Single Object Tracking Via Memory Networks

Beyond Local Search: Tracking Objects Everywhere with Instance-Specific Proposals

3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud

CDTracker: Coarse-to-Fine Feature Matching and Point Densification for 3D Single-Object Tracking

Tracking Randomly Moving Objects on Edge Box Proposals

DeepPCT: Single Object Tracking in Dynamic Point Cloud Sequences

Integrating Scaling Strategy and Central Guided Voting for 3D Point Cloud Object Tracking

OST: Efficient One-stream Network for 3D Single Object Tracking in Point Clouds

EasyTrack: Efficient and Compact One-stream 3D Point Clouds Tracker

TLPG-Tracker: Joint Learning of Target Localization and Proposal Generation for Visual Tracking.

GLT-T: Global-Local Transformer Voting for 3D Single Object Tracking in Point Clouds

Multi-Level Structure-Enhanced Network for 3D Single Object Tracking in Sparse Point Clouds

Small Object Tracking in LiDAR Point Cloud: Learning the Target-awareness Prototype and Fine-grained Search Region

Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds

Real-Time 3D Single Object Tracking With Transformer

Spatio-Temporal Bi-directional Cross-frame Memory for Distractor Filtering Point Cloud Single Object Tracking

STTracker: Spatio-Temporal Tracker for 3D Single Object Tracking