Abstract:Learning robust feature matching between the template and search area is crucial for 3-D Siamese tracking. The core of Siamese feature matching is how to assign high feature similarity to the corresponding points between the template and the search area for precise object localization. In this article, we propose a novel point cloud registration-driven Siamese tracking framework, with the intuition that spatially aligned corresponding points (via 3-D registration) tend to achieve consistent feature representations. Specifically, our method consists of two modules, including a tracking-specific nonlocal registration (TSNR) module and a registration-aided Sinkhorn template-feature aggregation module. The registration module targets the precise spatial alignment between the template and the search area. The tracking-specific spatial distance constraint is proposed to refine the cross-attention weights in the nonlocal module for discriminative feature learning. Then, we use the weighted singular value decomposition (SVD) to compute the rigid transformation between the template and the search area and align them to achieve the desired spatially aligned corresponding points. For the feature aggregation model, we formulate the feature matching between the transformed template and the search area as an optimal transport problem and utilize the Sinkhorn optimization to search for the outlier-robust matching solution. Also, a registration-aided spatial distance map is built to improve the matching robustness in indistinguishable regions (e.g., smooth surfaces). Finally, guided by the obtained feature matching map, we aggregate the target information from the template into the search area to construct the target-specific feature, which is then fed into a CenterPoint-like detection head for object localization. Extensive experiments on KITTI, NuScenes, and Waymo datasets verify the effectiveness of our proposed method.

Point Siamese Network for Person Tracking Using 3D Point Clouds.

PointSiamRCNN: Target-aware Voxel-based Siamese Tracker for Point Clouds

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Background-aware Siamese Network Tracking Based on Salient Feature Fusion

Beyond Traditional Driving Scenes: A Robotic-Centric Paradigm for 2D+3D Human Tracking Using Siamese Transformer Network

3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud

3D Single Object Tracking Network Based on Point Cloud Pre-segmentation

PointTrackNet: An End-to-End Network For 3-D Object Detection and Tracking From Point Clouds

Multi-Correlation Siamese Transformer Network with Dense Connection for 3D Single Object Tracking

3D Siamese Transformer Network for Single Object Tracking on Point Clouds

OS-DS tracker: Orientation-variant Siamese 3D tracking with Detection based Sampling

Object Preserving Siamese Network for Single Object Tracking on Point Clouds

EasyTrack: Efficient and Compact One-stream 3D Point Clouds Tracker

OSP2B: One-Stage Point-to-Box Network for 3D Siamese Tracking

Siamese Centerness Prediction Network for Real-Time Visual Object Tracking

Point Cloud Registration-Driven Robust Feature Matching for 3-D Siamese Object Tracking

Online Multiple Person Tracking Using Fully-Convolutional Neural Networks and Motion Invariance Constraints

OST: Efficient One-stream Network for 3D Single Object Tracking in Point Clouds

Siamese Attentional Cascade Keypoints Network for Visual Object Tracking

Object-Preserving Siamese Network for Single-Object Tracking on Point Clouds