Abstract:3D human tracking plays a crucial role in the automation intelligence system. Current approaches focus on achieving higher performance on traditional driving datasets like KITTI, which overlook the jitteriness of the platform and the complexity of the environments. Once the scenarios are migrated to jolting robot platforms, they all degenerate severely with only a 20-60% success rate, which greatly restricts the high-level application of autonomous systems. In this work, beyond traditional flat scenes, we introduce Multi-modal Human Tracking Paradigm (MHTrack), a unified multimodal transformer-based model that can effectively track the target person frame-by-frame in point and video sequences. Specifically, we design a speed-inertia module-assisted stabilization mechanism along with an alternate training strategy to better migrate the tracking algorithm to the robot platform. To capture more target-aware information, we combine the geometric and appearance features of point clouds and video frames together based on a hierarchical Siamese Transformer Network. Additionally, considering the prior characteristics of the human category, we design a lateral cross-attention pyramid head for deeper feature aggregation and final 3D BBox generation. Extensive experiments confirm that MHTrack significantly outperforms the previous state-of-the-arts on both open-source datasets and large-scale robotic datasets. Further analysis verifies each component’s effectiveness and shows the robotic-centric paradigm’s promising potential when deployed into dynamic robotic systems.

Optical coherence tomography findings in multiple evanescent white dot syndrome.

Exploit Spatiotemporal Contextual Information for 3D Single Object Tracking Via Memory Networks

Object tracking with 3D LIDAR via multi-task sparse learning

Beyond Traditional Driving Scenes: A Robotic-Centric Paradigm for 2D+3D Human Tracking Using Siamese Transformer Network

Joint Monocular 3D Vehicle Detection and Tracking

Object-Level Pseudo-3D Lifting for Distance-Aware Tracking

Tracking Objects with 3D Representation from Videos

Delving into Motion-Aware Matching for Monocular 3D Object Tracking

T-2 T-1 T Frame Input Image Region Proposals Monocular 3 D Estimation Deep Association Multi-frame Refinement

SeqTrack3D: Exploring Sequence Information for Robust 3D Point Cloud Tracking

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving

Tracking Objects as Points

Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds

MCTrack: A Unified 3D Multi-Object Tracking Framework for Autonomous Driving

Dynamic Object Tracking for Self-Driving Cars Using Monocular Camera and LIDAR.

MUTR3D: A Multi-camera Tracking Framework Via 3D-to-2d Queries

CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusion

EasyTrack: Efficient and Compact One-stream 3D Point Clouds Tracker

ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Box