Abstract:3D human tracking plays a crucial role in the automation intelligence system. Current approaches focus on achieving higher performance on traditional driving datasets like KITTI, which overlook the jitteriness of the platform and the complexity of the environments. Once the scenarios are migrated to jolting robot platforms, they all degenerate severely with only a 20-60% success rate, which greatly restricts the high-level application of autonomous systems. In this work, beyond traditional flat scenes, we introduce Multi-modal Human Tracking Paradigm (MHTrack), a unified multimodal transformer-based model that can effectively track the target person frame-by-frame in point and video sequences. Specifically, we design a speed-inertia module-assisted stabilization mechanism along with an alternate training strategy to better migrate the tracking algorithm to the robot platform. To capture more target-aware information, we combine the geometric and appearance features of point clouds and video frames together based on a hierarchical Siamese Transformer Network. Additionally, considering the prior characteristics of the human category, we design a lateral cross-attention pyramid head for deeper feature aggregation and final 3D BBox generation. Extensive experiments confirm that MHTrack significantly outperforms the previous state-of-the-arts on both open-source datasets and large-scale robotic datasets. Further analysis verifies each component’s effectiveness and shows the robotic-centric paradigm’s promising potential when deployed into dynamic robotic systems.

On Automatic Camera Shooting Systems Via PTZ Control and DNN-based Visual Sensing.

On a videoing control system based on object detection and tracking

A Method of Video Analysis in Vehicle Based on A Single Pal Camera

Visual Object Servo Tracking Based on the Particle Filter Method Using a Pan-Tilt-Zoom Camera

Beyond Traditional Driving Scenes: A Robotic-Centric Paradigm for 2D+3D Human Tracking Using Siamese Transformer Network

Capture Of Vehicle Surroundings Using A Pair Of Panoramic Annular Lens Cameras

Semantic Visual Odometry Based on Panoramic Annular Imaging

Real-time Object Tracking System Using PTZ Camera

Building an Intelligent Camera Management System

Pan-and-tilt Self-Portrait System Using Gesture Interface

Research on Camera Rotation Strategies for Active Visual Perception in the Self-Driving Vehicles

A Real-time Image Capture and Processing System for a Wide-area Aerial Camera

Intelligent Object Tracking with an Automatic Image Zoom Algorithm for a Camera Sensing Surveillance System

Autonomous Field-of-View Adjustment Using Adaptive Kinematic Constrained Control with Robot-Held Microscopic Camera Feedback

Automatic Walking Method of Construction Machinery Based on Binocular Camera Environment Perception

Anatomy of a Multicamera Video Surveillance System

Pan-tilt-zoom SLAM for Sports Videos

Vision-Based Detection and Real-Time Adaptive Control Schemes for Autonomous Ultrasound Scanning Robots

Video Surveillance over Wireless Sensor and Actuator Networks Using Active Cameras

Imitation-Based Active Camera Control with Deep Convolutional Neural Network