Abstract:Multi-object tracking (MOT) in the scenario of low-frame-rate videos is a promising solution to better meet the computing, storage, and transmitting bandwidth resource constraints of edge devices. Tracking with a low frame rate poses particular challenges in the association stage as objects in two successive frames typically exhibit much quicker variations in locations, velocities, appearances, and visibilities than those in normal frame rates. In this paper, we observe severe performance degeneration of many existing association strategies caused by such variations. Though optical-flow-based methods like CenterTrack can handle the large displacement to some extent due to their large receptive field, the temporally local nature makes them fail to give reliable displacement estimations of objects that newly appear in the current frame (i.e., not visible in the previous frame). To overcome the local nature of optical-flow-based methods, we propose an online tracking method by extending the CenterTrack architecture with a new head, named APP, to recognize unreliable displacement estimations. Further, to capture the fine-grained and private unreliability of each displacement estimation, we extend the binary APP predictions to displacement uncertainties. To this end, we reformulate the displacement estimation task via Bayesian deep learning tools. With APP predictions, we propose to conduct association in a multi-stage manner where vision cues or historical motion cues are leveraged in the corresponding stage. By rethinking the commonly used bipartite matching algorithms, we equip the proposed multi-stage association policy with a hybrid matching strategy conditioned on displacement uncertainties. Our method shows robustness in preserving identities in low-frame-rate video sequences. Experimental results on public datasets in various low-frame-rate settings demonstrate the advantages of the proposed method.

Inertia Sensor Aided Alignment for Burst Pipeline in Low Light Conditions

Image Deblurring Utilizing Inertial Sensors and a Short-Long-Short Exposure Strategy.

APPTracker Plus : Displacement Uncertainty for Occlusion Handling in Low-Frame-Rate Multiple Object Tracking

A Plug-and-play Demosaicing and Denoising Method on Mobile Cameras

Burst photography for high dynamic range and low-light imaging on mobile cameras

Relative Depth Estimation With An Uncalibrated Camera For Image Refocus

Fine Dense Alignment of Image Bursts through Camera Pose and Depth Estimation

Inertial Sensors Aided Image Alignment and Stitching for Panorama on Mobile Phones

Dual-Camera Joint Deblurring-Denoising

A Joint Visual-Inertial Image Registration For Mobile Hdr Imaging

Progressive Joint Low-Light Enhancement and Noise Removal for Raw Images

CANDID: Correspondence AligNment for Deep-burst Image Denoising

Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Network

Line-preserving video stitching for asymmetric cameras

A Fast and Automatic Fusion Algorithm for Unregistered Multi-Exposure Image Sequence

Hybrid Synthesis for Exposure Fusion from Hand-Held Camera Inputs

A Bio-Inspired Multi-Exposure Fusion Framework for Low-light Image Enhancement

Burst Imaging for Light-Constrained Structure-From-Motion

Exposure Fusion for Hand-held Camera Inputs with Optical Flow and PatchMatch

Benchmark Evaluation of Image Fusion algorithms for Smartphone Camera Capture

Robust Depth Estimation from Auto Bracketed Images