Multi-Object Tracking in the Dark

Xinzhe Wang,Kang Ma,Qiankun Liu,Yunhao Zou,Ying Fu

2024-05-11

Abstract:Low-light scenes are prevalent in real-world applications (e.g. autonomous driving and surveillance at night). Recently, multi-object tracking in various practical use cases have received much attention, but multi-object tracking in dark scenes is rarely considered. In this paper, we focus on multi-object tracking in dark scenes. To address the lack of datasets, we first build a Low-light Multi-Object Tracking (LMOT) dataset. LMOT provides well-aligned low-light video pairs captured by our dual-camera system, and high-quality multi-object tracking annotations for all videos. Then, we propose a low-light multi-object tracking method, termed as LTrack. We introduce the adaptive low-pass downsample module to enhance low-frequency components of images outside the sensor noises. The degradation suppression learning strategy enables the model to learn invariant information under noise disturbance and image quality degradation. These components improve the robustness of multi-object tracking in dark scenes. We conducted a comprehensive analysis of our LMOT dataset and proposed LTrack. Experimental results demonstrate the superiority of the proposed method and its competitiveness in real night low-light scenes. Dataset and Code: https: //github.com/ying-fu/LMOT

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily addresses the issue of multi-object tracking in low-light environments, aiming to solve two core challenges:

1. **Lack of multi-object tracking datasets under low-light conditions**: Existing multi-object tracking datasets are mainly collected under well-lit conditions, while collecting high-quality videos and their annotations in low-light environments is very difficult and costly.

2. **Technical challenges of multi-object tracking under low-light conditions**: In low-light conditions, image quality is poor and noise is high, which directly affects the performance of object detectors and appearance-based association modules, thereby impacting the effectiveness of multi-object tracking.

To address the above issues, the paper proposes the following contributions:

- **Constructed the first multi-object tracking dataset under low-light conditions (LMOT)**: Researchers designed a dual-camera system that can simultaneously capture video frames under well-lit and low-light conditions. This setup allows annotation work to be done on well-lit videos, while these videos can also provide additional supervision information for the model during the training phase. The LMOT dataset contains 32 video sequences, over 35,000 frames, and more than 815,000 bounding boxes.

- **Proposed a multi-object tracking method under low-light conditions (LTrack)**: To improve tracking performance, the LTrack method includes two key components:
- **Adaptive Low-pass Downsampling Module (ALD)**: Enhances feature maps by extracting low-frequency components from images through spatial low-pass convolution and filtering out high-frequency noise.
- **Degradation Suppression Learning Strategy (DSL)**: Utilizes paired low-light videos to help the model suppress image noise and encourage content response in the feature domain, thereby improving the model's robustness to noise.

Experimental results show that LTrack demonstrates superiority and competitiveness in multi-object tracking tasks under low-light conditions, especially in real night scene tests.