HazyDet: Open-source Benchmark for Drone-view Object Detection with Depth-cues in Hazy Scenes

Changfeng Feng,Zhenyuan Chen,Renke Kou,Guangwei Gao,Chunping Wang,Xiang Li,Xiangbo Shu,Yimian Dai,Qiang Fu,Jian Yang

2024-09-30

Abstract:Drone-based object detection in adverse weather conditions is crucial for enhancing drones' environmental perception, yet it remains largely unexplored due to the lack of relevant benchmarks. To bridge this gap, we introduce HazyDet, a large-scale dataset tailored for drone-based object detection in hazy scenes. It encompasses 383,000 real-world instances, collected from both naturally hazy environments and normal scenes with synthetically imposed haze effects to simulate adverse weather conditions. By observing the significant variations in object scale and clarity under different depth and haze conditions, we designed a Depth Conditioned Detector (DeCoDet) to incorporate this prior knowledge. DeCoDet features a Multi-scale Depth-aware Detection Head that seamlessly integrates depth perception, with the resulting depth cues harnessed by a dynamic Depth Condition Kernel module. Furthermore, we propose a Scale Invariant Refurbishment Loss to facilitate the learning of robust depth cues from pseudo-labels. Extensive evaluations on the HazyDet dataset demonstrate the flexibility and effectiveness of our method, yielding significant performance improvements. Our dataset and toolkit are available at <a class="link-external link-https" href="https://github.com/GrokCV/HazyDet" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to address the challenges faced by unmanned aerial vehicles (UAVs) in object detection under adverse weather conditions, especially in hazy weather. Specifically, the paper proposes solutions to the following problems:

1. **Lack of object - detection benchmark datasets suitable for the UAV perspective**: Most of the existing datasets mainly focus on ideal, sunny weather conditions and cannot fully reflect the actual application scenarios of UAVs in adverse weather such as haze.

2. **Specialties under the UAV perspective**:
- **Scale variation**: The size of objects in images captured by UAVs can vary significantly due to changes in altitude and angle, resulting in an increase in the proportion of small objects.
- **Uneven distribution**: The distribution of objects in UAV images is irregular, which is different from the concentrated distribution under the normal perspective.

3. **The impact of adverse weather conditions on image quality**:
- **Image degradation**: Hazy weather can reduce the atmospheric transmission efficiency, leading to a decrease in image visibility and color distortion, which affects subsequent vision - based perception tasks.
- **Domain gap**: Image degradation caused by weather can affect feature recognition, making it difficult for neural networks to process blurred and semantically ambiguous features, thus resulting in a large domain gap.

To solve these problems, the author proposes the following solutions:

- **HazyDet dataset**: A large - scale dataset has been constructed, containing 383,000 real - world instances, covering normal scenes in natural hazy environments and with synthetic hazy effects. This dataset fills the resource gap for UAVs in object detection under adverse weather.

- **Depth - information - guided detection framework (DeCoDet)**: A new detection framework has been introduced, which uses depth information to improve the detection performance of UAVs in hazy conditions. This framework effectively copes with the challenges brought by the UAV perspective and hazy environment by dynamically adjusting the detection strategy based on depth cues.

- **Benchmark testing and leaderboard**: A comprehensive quantitative and qualitative evaluation of the state - of - the - art detection and defogging methods has been carried out using the HazyDet dataset, and a benchmark and leaderboard have been established, providing a platform for the research community to understand the limitations of existing methods and develop robust solutions.

### Formula representation

The formulas involved in the paper are as follows:

- The classic formula for hazy image generation:
\[
I(x, y) = J(x, y)t(x, y)+A(1 - t(x, y))
\]
where \(I(x, y)\) is the observed hazy image, \(J(x, y)\) is the recoverable scene radiance, \(A\) represents the global atmospheric light, and \(t(x, y)\) is the transmission matrix, defined as:
\[
t(x, y)=e^{-\beta d(x, y)}
\]
In this formula, \(\beta\) represents the atmospheric scattering coefficient, and \(d(x, y)\) represents the relative distance between the scene object and the camera.

These formulas are used to simulate and generate realistic hazy images to support the development and evaluation of algorithms.