Towards RAW Object Detection in Diverse Conditions

Zhong-Yu Li,Xin Jin,Boyuan Sun,Chun-Le Guo,Ming-Ming Cheng

2024-11-24

Abstract:Existing object detection methods often consider sRGB input, which was compressed from RAW data using ISP originally designed for visualization. However, such compression might lose crucial information for detection, especially under complex light and weather conditions. We introduce the AODRaw dataset, which offers 7,785 high-resolution real RAW images with 135,601 annotated instances spanning 62 categories, capturing a broad range of indoor and outdoor scenes under 9 distinct light and weather conditions. Based on AODRaw that supports RAW and sRGB object detection, we provide a comprehensive benchmark for evaluating current detection methods. We find that sRGB pre-training constrains the potential of RAW object detection due to the domain gap between sRGB and RAW, prompting us to directly pre-train on the RAW domain. However, it is harder for RAW pre-training to learn rich representations than sRGB pre-training due to the camera noise. To assist RAW pre-training, we distill the knowledge from an off-the-shelf model pre-trained on the sRGB domain. As a result, we achieve substantial improvements under diverse and adverse conditions without relying on extra pre-processing modules. Code and dataset are available at <a class="link-external link-https" href="https://github.com/lzyhha/AODRaw" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenge of object detection based on RAW images under different lighting and weather conditions. Specifically:

1. **Limitations of existing methods**: Most of the existing object detection methods consider sRGB inputs, which are compressed from RAW data using ISPs originally designed for visualization. However, this compression may lose information crucial for detection, especially under complex lighting and weather conditions.

2. **Advantages of RAW images**: Compared with the compressed sRGB images, RAW images retain a higher bit - depth and thus contain more distinguishable information, which is crucial for computer vision tasks, especially under harsh lighting and weather conditions.

3. **Scarcity of datasets**: The number of existing RAW object detection datasets is limited and lacks diversity, which restricts the development of RAW object detection. For example, some datasets only focus on specific scenarios or conditions, such as outdoor scenes in daylight and low - light conditions, while ignoring other harsh conditions.

To solve these problems, the paper proposes the following:

- **Constructing a high - quality dataset**: Proposes a high - quality dataset named AODRaw, which contains 7,785 high - resolution real RAW images, covering 62 categories and 135,601 annotated instances, involving 9 different lighting and weather conditions.

- **Benchmarking**: Based on the AODRaw dataset, provides a comprehensive benchmark for evaluating current object detection methods.

- **RAW pre - training**: Explores the method of directly pre - training models in the RAW domain, extracts knowledge from sRGB pre - trained models through cross - domain distillation to overcome the noise problem in RAW images and improve the performance of models in the RAW domain.

In conclusion, this paper aims to improve the performance of RAW object detection under complex lighting and weather conditions by constructing high - quality datasets and improving pre - training methods.