ConsistencyDet: A Robust Object Detector with a Denoising Paradigm of Consistency Model

Lifan Jiang,Zhihui Wang,Changmiao Wang,Ming Li,Jiaxu Leng,Xindong Wu

2024-05-14

Abstract:Object detection, a quintessential task in the realm of perceptual computing, can be tackled using a generative methodology. In the present study, we introduce a novel framework designed to articulate object detection as a denoising diffusion process, which operates on the perturbed bounding boxes of annotated entities. This framework, termed ConsistencyDet, leverages an innovative denoising concept known as the Consistency Model. The hallmark of this model is its self-consistency feature, which empowers the model to map distorted information from any temporal stage back to its pristine state, thereby realizing a "one-step denoising" mechanism. Such an attribute markedly elevates the operational efficiency of the model, setting it apart from the conventional Diffusion Model. Throughout the training phase, ConsistencyDet initiates the diffusion sequence with noise-infused boxes derived from the ground-truth annotations and conditions the model to perform the denoising task. Subsequently, in the inference stage, the model employs a denoising sampling strategy that commences with bounding boxes randomly sampled from a normal distribution. Through iterative refinement, the model transforms an assortment of arbitrarily generated boxes into definitive detections. Comprehensive evaluations employing standard benchmarks, such as MS-COCO and LVIS, corroborate that ConsistencyDet surpasses other leading-edge detectors in performance metrics. Our code is available at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

The paper proposes a new object detection framework called ConsistencyDet, which aims to improve the efficiency and accuracy of object detection through a denoising paradigm (i.e., consistency model). Specifically, the paper attempts to address the following issues:

1. **Shortcomings of Existing Object Detection Methods**:
- Traditional object detection methods rely on manually designed candidate region selection processes, which perform poorly in complex visual scenes.
- Although anchor boxes have been introduced, these methods are still limited by prior information, making it difficult to generalize across different datasets and scenarios.

2. **Improving the Limitations of Existing Diffusion Models**:
- While diffusion models perform well in object detection, their iterative denoising process leads to low execution efficiency, limiting their feasibility in real-time applications.
- By adopting a single-step denoising mechanism in the consistency model, the paper aims to optimize the balance between detection accuracy and computational speed.

3. **Proposing an Efficient Object Detection Method**:
- Utilizing self-consistency features to achieve rapid mapping from any time point to the initial state, thereby reducing the number of denoising iterations and enhancing detection efficiency.
- During training, by introducing noise and gradually removing it to predict the true position and size of the bounding boxes, ultimately achieving precise object detection.

In summary, the main objective of the paper is to overcome the limitations of existing object detection methods and significantly enhance the efficiency and performance of detection tasks by introducing the ConsistencyDet framework.