GMC: A General Framework of Multi-stage Context Learning and Utilization for Visual Detection Tasks

Xuan Wang,Hao Tang,Zhigang Zhu

DOI: https://doi.org/10.1016/j.cviu.2024.103944

2024-07-08

Abstract:Various contextual information has been employed by many approaches for visual detection tasks. However, most of the existing approaches only focus on specific context for specific tasks. In this paper, GMC, a general framework is proposed for multistage context learning and utilization, with various deep network architectures for various visual detection tasks. The GMC framework encompasses three stages: preprocessing, training, and post-processing. In the preprocessing stage, the representation of local context is enhanced by utilizing commonly used labeling standards. During the training stage, semantic context information is fused with visual information, leveraging prior knowledge from the training dataset to capture semantic relationships. In the post-processing stage, general topological relations and semantic masks for stuff are incorporated to enable spatial context reasoning between objects. The proposed framework provides a comprehensive and adaptable solution for context learning and utilization in visual detection scenarios. The framework offers flexibility with user-defined configurations and provide adaptability to diverse network architectures and visual detection tasks, offering an automated and streamlined solution that minimizes user effort and inference time in context learning and reasoning. Experimental results on the visual detection tasks, for storefront object detection, pedestrian detection and COCO object detection, demonstrate that our framework outperforms previous state-of-the-art detectors and transformer architectures. The experiments also demonstrate that three contextual learning components can not only be applied individually and in combination, but can also be applied to various network architectures, and its flexibility and effectiveness in various detection scenarios.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the problem of how to effectively utilize multi-stage contextual information to improve detection performance in visual detection tasks. Specifically, the authors propose a Generalized Multi-stage Context (GMC) framework, which leverages local context, semantic context, and spatial context at three different stages: data annotation, model training, and post-processing. By doing so, the GMC framework aims to provide a comprehensive and flexible solution for various visual detection tasks, capable of enhancing detection performance across different network architectures while reducing the time and user intervention required for context learning and inference. Additionally, the framework pays special attention to the challenges of detecting small and occluded objects, enhancing recognition accuracy in these scenarios by incorporating more contextual information. In summary, the core issue addressed by the paper is the development of a context learning framework that can adapt to various visual detection tasks and network models to achieve better detection results.