Abstract:Object detection and semantic segmentation are two fundamental techniques for Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Motivated by recent studies demonstrating that object detection and semantic segmentation are two highly-correlated tasks, this paper handles the problem of joint object detection and semantic segmentation in traffic scenes. Existing methods perform the joint object detection and semantic segmentation by sharing the same backbone network, but always ignore the interactive connection between the subdividing detection branch and segmentation branch, leading to the insufficient interaction between the two branches. Considering this situation, this paper proposes a joint object detection and semantic segmentation model with the cross-attention and inner-attention mechanisms. The cross-attention mechanism enables to build up the essential interaction between the subdividing detection branch and segmentation branch to fully make use of their correlation. In addition, the inner-attention contributes to strengthening the representations of feature maps in the model. Given an image, an encoder-decoder network is firstly used to extract initial feature maps. Then, the inner-attention mechanism is applied to strengthen the initial feature maps to obtain segmentation feature maps. Subsequently, the cross-attention mechanism utilizes the segmentation feature maps to guide the generation of object detection feature maps. Finally, the semantic segmentation is performed on the segmentation feature maps and object detection is performed on the detection feature maps. In the experiments, two well-known public traffic datasets are used to evaluate our model. Our model achieves the highest performance in comparison with several recently-proposed methods. In addition, some ablation studies are conducted to evaluate the proposed inner-attention and cross-attention mechanisms, and experiment results validate their effectiveness.

Coupling-and-Decoupling: a hierarchical model for occlusion-free car detection

Leveraging Front and Side Cues for Occlusion Handling in Monocular 3D Object Detection

Inference Fusion with Associative Semantics for Unseen Object Detection

Modeling Occlusion by Discriminative AND-OR Structures

Integrating Context And Occlusion For Car Detection By Hierarchical And-Or Model

Learning And-Or Model to Represent Context and Occlusion for Car Detection and Viewpoint Estimation.

Learning And-Or Models to Represent Context and Occlusion for Car Detection and Viewpoint Estimation

Adaptive occlusion object detection algorithm based on OL-IoU

Detecting Semantic Parts on Partially Occluded Objects

A Joint Object Detection and Semantic Segmentation Model with Cross-Attention and Inner-Attention Mechanisms

Vehicle Detection by Fusing Part Model Learning and Semantic Scene Information for Complex Urban Surveillance

OSMO: Online Specific Models for Occlusion in Multiple Object Tracking under Surveillance Scene.

3D Detection for Occluded Vehicles From Point Clouds

Discriminatively Trained And-Or Tree Models for Object Detection

A lightweight YOLOv5-FFM model for occlusion pedestrian detection

CF-YOLOX: An Autonomous Driving Detection Model for Multi-Scale Object Detection

Small-Object Detection Model for Optical Remote Sensing Images Based on Tri-Decoupling++Head

HTD: Heterogeneous Task Decoupling for Two-Stage Object Detection

Multi-View Vehicle Detection Based on Fusion Part Model With Active Learning

Weighted Deformable Part Model for Robust Human Detection

Combining Compositional Models and Deep Networks For Robust Object Classification under Occlusion