Abstract:Object detection and semantic segmentation are two fundamental techniques for Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Motivated by recent studies demonstrating that object detection and semantic segmentation are two highly-correlated tasks, this paper handles the problem of joint object detection and semantic segmentation in traffic scenes. Existing methods perform the joint object detection and semantic segmentation by sharing the same backbone network, but always ignore the interactive connection between the subdividing detection branch and segmentation branch, leading to the insufficient interaction between the two branches. Considering this situation, this paper proposes a joint object detection and semantic segmentation model with the cross-attention and inner-attention mechanisms. The cross-attention mechanism enables to build up the essential interaction between the subdividing detection branch and segmentation branch to fully make use of their correlation. In addition, the inner-attention contributes to strengthening the representations of feature maps in the model. Given an image, an encoder-decoder network is firstly used to extract initial feature maps. Then, the inner-attention mechanism is applied to strengthen the initial feature maps to obtain segmentation feature maps. Subsequently, the cross-attention mechanism utilizes the segmentation feature maps to guide the generation of object detection feature maps. Finally, the semantic segmentation is performed on the segmentation feature maps and object detection is performed on the detection feature maps. In the experiments, two well-known public traffic datasets are used to evaluate our model. Our model achieves the highest performance in comparison with several recently-proposed methods. In addition, some ablation studies are conducted to evaluate the proposed inner-attention and cross-attention mechanisms, and experiment results validate their effectiveness.

An Integrated Bottom-Up and Top-Down Computing Process for Car Parsing

Discriminative Hierarchical Part-Based Models for Human Parsing and Action Recognition.

From Time to Space: Automatic Annotation of Unmarked Traffic Scene Based on Trajectory Data.

Part-level Car Parsing and Reconstruction in Single Street View Images

Integrating Context And Occlusion For Car Detection By Hierarchical And-Or Model

Part-level Car Parsing and Reconstruction from Single Street View

Modeling Inter- and Intra-Part Deformations for Object Structure Parsing

Hierarchical Object Parsing from Structured Noisy Point Clouds

Image Parsing: Unifying Segmentation, Detection, and Recognition.

A Hierarchical and Contextual Model for Aerial Image Parsing

Vehicle Perception from a Single Image for Autonomous Driving Using Deformable Model Representation and Deep Learning

Learning And-Or Model to Represent Context and Occlusion for Car Detection and Viewpoint Estimation.

A Joint Object Detection and Semantic Segmentation Model with Cross-Attention and Inner-Attention Mechanisms

PerMO: Perceiving More at Once from a Single Image for Autonomous Driving

Learning And-Or Models to Represent Context and Occlusion for Car Detection and Viewpoint Estimation

Automatic non-parametric image parsing via hierarchical semantic voting based on sparse-dense reconstruction and spatial-contextual cues.

Modeling Occlusion by Discriminative AND-OR Structures

Towards Panoptic 3D Parsing for Single Image in the Wild

Bottom-up/top-down image parsing by attribute graph grammar

Human Carving: A Parsing-Based Framework for 3d Human Reconstruction

A Numerical Study of the Bottom-Up and Top-Down Inference Processes in And-Or Graphs.