Abstract:Task-specific data-fusion networks have marked considerable achievements in urban scene parsing. Among these networks, our recently proposed RoadFormer successfully extracts heterogeneous features from RGB images and surface normal maps and fuses these features through attention mechanisms, demonstrating compelling efficacy in RGB-Normal road scene parsing. However, its performance significantly deteriorates when handling other types/sources of data or performing more universal, all-category scene parsing tasks. To overcome these limitations, this study introduces RoadFormer+, an efficient, robust, and adaptable model capable of effectively fusing RGB-X data, where ``X'', represents additional types/modalities of data such as depth, thermal, surface normal, and polarization. Specifically, we propose a novel hybrid feature decoupling encoder to extract heterogeneous features and decouple them into global and local components. These decoupled features are then fused through a dual-branch multi-scale heterogeneous feature fusion block, which employs parallel Transformer attentions and convolutional neural network modules to merge multi-scale features across different scales and receptive fields. The fused features are subsequently fed into a decoder to generate the final semantic predictions. Notably, our proposed RoadFormer+ ranks first on the KITTI Road benchmark and achieves state-of-the-art performance in mean intersection over union on the Cityscapes, MFNet, FMB, and ZJU datasets. Moreover, it reduces the number of learnable parameters by 65\% compared to RoadFormer. Our source code will be publicly available at mias.group/RoadFormerPlus.

A Novel Approach for Detecting Road Based on Two-Stream Fusion Fully Convolutional Network

Multi-Stage Residual Fusion Network for LIDAR-Camera Road Detection

Two-View Fusion Based Convolutional Neural Network for Urban Road Detection

A Cascaded LiDAR-Camera Fusion Network for Road Detection

Deep Representation Learning for Road Detection Using Siamese Network.

Road detection via a dual-task network based on cross-layer graph fusion modules

Multi-feature fusion and multi-attention deep network for enhancing road extraction in remote sensing images

A duplex transform heterogeneous feature fusion network for road segmentation

Traffic Sign Detection using Feature Fusion and Contextual Information

Embedding Structured Contour and Location Prior in Siamesed Fully Convolutional Networks for Road Detection

3-D LiDAR + Monocular Camera: an Inverse-Depth-Induced Fusion Framework for Urban Road Detection

Feature Plus Plus : Cross Dimension Feature Fusion For Road Detection

Feature++: Cross dimension feature fusion for road detection

A Multi-view 3D Vehicle Detection Method Based On Novel 3D Proposal Generation Method

Combining 2D and 3D Features to Improve Road Detection Based on Stereo Cameras

Road Detection through CRF based LiDAR-Camera Fusion.

Integrating Dense LiDAR-Camera Road Detection Maps by a Multi-Modal CRF Model

A Fusion Model for Road Detection based on Deep Learning and Fully Connected CRF

RoadFormer+: Delivering RGB-X Scene Parsing through Scale-Aware Information Decoupling and Advanced Heterogeneous Feature Fusion

Capitalizing on RGB-FIR Hybrid Imaging for Road Detection

Residual Channel Attention Fusion Network for Road Extraction Based on Remote Sensing Images and GPS Trajectories