Abstract:Multi-modal 3D object detection is instrumental in identifying and localizing objects within 3D space. It combines RGB images from cameras and point-clouds data from lidar sensors, serving as a fundamental technology for autonomous driving applications. Current methods commonly employ simplistic element-wise additions or multiplications to aggregate multi-modal features extracted from point-clouds and images. While these methods enhance detection accuracy, the utilization of basic operations presents challenges in effectively balancing the significance between modalities. This can potentially introduce noise and irrelevant information during the feature aggregation process. Additionally, the multi-level features extracted from images display imbalances in receptive fields. To tackle the aforementioned challenges, we propose two innovative networks: a cross-modality balance network (CMN) and a cross-scale balance network (CSN). CMN incorporates cross-modality attention mechanisms and introduces an auxiliary 2D detection head to balance the significance of both modalities. Meanwhile, CSN leverages cross-scale attention mechanisms to mitigate the gap in receptive fields between different image levels. Additionally, we introduce a novel Local with Global Voxel Attention Encoder (LGVAE) designed to capture global semantics by extracting more comprehensive point-level information into voxel-level features. We perform comprehensive experiments on three challenging public benchmarks: KITTI, Dense and nuScenes. The results consistently demonstrate improvements across multiple 3D object detection frameworks, affirming the effectiveness and versatility of our proposed method. Remarkably, our approach achieves a substantial absolute gain of 3.1% over the baseline MVXNet on the challenging Hard set of the Dense test set.

CSANet: Cross-Semantic Attention Network for Open-Set Object Recognition

Convolutional Prototype Network for Open Set Recognition

Class-Specific Semantic Reconstruction for Open Set Recognition

GCSANet: A Global Context Spatial Attention Deep Learning Network for Remote Sensing Scene Classification

Semantic-Aware Alignment Network for Cross-Resolution Change Detection

SACANet: scene-aware class attention network for semantic segmentation of remote sensing images

AANet: Adaptive Attention Networks for Semantic Segmentation of High-Resolution Remote Sensing Imagery

C2BG-Net: Cross-modality and cross-scale balance network with global semantics for multi-modal 3D object detection

Category attention guided network for semantic segmentation of Fine-Resolution remote sensing images

P-ODN: Prototype based Open Deep Network for Open Set Recognition

Spatial-Scale Aligned Network for Fine-Grained Recognition

ASAN: Self-Attending and Semantic Activating Network Towards Better Object Detection

Spatial-Temporal Attention Network for Open-Set Fine-Grained Image Recognition

Open-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning

Unbalanced Class Learning Network With Scale-Adaptive Perception for Complicated Scene in Remote Sensing Images Segmentation

SAN: Side Adapter Network for Open-vocabulary Semantic Segmentation

ASNet: Adaptive Semantic Network Based on Transformer–CNN for Salient Object Detection in Optical Remote Sensing Images

TCANet: Triple Context-Aware Network for Weakly Supervised Object Detection in Remote Sensing Images

Cross Channel Aggregation Similarity Network for Salient Object Detection

CLASS: Cross-Level Attention and Supervision for Salient Objects Detection

Hybridizing Cross-Level Contextual and Attentive Representations for Remote Sensing Imagery Semantic Segmentation