Abstract:Bird's-eye-view (BEV) semantic segmentation is becoming crucial in autonomous driving systems. It realizes ego-vehicle surrounding environment perception by projecting 2D multi-view images into 3D world space. Recently, BEV segmentation has made notable progress, attributed to better view transformation modules, larger image encoders, or more temporal information. However, there are still two issues: 1) a lack of effective understanding and enhancement of BEV space features, particularly in accurately capturing long-distance environmental features and 2) recognizing fine details of target objects. To address these issues, we propose OE-BevSeg, an end-to-end multimodal framework that enhances BEV segmentation performance through global environment-aware perception and local target object enhancement. OE-BevSeg employs an environment-aware BEV compressor. Based on prior knowledge about the main composition of the BEV surrounding environment varying with the increase of distance intervals, long-sequence global modeling is utilized to improve the model's understanding and perception of the environment. From the perspective of enriching target object information in segmentation results, we introduce the center-informed object enhancement module, using centerness information to supervise and guide the segmentation head, thereby enhancing segmentation performance from a local enhancement perspective. Additionally, we designed a multimodal fusion branch that integrates multi-view RGB image features with radar/LiDAR features, achieving significant performance improvements. Extensive experiments show that, whether in camera-only or multimodal fusion BEV segmentation tasks, our approach achieves state-of-the-art results by a large margin on the nuScenes dataset for vehicle segmentation, demonstrating superior applicability in the field of autonomous driving.

AEFusion: an Attention-Based Ensemble Learning Approach for BEV Fusion Perception in Autonomous Modular Buses

BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation

CoBEVFusion: Cooperative Perception with LiDAR-Camera Bird's-Eye View Fusion

Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving

BEVFusion4D: Learning LiDAR-Camera Fusion Under Bird's-Eye-View via Cross-Modality Guidance and Temporal Aggregation

Towards Robust LiDAR-Camera Fusion in BEV Space via Mutual Deformable Attention and Temporal Aggregation

HEAD: A Bandwidth-Efficient Cooperative Perception Approach for Heterogeneous Connected and Autonomous Vehicles

OE-BevSeg: An Object Informed and Environment Aware Multimodal Framework for Bird's-eye-view Vehicle Semantic Segmentation

CMGFA: A BEV Segmentation Model Based on Cross-Modal Group-Mix Attention Feature Aggregator

MetaBEV: Solving Sensor Failures for BEV Detection and Map Segmentation

UniFusion: Unified Multi-view Fusion Transformer for Spatial-Temporal Representation in Bird's-Eye-View

GraphBEV: Towards Robust BEV Feature Alignment for Multi-Modal 3D Object Detection

Enhancing 3D object detection through multi-modal fusion for cooperative perception

BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework

Fisheye Camera and Ultrasonic Sensor Fusion For Near-Field Obstacle Perception in Bird's-Eye-View

V2I-BEVF: Multi-modal Fusion Based on BEV Representation for Vehicle-Infrastructure Perception

M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Driving

KAN-RCBEVDepth: A multi-modal fusion algorithm in object detection for autonomous driving

Hierarchical and Decoupled BEV Perception Learning Framework for Autonomous Driving

UniBEV: Multi-modal 3D Object Detection with Uniform BEV Encoders for Robustness against Missing Sensor Modalities