Abstract:Learning powerful representations in bird's-eye-view (BEV) for perception tasks is trending and drawing extensive attention both from industry and academia. Conventional approaches for most autonomous driving algorithms perform detection, segmentation, tracking, etc., in a front or perspective view. As sensor configurations get more complex, integrating multi-source information from different sensors and representing features in a unified view come of vital importance. BEV perception inherits several advantages, as representing surrounding scenes in BEV is intuitive and fusion-friendly; and representing objects in BEV is most desirable for subsequent modules as in planning and/or control. The core problems for BEV perception lie in (a) how to reconstruct the lost 3D information via view transformation from perspective view to BEV; (b) how to acquire ground truth annotations in BEV grid; (c) how to formulate the pipeline to incorporate features from different sources and views; and (d) how to adapt and generalize algorithms as sensor configurations vary across different scenarios. In this survey, we review the most recent works on BEV perception and provide an in-depth analysis of different solutions. Moreover, several systematic designs of BEV approach from the industry are depicted as well. Furthermore, we introduce a full suite of practical guidebook to improve the performance of BEV perception tasks, including camera, LiDAR and fusion inputs. At last, we point out the future research directions in this area. We hope this report will shed some light on the community and encourage more research effort on BEV perception. We keep an active repository to collect the most recent work and provide a toolbox for bag of tricks at https://github.com/OpenDriveLab/Birds-eye-view-Perception.

Environmental Perception Using Fish-Eye Cameras for Autonomous Driving

Survey on Fish-Eye Cameras and Their Applications in Intelligent Vehicles

Object Detection and Localization in 3D Environment by Fusing Raw Fisheye Image and Attitude Data

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges

3-D Surround View for Advanced Driver Assistance Systems.

An Open-Source Platform For Underwater Image And Video Analytics

Surround-View Fisheye Camera Perception for Automated Driving: Overview, Survey & Challenges

Downside Hemisphere Object Detection and Localization of MAV by Fisheye Camera

Disentangling and Vectorization: A 3D Visual Perception Approach for Autonomous Driving Based on Surround-View Fisheye Cameras

Universal Semantic Segmentation for Fisheye Urban Driving Images

Real-Time Dense Mapping for Self-driving Vehicles using Fisheye Cameras

Moving Object Detection Using an In-Vehicle Fish-Eye Camera

CNN Based Semantic Segmentation for Urban Traffic Scenes Using Fisheye Camera

FisheyeDetNet: 360° Surround view Fisheye Camera based Object Detection System for Autonomous Driving

Vision-Based Environmental Perception for Autonomous Driving

Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges

FisheyePP4AV: A privacy-preserving method for autonomous vehicles on fisheye camera images

Delving Into the Devils of Bird's-Eye-View Perception: A Review, Evaluation and Recipe

Surround-View Fisheye BEV-Perception for Valet Parking: Dataset, Baseline and Distortion-Insensitive Multi-Task Framework