Abstract:Humans assimilate information from the traffic environment mainly through visual perception. Obviously, the dominant information required to conduct a vehicle can be acquired with visual sensors. However, in contrast to most other sensor principles, video signals contain relevant information in a highly indirect manner and hence visual sensing requires sophisticated machine vision and image understanding techniques. This paper provides an overview on the state of research in the field of machine vision for intelligent vehicles. The functional spectrum addressed covers the range from advanced driver assistance systems to autonomous driving. The organization of the article adopts the typical order in image processing pipelines that successively condense the rich information and vast amount of data in video sequences. Data-intensive low-level early vision techniques first extract features that are later grouped and further processed to obtain information of direct relevance for vehicle guidance. Recognition and classification schemes allow to identify specific objects in a traffic scene. Recently, semantic labeling techniques using convolutional neural networks have achieved impressive results in this field. High-level decisions of intelligent vehicles are often influenced by map data. The emerging role of machine vision in the mapping and localization process is illustrated at the example of autonomous driving. Scene representation methods are discussed that organize the information from all sensors and data sources and thus build the interface between perception and planning. Recently, vision benchmarks have been tailored to various tasks in traffic scene perception that provide a metric for the rich diversity of machine vision methods. Finally, the paper addresses computing architectures suited to real-time implementation. Throughout the paper, numerous specific examples and real world experiments with prototype vehicles are presented.

Emergent Visual Sensors for Autonomous Vehicles

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception.

Intelligent Vision Systems for Automotive Applications

3D Vehicle Detection Using Cheap LiDAR and Camera Sensors.

Vision-Based Environmental Perception for Autonomous Driving

Application of Intelligence Binocular Vision Sensor: Mobility Solutions for Automotive Perception System

Applications of Computer Vision in Autonomous Vehicles: Methods, Challenges and Future Directions

Application of Intelligence Binocular VisionSensor: Mobility Solutions for AutomotivePerception System

Exploring the challenges and opportunities of image processing and sensor fusion in autonomous vehicles: A comprehensive review

Decisive Data using Multi-Modality Optical Sensors for Advanced Vehicular Systems

Recent Advances in Automated Driving Technologies [From the Guest Editors]

Applications of visual perception techniques using neural networks in autonomous driving

A Survey of Computer Vision Algorithm Applied in Driverless Robotic Vehicles with Sensing Capability

The Role of Machine Vision for Intelligent Vehicles

Emerging Trends in Autonomous Vehicle Perception: Multimodal Fusion for 3D Object Detection

Sensor-Based Environmental Perception Technology for Intelligent Vehicles

IVS 09: Future Research in Vehicle Vision Systems

Autonomous Multi-Sensor Fusion Techniques for Environmental Perception in Self-Driving Vehicles

An Overview of Autonomous Vehicles Sensors and Their Vulnerability to Weather Conditions

Practical Sensing Techniques for Intelligent Vehicles