Abstract:Accurate visual perception and comprehensive scene understanding are critical for the safety and reliability of autonomous vehicles (AVs). Nevertheless, the efficacy of visual perception systems can be impaired by the intricacy of road scenes, and the existing scene understanding approach may be insufficient. Consequently, this study proposes an enhanced scene understanding model to achieve precise awareness of driving situations. Recognizing the limitations posed by the oversimplification of samples in current urban scene datasets, we selected critical frames from 336 000 video frames, sourced from real-world driving environments, to assemble a more complex road scene (CRS) dataset. We integrated Residual Neural Network and pyramid scene parsing network architectures and refined them through class mapping and targeted network fine-tuning. Based on the segmentation outputs and the XGBoost algorithm, we identified the driving scenarios for the ego vehicle, enabling instantaneous driving situation analysis. The predictive model also evaluated the trajectory of interactive vehicles and estimated their kinematic states. Furthermore, we have conducted a thorough evaluation of scenario complexity, integrating the features described above. The findings indicate that our model achieves a segmentation accuracy of 78.8% in CRSs, with a twofold improvement in training efficiency. We also confirmed the effectiveness of the scene understanding approach through real-world road testing in China. This research provides insight into situation awareness within CRSs, thereby enhancing the visual perception capabilities of AVs. The implications of these results are substantial for their application in autonomous driving tests and advancing decision-making and control algorithms.

Semantic perception of curbs beyond traversability for real-world navigation assistance systems

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Robustifying Semantic Cognition of Traversability Across Wearable RGB-depth Cameras

An Environmental Perception and Navigational Assistance System for Visually Impaired Persons Based on Semantic Stixels and Sound Interaction

Perception Framework of Water Hazards Beyond Traversability for Real-World Navigation Assistance Systems

Long-Range Traversability Awareness and Low-Lying Obstacle Negotiation with RealSense for the Visually Impaired

Unifying Terrain Awareness for the Visually Impaired through Real-Time Semantic Segmentation

Intersection Perception Through Real-Time Semantic Segmentation to Assist Navigation of Visually Impaired Pedestrians

Can We PASS Beyond the Field of View? Panoramic Annular Semantic Segmentation for Real-World Surrounding Perception

Semi-Supervised Learning for Visual Bird's Eye View Semantic Segmentation

Expanding the Detection of Traversable Area with RealSense for the Visually Impaired

Intersection Navigation For People With Visual Impairment

Can We Unify Perception and Localization in Assisted Navigation? An Indoor Semantic Visual Positioning System for Visually Impaired People

Multi-faceted Sensory Substitution for Curb Alerting: A Pilot Investigation in Persons with Blindness and Low Vision

Self-Supervised Sidewalk Perception Using Fast Video Semantic Segmentation for Robotic Wheelchairs in Smart Mobility

Utilizing Semantic Visual Landmarks for Precise Vehicle Navigation

A Wearable Navigation Device for Visually Impaired People Based on the Real-Time Semantic Visual SLAM System

Enhanced Scene Understanding and Situation Awareness for Autonomous Vehicles Based on Semantic Segmentation

Infrastructure-Assisted Collaborative Perception in Automated Valet Parking: A Safety Perspective

Agent Journey Beyond RGB: Unveiling Hybrid Semantic-Spatial Environmental Representations for Vision-and-Language Navigation

Road-Segmentation-Based Curb Detection Method for Self-Driving Via a 3D-Lidar Sensor