Abstract:Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, Ahead of Print. In recent years, Sound Event Classification (SEC) is a remarkable field of computer audition research. The abilities to automatically capture and classify sound events are meaningful for expanding the auditory ability of autonomous vehicle in a complex driving environment. The Autonomous vehicles will encounter some special road conditions during driving, such as large truck occlusion, tunnels, and buildings that hinder radar detection. Therefore, the perception capabilities of on-board lidar, vision, and other sensors are impacted, resulting in a great threat to the safe driving of the car. At this time, auditory environment perception can provide auditory dimension information for the vehicle. However, with the increase of noise level, the SEC task of autonomous vehicle becomes seriously difficult. Traditional machine learning methods show weak robustness in the highly noisy environment. In this work, an autonomous vehicle Sound Event Classification (AVSEC) framework based on Spectrogram Image Features (SIFs) and Convolutional Neural Networks (CNN) algorithm is proposed to solve the SEC question. Further on, the ambulance, bus, civil defense siren, fire truck, screaming, etc. are taken as the research traffic sounds. Traffic Sound Datasets (TSD) are collected, augmented and customized. Then, directly connected channel (DCC) between convolutional blocks and Convolutional Block Attention Module (CBAM) are designed to optimize the neural network. Testing results show that the AVSEC system achieves an accuracy of 97.18%. Furthermore, the results of AVSEC and other machine learning methods are compared, which shows that the combination of SIFs and AVSEC-net provides better noise robustness and classification accuracy in AVSEC task. The sound classification algorithm designed in this paper, as an advanced auxiliary driving system for sound detection, can identify the sound types in the current traffic environment of the Autonomous vehicles, that is, the traffic scene category. The vehicle computer can remind the driver to pay attention to driving safety through the above sound types, so as to improve the safety of the driver and the vehicle.

A safety-oriented framework for sound event detection in driving scenarios

Deep Convolutional Neural Network for Roadway Incident Surveillance Using Audio Data

Traffic Sign Detection With Event Cameras and DCNN

Improved convolutional neural network and spectrogram image feature for traffic sound event classification

Moving vehicle tracking and scene understanding: A hybrid approach

CARgram: CNN-based accident recognition from road sounds through intensity-projected spectrogram analysis

Machine Learning Framework for Real-Time Assessment of Traffic Safety Utilizing Connected Vehicle Data

A Machine Learning Framework for Automated Accident Detection Based on Multimodal Sensors in Cars

Real-Time Vehicle Sound Detection System Based on Depthwise Separable Convolution Neural Network and Spectrogram Augmentation

Object Detection and Classification Framework for Analysis of Video Data Acquired from Indian Roads

Road Feature Detection for Advance Driver Assistance System Using Deep Learning

Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

Real-time Hazard Detection in Autonomous Vehicles: Deep Learning Approaches for Enhancing Road Safety

Self-Updating Vehicle Monitoring Framework Employing Distributed Acoustic Sensing towards Real-World Settings

A deep neural framework for real-time vehicular accident detection based on motion temporal templates

Real-Time Vehicle Accident Recognition from Traffic Video Surveillance using YOLOV8 and OpenCV

Machine vision-based autonomous road hazard avoidance system for self-driving vehicles

Improving the Environmental Perception of Autonomous Vehicles using Deep Learning-based Audio Classification

Smart City Transportation: Deep Learning Ensemble Approach for Traffic Accident Detection

A Deep Learning Based Framework for Blind Road Occupancy Detection

DeepSecDrive: an Explainable Deep Learning Framework for Real-Time Detection of Cyberattack in In-Vehicle Networks