Abstract:Aiming at the actual traffic driving environment, this paper proposes an acoustic perception method of intelligent vehicle based on microphone array, and combines the visual perception method of monocular camera to study the acoustic visual fusion perception method. The main research contents are as follows: Firstly, an acoustic simulation environment for intelligent vehicles is designed. Secondly, a sound source localization algorithm based on time delay estimation is proposed. Based on the geometry of the microphone array, the sound source localization problem is abstracted as a time delay estimation problem, and the sound source localization is realized. The received signal of the microphone is added by weighted delay to form a beam with a certain direction, which is visualized as a spatial SRP power map. The passive sound reception is transformed into an active search for space, which improves the positioning accuracy in low SNR and reverberation environment. Thirdly, a sound source perception localization algorithm combining deep learning network and delay algorithm is developed. Based on SELDnet neural network, the SRP power map is used as input, and the cyclic convolution layer is replaced by the temporal convolution layer. The simulation results show that the positioning accuracy of the fusion positioning network is 65.04 % higher than that of the SELDnet network in the ideal environment and 76.19 % higher in the low SNR environment. The feasibility verification experiment and simulation experiment of the algorithm are carried out, and the sensor data in the real vehicle environment is collected for the real vehicle data verification experiment. The results show that the average positioning error of the acoustic vision fusion algorithm is 0.1 m at the 60 m scale. At the same time, the algorithm also has strong adaptability to visual blind spots, which proves the deployment potential of the algorithm in the actual environment.

Development and Calibration of Acoustic Video Camera System for Moving Vehicles

Development of sound visualization system based on binocular vision and short-time wave superposition algorithm

Primary Research for Voice-Activated Camera Tracking

Indoor Wireless Positioning Utilizing WSN and Machine Vision

A Multi-Aperture Acoustic Holography Method For Moving Vehicles

Noise Sources Measurement and Identification for Moving Automobiles

Video Visualization For Moving Sound Sources Based On Binocular Vision And Short-Time Beamforming

An Automatic Detecting System Based on Sound Detection and Image Analysis

A Method of Video Analysis in Vehicle Based on A Single Pal Camera

Development of acoustic vidicon based on binocular vision and microphone array

Video Visualization for Moving Sound Sources Based on Binoculars Stereo and Acoustical Holography

Camera Orientation Calibration Technique For Sound Visualization System

Design and implementation of smart vehicle auditory perception system

Research on intelligent driving technology based on acoustic vision fusion

Acoustic Holography Method for the Identification of Moving Sound Source

Quantitative Measurement of Pass-by Noise Radiated by Vehicles Running at High Speeds

Adding Ears to Intelligent Connected Vehicles by Combining Microphone Arrays and High Definition Map

Research on Spatial Relationship Calibration Technique for Microphone Arrayand Camera in Sound Field Visualization System

Decomposition of Moving Vehicle Noise with Dynamic Transfer Model and Acoustic Holograph

Self-supervised Moving Vehicle Tracking with Stereo Sound