Abstract:With the increasing popularity of the Internet of Things and smart factories, robust sound acquisition plays an important role in communication and human–machine interaction, for example, in-vehicle voice control interactions, smart home, and robotic voice interaction. However, due to the inherent sensing manners of the widely used microphone system, the existing audio-only sound acquisition system has fundamental limitations in solving the cocktail problem and is difficult to adapt to complex acoustic environments. Considering the advantages of millimeter-wave (mmWave) radar in the field of target positioning and precise vibration measurement, the mmWave vibration measurement-based sound perception has great potential to solve the above problems. However, there are still difficulties in high-quality sound recovery. In this letter, we propose RFMic-Phone, a robust sound acquisition system combining a mmWave radar and a traditional microphone, which combines the sound signals captured by the mmWave radar and microphone, offering a novel approach for reliable sound acquisition in the complex acoustic environment. The microphone is used to obtain the mixed and high-fidelity audio signals, while the radar is used to obtain the vibration of the sound source. To achieve intelligent and effective feature fusion, we employ a deep learning framework with a modified convolutional encoder–decoder neural network structure. Moreover, we propose to utilize the real and imaginary spectra of the target sound source as the input of the network, allowing achieve high-quality target sound signals. The experimental results show that the RFMic-Phone can achieve robust and high-quality sound signals acquisition in a variety of complex acoustic environments.

Integrated Sensing and Communication between Daily Devices and mmWave Radars.

Poster Abstract: Integrated Sensing and Communication between Daily Devices and mmWave Radars

Radar2: Passive Spy Radar Detection and Localization Using COTS Mmwave Radar

Low Power Millimeter Wave Radar System for the Visually Impaired

Mmripple: Communicating with Mmwave Radars Through Smartphone Vibration

Assisting the Visually Impaired: Multitarget Warning Through Millimeter Wave Radar and RGB-depth Sensors.

Underwater cross-medium communication and water surface target detection and tracking integrated system

Towards ISAC-Empowered Mmwave Radars by Capturing Modulated Vibrations

Talk2Radar: Talking to Mmwave Radars Via Smartphone Speaker

Poster Abstract: Environment-Independent Electronic Device Detection using Millimeter-Wave

mmVib: Micrometer-Level Vibration Measurement with mmWave Radar

Measuring Micrometer-Level Vibrations With mmWave Radar

VANET-Assisted Interference Mitigation for Millimeter-Wave Automotive Radar Sensors.

A $K$ -Band Portable FMCW Radar with Beamforming Array for Short-Range Localization and Vital-Doppler Targets Discrimination

Contactless Micron-Level Vibration Measurement with Millimeter Wave Radar

A Survey of Mmwave Radar-Based Sensing in Autonomous Vehicles, Smart Homes and Industry

RadioMic: Sound Sensing via mmWave Signals

Detection Of Bio-Signals From Body Movement Based On High-Dynamic-Range Doppler Radar Sensor (Invited)

Instrument-Based Noncontact Doppler Radar Vital Sign Detection System Using Heterodyne Digital Quadrature Demodulation Architecture

Radar Rainbow Beams For Wideband mmWave Communication: Beam Training And Tracking

RFMic-Phone: Robust Sound Acquisition Combining Millimeter-Wave Radar and Microphone