Abstract:With the increasing popularity of the Internet of Things and smart factories, robust sound acquisition plays an important role in communication and human–machine interaction, for example, in-vehicle voice control interactions, smart home, and robotic voice interaction. However, due to the inherent sensing manners of the widely used microphone system, the existing audio-only sound acquisition system has fundamental limitations in solving the cocktail problem and is difficult to adapt to complex acoustic environments. Considering the advantages of millimeter-wave (mmWave) radar in the field of target positioning and precise vibration measurement, the mmWave vibration measurement-based sound perception has great potential to solve the above problems. However, there are still difficulties in high-quality sound recovery. In this letter, we propose RFMic-Phone, a robust sound acquisition system combining a mmWave radar and a traditional microphone, which combines the sound signals captured by the mmWave radar and microphone, offering a novel approach for reliable sound acquisition in the complex acoustic environment. The microphone is used to obtain the mixed and high-fidelity audio signals, while the radar is used to obtain the vibration of the sound source. To achieve intelligent and effective feature fusion, we employ a deep learning framework with a modified convolutional encoder–decoder neural network structure. Moreover, we propose to utilize the real and imaginary spectra of the target sound source as the input of the network, allowing achieve high-quality target sound signals. The experimental results show that the RFMic-Phone can achieve robust and high-quality sound signals acquisition in a variety of complex acoustic environments.

Towards ISAC-Empowered Mmwave Radars by Capturing Modulated Vibrations

Mmripple: Communicating with Mmwave Radars Through Smartphone Vibration

Talk2Radar: Talking to Mmwave Radars Via Smartphone Speaker

MagneComm+: Near-Field Electromagnetic Induction Communication with Magnetometer

Poster Abstract: Integrated Sensing and Communication between Daily Devices and mmWave Radars

Integrated Sensing and Communication between Daily Devices and mmWave Radars.

Ultracomm: High-Speed And Inaudible Acoustic Communication

RFMic-Phone: Robust Sound Acquisition Combining Millimeter-Wave Radar and Microphone

Cross-Medium Communication Combining Acoustic Wave and Millimeter Wave: Anechoic Pool, Lake, Swimming Pool Experiments

Research on Information Transmission Technology Through the WaterAir Interface Combining a Sonar and a Radar

An Experimental Proof of Concept for Integrated Sensing and Communications Waveform Design

Mmphone: Sound Recovery Using Millimeter-Wave Radios with Adaptive Fusion Enhanced Vibration Sensing

Co-sharing Waveform Design for Millimeter-Wave Radar Communication Systems

A Novel Method for Speech Acquisition and Enhancement by 94 GHz Millimeter-Wave Sensor

An EBSPK-/ MPPSK-Modem Based Transceiver for Radar-Communications

RadioMic: Sound Sensing via mmWave Signals

Photonics-aided Integrated Sensing and Communications in Mmw Bands Based on a DC-offset QPSK-encoded LFMCW.

mmWave-Whisper: Phone Call Eavesdropping and Transcription Using Millimeter-Wave Radar

Mmeavesdropper: Signal Augmentation-based Directional Eavesdropping with Mmwave Radar.

Reconfigurable Intelligent Surface Assisted Spatial Modulation for Symbiotic Radio

ISAC Meets SWIPT: Multi-functional Wireless Systems Integrating Sensing, Communication, and Powering