Acoustic-based sensing and applications: A survey

Yang Bai,Li Lu,Jerry Cheng,Jian Liu,Yingying Chen,Jiadi Yu
DOI: https://doi.org/10.1016/j.comnet.2020.107447
IF: 5.493
2020-11-01
Computer Networks
Abstract:<p>With advancements of wireless and sensing technologies, recent studies have demonstrated technical feasibility and effectiveness of using acoustic signals for sensing. In the past decades, low-cost audio infrastructures are widely-deployed and integrated into mobile and Internet of Things (IoT) devices to facilitate a broad array of applications including human activity recognition, tracking, localization, and security monitoring. The technology underpinning these applications lies in the analysis of propagation properties of acoustic signals (e.g., reflection, diffraction, and scattering) when they encounter human bodies. As a result, these applications serve as the foundation to support various daily functionalities such as safety protection, smart healthcare, and smart appliance interaction. The already-existing acoustic infrastructure could also complement RF-based localization and other approaches based on short-range communications such as Near-Field Communication (NFC) and Quick Response (QR) code. In this paper, we provide a comprehensive review on acoustic-based sensing in terms of hardware infrastructure, technical approaches, and its broad applications. First we describe different methodologies and techniques of using acoustic signals for sensing including Time-of-Arrival (ToA), Frequency Modulated Continuous Wave (FMCW), Time-Difference-of-Arrival (TDoA), and Channel Impulse Response (CIR). Then we classify various applications and compare different acoustic-based sensing approaches: in recognition and tracking, we review daily activity recognition, human health and behavioral monitoring hand gesture recognition, hand movement tracking, and speech recognition; in localization and navigation, we discuss ranging and direction finding, indoor and outdoor localization, and floor map construction; in security and privacy, we survey user authentication, keystroke snooping attacks, audio adversarial attacks, acoustic vibration attacks, and privacy protection schemes. Lastly we discuss future research directions and limitations of the acoustic-based sensing.</p>
computer science, information systems,telecommunications,engineering, electrical & electronic, hardware & architecture
What problem does this paper attempt to address?
The problems that this paper attempts to solve are: how to use acoustic signals for sensing and the technical feasibility and effectiveness of its wide - range applications. Specifically, the paper aims to provide a comprehensive review of acoustic - based sensing technologies, covering hardware infrastructure, main sensing technologies and their wide - range application fields. ### Problem Background In recent years, with the progress of wireless and sensing technologies, the technical feasibility of using acoustic signals for sensing has been verified and has shown good performance in practical applications. The existing low - power audio infrastructure has been widely deployed and integrated into mobile devices and Internet of Things (IoT) devices, supporting multiple applications such as human activity recognition, tracking, positioning and security monitoring. The basis of these applications lies in analyzing the propagation characteristics (such as reflection, diffraction and scattering) of acoustic signals when they encounter the human body. Therefore, these applications can support various daily functions such as security protection, smart healthcare and smart home appliance interaction. ### Main Problems 1. **Technical Implementation**: How to use acoustic signals for sensing through different methods and techniques (such as Time of Arrival (ToA), Frequency - Modulated Continuous Wave (FMCW), Time Difference of Arrival (TDoA) and Channel Impulse Response (CIR)). 2. **Application Scenarios**: - **Recognition and Tracking**: Including daily activity recognition, human health and behavior monitoring, gesture recognition, hand movement tracking and speech recognition. - **Positioning and Navigation**: Including ranging and direction finding, indoor and outdoor positioning and floor map construction. - **Security and Privacy**: Including user authentication, keyboard - listening attacks, audio adversarial attacks, acoustic vibration attacks and privacy protection schemes. 3. **Future Research Directions and Limitations**: Explore the limitations of existing technologies and future research directions. ### Solutions The paper provides detailed solutions in the following aspects: - **Hardware Infrastructure**: Describes hardware facilities such as microphones, speakers and audio chips for recording, generating and processing acoustic signals. - **Technical Methods**: Introduces different acoustic sensing techniques, such as signal strength change, phase change, Doppler effect, ToA, FMCW, TDoA and CIR. - **Application Classification**: Classifies and compares various applications, and discusses in detail the specific techniques and implementation methods for each application. - **Future Prospects**: Points out the limitations of existing technologies and proposes possible future research directions. ### Summary Through a comprehensive review of acoustic sensing technologies, the paper aims to provide researchers and engineers with a systematic understanding to help them better use acoustic signals for sensing and its wide - range applications.