Abstract:The rapidly growing adoption of sensor-enabled smartphones has greatly fueled the proliferation of applications that use phone sensors to monitor user behavior. A central sensor among these is the microphone which enables, for instance, the detection of valence in speech, or the identification of speakers. Deploying multiple of these applications on a mobile device to continuously monitor the audio environment allows for the acquisition of a diverse range of sound-related contextual inferences. However, the cumulative processing burden critically impacts the phone battery. To address this problem, we propose DSP.Ear - an integrated sensing system that takes advantage of the latest low-power DSP co-processor technology in commodity mobile devices to enable the continuous and simultaneous operation of multiple established algorithms that perform complex audio inferences. The system extracts emotions from voice, estimates the number of people in a room, identifies the speakers, and detects commonly found ambient sounds, while critically incurring little overhead to the device battery. This is achieved through a series of pipeline optimizations that allow the computation to remain largely on the DSP. Through detailed evaluation of our prototype implementation we show that, by exploiting a smartphone's co-processor, DSP.Ear achieves a 3 to 7 times increase in the battery lifetime compared to a solution that uses only the phone's main processor. In addition, DSP.Ear is 2 to 3 times more power efficient than a naive DSP solution without optimizations. We further analyze a large-scale dataset from 1320 Android users to show that in about 80-90% of the daily usage instances DSP.Ear is able to sustain a full day of operation (even in the presence of other smartphone workloads) with a single battery charge.

Undergraduate Signal Processing Laboratories for the Android Operating System

Labview-Based Virtual Laboratory for Digital Signal Processing

Android-based mobile educational platform for speech signal processing

Design of multi-functional portable electronic experimental box based on Labview and ARM

Virtual experiment system of signal analysis based on Web and MATLAB

A multipurpose toolkit for teaching DSP in an undergraduate course

An Audio Generator System For Experimental Studies On Acoustic Biology

Implementation of Remote Audio Acquisition System Based on Android Intelligent Mobile Terminal

Web-Based Interactive Virtual Laboratory System For Digital Circuit Experiment

Design of Inquiry Experiment of Spectrum Analyzer Based on Android

Development of Exploratory Experiment Teaching Platform for Signal Analysis and Processing

Portable Intelligent Oscilloscope Based on Innovative Education

Development of signal processing experimental platform based on GNU Radio framework

Learning digital signal processing using an interactive Jupyter notebook and smartphone accelerometer data

DSP.Ear: Leveraging Co-Processor Support for Continuous Audio Sensing on Smartphones

Non-Invasive Glucose Measuring Apparatus of the DSP and Android Software System Design

Real-World Relevant Learning with Android Smartphones

The Design of Matlab-DSP Development Environment for Control System

Computer-based learning in an undergraduate physics course: Interfacing a mobile phone and matlab to study oscillatory motion

Design and Implementation of Music Teaching Platform in College Based on Android Mobile Technology

Design and Implementation of Physics Simulation Experiment Based on the Android OS