Accelerating Activity Inference on Edge Devices Through Spatial Redundancy in Coarse-Grained Dynamic Networks

Nanfu Ye,Lei Zhang,Di Xiong,Hao Wu,Aiguo Song

DOI: https://doi.org/10.1109/jiot.2024.3458441

IF: 10.6

2024-01-01

IEEE Internet of Things Journal

Abstract:During recent years, deep neural networks have achieved outstanding success in sensor-based human activity recognition (HAR). Particularly, dynamic convolution has emerged as a promising solution to accelerate activity inference of deep networks on mobile devices. Exploiting spatial redundancy, such a dynamic strategy can adaptively sample the salient areas of interest over sensor feature maps while skipping unimportant locations to avoid computational expenditure on activity-irrelevant disturbing areas. Despite theoretic efficiency, it has to rely on a binary-valued mask combined with element-wise multiplication, which potentially incurs non-contiguous memory access while performed at the finest granularity. To the best of our knowledge, most existing HAR literatures have always adopted hardware-agnostic FLOPs as an indicator to guide algorithm design, lacking delay-aware considerations about scheduling strategy and specific hardware characteristic. In this paper, we propose a delay-aware coarse-grained dynamic convolutional network called DACDNet to bridge the gap between theoretical FLOPs and realistic delay, which is highly challenging but less explored in ubiquitous HAR environments. Instead of theoretic FLOPs, we introduce a novel delay prediction model to guide HAR algorithm design while simultaneously considering scheduling strategy on various hardware platforms, especially multi-core processors like edge GPU devices. Experiments on multiple HAR benchmarks including WISDM, UniMiB-SHAR, and PAMAP2 demonstrate that our approach can significantly accelerate activity inference without sacrificing accuracy.