Abstract:Deep convolutional neural networks (DCNNs) are currently popular in human activity recognition (HAR) applications. However, in the face of modern artificial intelligence sensor-based games, many research achievements cannot be practically applied on portable devices (i.e., smart phone, VR/AR). DCNNs are typically resource-intensive and too large to be deployed on portable devices, and thus, this limits the practical application of complex activity detection. In addition, since portable devices do not possess high-performance graphic processing units, there is hardly any improvement in Action Game (ACT) experience. Besides, in order to deal with multi-sensor collaboration, all previous HAR models typically treated the representations from different sensor signal sources equally. However, distinct types of activities should adopt different fusion strategies. In this paper, a novel scheme is proposed. This scheme is used to train 2-bit CNNs with weights and activations constrained to {-0.5, 0, 0.5}. It takes into account the correlation between different sensor signal sources and the activity types. This model, which we refer to as DFTerNet, aims at producing a more reliable inference and better trade-offs for practical applications. It is known that quantization of weights and activations can substantially reduce memory size and use more efficient bitwise operations to replace floating or matrix operations to achieve much faster calculation and lower power consumption. Our basic idea is to exploit quantization of weights and activations directly in pre-trained filter banks and adopt dynamic fusion strategies for different activity types. Experiments demonstrate that by using a dynamic fusion strategy, it is possible to exceed the baseline model performance by up to ~5% on activity recognition data sets, such as the OPPORTUNITY and PAMAP2 data sets. Using the quantization method proposed, we were able to achieve performances closer to that of the full-precision counterpart. These results were also verified using the UniMiB-SHAR data set. In addition, the proposed method can achieve ~9x acceleration on CPUs and ~11x memory saving.

FreqSense: Adaptive Sampling Rates for Sensor-Based Human Activity Recognition Under Tunable Computational Budgets

Dynamic Inference via Localizing Semantic Intervals in Sensor Data for Budget-Tunable Activity Recognition

Deep Convolutional Networks With Tunable Speed–Accuracy Tradeoff for Human Activity Recognition Using Wearables

CoSS: Co-optimizing Sensor and Sampling Rate for Data-Efficient AI in Human Activity Recognition

Continual learning in sensor-based human activity recognition: An empirical benchmark analysis

Learning Datum-Wise Sampling Frequency for Energy-Efficient Human Activity Recognition

Channel Attention for Sensor-based Activity Recognition: Embedding Features into All Frequencies in DCT Domain

Innovative Dual-Decoupling CNN with Layer-wise Temporal-Spatial Attention for Sensor-Based Human Activity Recognition

DFTerNet: Towards 2-Bit Dynamic Fusion Networks for Accurate Human Activity Recognition.

Online Continual Learning for Human Activity Recognition

Smartphone Sensor-Based Human Activity Recognition Robust to Different Sampling Rates

Improving Human Activity Recognition Performance by Data Fusion and Feature Engineering.

Beyond Isolated Frames: Enhancing Sensor-Based Human Activity Recognition through Intra- and Inter-Frame Attention

Multiscale Deep Feature Learning for Human Activity Recognition Using Wearable Sensors

The SES Framework and Frequency Domain Information Fusion Strategy for Human Activity Recognition

RF-Based Human Activity Recognition Using Signal Adapted Convolutional Neural Network

Wearable Sensor-Based Residual Multifeature Fusion Shrinkage Networks for Human Activity Recognition

An Efficient Diverse-Branch Convolution Scheme for Sensor-Based Human Activity Recognition

Attention-Based Deep Learning Framework for Human Activity Recognition with User Adaptation

RapidHARe: A computationally inexpensive method for real-time human activity recognition from wearable sensors

Deep Neural Networks for Sensor-Based Human Activity Recognition Using Selective Kernel Convolution