Abstract:This paper presents SensorNet which is a scalable and low-power embedded deep convolutional neural network (DCNN), designed to classify multimodal time series signals. Time series signals generated by different sensor modalities with different sampling rates are first converted to images (2-D signals), and then DCNN is utilized to automatically learn shared features in the images and perform the classification. SensorNet: 1) is scalable as it can process different types of time series data with variety of input channels and sampling rates; 2) does not need to employ separate signal processing techniques for processing the data generated by each sensor modality; 3) does not require expert knowledge for extracting features for each sensor data; 4) makes it easy and fast to adapt to new sensor modalities with a different sampling rate; 5) achieves very high detection accuracy for different case studies; and 6) has a very efficient architecture which makes it suitable to be deployed at Internet of Things and wearable devices. A custom low-power hardware architecture is also designed for the efficient deployment of SensorNet at embedded real-time systems. SensorNet performance is evaluated using three different case studies including physical activity monitoring, stand-alone tongue drive system, and stress detection, and it achieves an average detection accuracy of 98%, 96.2%, and 94% for each case study, respectively. We implement SensorNet using our custom hardware architecture on Xilinx FPGA (Artix-7) which on average consumes 246-<inline-formula> <tex-math notation="LaTeX">$\mu \text{J}$ </tex-math></inline-formula> energy. To further reduce the power consumption, SensorNet is implemented using application-specified integrated circuit at the post layout level in 65-nm CMOS technology which consumes approximately <inline-formula> <tex-math notation="LaTeX">$8\times $ </tex-math></inline-formula> lower power compared to the FPGA implementation. In addition, SensorNet is implemented on NVIDIA Jetson TX2 SoC (CPU + GPU) and compared to TX2 single-core CPU and GPU implementations, FPGA-based SensorNet obtains <inline-formula> <tex-math notation="LaTeX">$15\times $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$4\times $ </tex-math></inline-formula> improvement in energy consumption.

Real-time convolutional networks for sonar image classification in low-power embedded systems

High-performance Reconfigurable DNN Accelerator on a Bandwidth-limited Embedded System

A Scalable Real-time Semantic Segmentation Network for Autonomous Driving

A convolutional neural network accelerator for real-time underwater image recognition of autonomous underwater vehicle

A Low-Rank CNN Architecture for Real-Time Semantic Segmentation in Visual SLAM Applications

Efficient and Compact Convolutional Neural Network Architectures for Non-temporal Real-time Fire Detection

Rt-Seg: A Real-Time Semantic Segmentation Network For Side-Scan Sonar Images

Best Practices in Convolutional Networks for Forward-Looking Sonar Image Recognition

SensorNet: A Scalable and Low-Power Deep Convolutional Neural Network for Multimodal Data Classification

Active Sonar Image Classification Using Deep Convolutional Neural Network Evolved by Robust Comprehensive Grey Wolf Optimizer

Objectness Scoring and Detection Proposals in Forward-Looking Sonar Images with Convolutional Neural Networks

Towards Real-Time Object Detection on Embedded Systems.

Design and Implementation of Tiny Deep Neural Networks for Landing Pad Detection on UAVs

Temporal Patience: Efficient Adaptive Deep Learning for Embedded Radar Data Processing

ECNet: Efficient Convolutional Networks for Side Scan Sonar Image Segmentation

A Shadow Capture Deep Neural Network for Underwater Forward-Looking Sonar Image Detection

MiniNet: An extremely lightweight convolutional neural network for real-time unsupervised monocular depth estimation

Efficient convolutional neural networks on Raspberry Pi for image classification

MROD-CNN: A Miniature Re-parameterized One-Dimension Convolutional Neural Network for Underwater Source Localization

In-Air Imaging Sonar Sensor Network with Real-Time Processing Using GPUs

Real-Time Embedded Implementation of Improved Object Detector for Resource-Constrained Devices