Resource-Efficient Sensor Fusion via System-Wide Dynamic Gated Neural Networks

Chetna Singhal,Yashuo Wu,Francesco Malandrino,Sharon Ladron de Guevara Contreras,Marco Levorato,Carla Fabiana Chiasserini

2024-10-22

Abstract:Mobile systems will have to support multiple AI-based applications, each leveraging heterogeneous data sources through DNN architectures collaboratively executed within the network. To minimize the cost of the AI inference task subject to requirements on latency, quality, and - crucially - reliability of the inference process, it is vital to optimize (i) the set of sensors/data sources and (ii) the DNN architecture, (iii) the network nodes executing sections of the DNN, and (iv) the resources to use. To this end, we leverage dynamic gated neural networks with branches, and propose a novel algorithmic strategy called Quantile-constrained Inference (QIC), based upon quantile-Constrained policy optimization. QIC makes joint, high-quality, swift decisions on all the above aspects of the system, with the aim to minimize inference energy cost. We remark that this is the first contribution connecting gated dynamic DNNs with infrastructure-level decision making. We evaluate QIC using a dynamic gated DNN with stems and branches for optimal sensor fusion and inference, trained on the RADIATE dataset offering Radar, LiDAR, and Camera data, and real-world wireless measurements. Our results confirm that QIC matches the optimum and outperforms its alternatives by over 80%.

Artificial Intelligence,Networking and Internet Architecture

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problem of how to optimize sensor fusion and deep neural network (DNN) architectures when supporting multiple AI - based applications in mobile systems. Specifically, the paper focuses on minimizing the cost (such as energy consumption) during the inference process while meeting the latency and quality (especially reliability) requirements of inference tasks. To achieve this goal, the paper proposes a new algorithmic strategy named **Quantile - constrained Inference (QIC)**. #### Main problems include: 1. **Selecting sensors / data sources**: Determine which sensors or data sources should be used for each application. 2. **DNN architecture design**: Select parts of the DNN architecture (such as stems and branches) suitable for specific application scenarios. 3. **Network node allocation**: Decide on which network nodes (such as mobile devices and edge servers) to deploy different parts of the DNN. 4. **Resource allocation**: Allocate computing and communication resources reasonably to ensure an efficient and low - energy - consumption inference process. #### Key contributions of the paper: - **Dynamic gated neural networks**: Introduced dynamic gated neural networks with branches, which can adaptively adjust the execution path according to the characteristics of the input data. - **QIC algorithm**: Based on quantile - constrained policy optimization (QCPO), QIC can minimize energy consumption while meeting the inference quality (such as accuracy) and latency requirements. - **Infrastructure - level decision - making**: Combine the internal structure of the DNN with infrastructure operations and optimize resource utilization by dynamically adjusting the network configuration. #### Application scenarios: The paper uses the RADIATE dataset, which contains radar, LiDAR, and camera data as well as actual wireless measurement data, to evaluate the performance of the QIC algorithm. The experimental results show that QIC not only significantly outperforms existing methods (more than 80% performance improvement) but also approaches the optimal solution. #### Summary: This paper solves the problem of how to improve AI inference efficiency through intelligent decision - making and resource optimization in a multi - sensor, multi - application environment. The method it proposes not only improves the accuracy and speed of inference but also significantly reduces energy consumption and is applicable to various real - world mobile and edge - computing scenarios.

Resource-Efficient Sensor Fusion via System-Wide Dynamic Gated Neural Networks

Resource-aware Deployment of Dynamic DNNs over Multi-tiered Interconnected Systems

Optimized Gated Deep Learning Architectures for Sensor Fusion

Joint Device Scheduling and Resource Allocation for ISCC-Based Multi-View-Multi-Task Inference

Multi-Agent Systems for Collaborative Inference Based on Deep Policy Q-Inference Network

QoS-Aware Inference Acceleration Using Adaptive Depth Neural Networks

Enabling Efficient Deep Convolutional Neural Network-based Sensor Fusion for Autonomous Driving

Scalable Primitives for Generalized Sensor Fusion in Autonomous Vehicles

Integrated Sensing and Communication for Edge Inference with End-to-End Multi-View Fusion

QIANets: Quantum-Integrated Adaptive Networks for Reduced Latency and Improved Inference Times in CNN Models

Learning Dynamic Computing Resource Allocation in Convolutional Neural Networks for Wireless Interference Identification

A Fine-Grained End-to-End Latency Optimization Framework for Wireless Collaborative Inference

Energy-efficient and Privacy-aware Social Distance Monitoring with Low-resolution Infrared Sensors and Adaptive Inference

Heterogeneous Scheduling of Deep Neural Networks for Low-power Real-time Designs

Collaborative Inference Acceleration Integrating DNN Partitioning and Task Offloading in Mobile Edge Computing

Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AI

Fusion-FlowNet: Energy-Efficient Optical Flow Estimation using Sensor Fusion and Deep Fused Spiking-Analog Network Architectures

Optimized Data Rate Allocation for Dynamic Sensor Fusion over Resource Constrained Communication Networks

Incremental Training and Group Convolution Pruning for Runtime DNN Performance Scaling on Heterogeneous Embedded Platforms

FlexSensing: A QoI and Latency-Aware Task Allocation Scheme for Vehicle-Based Visual Crowdsourcing Via Deep Q-Network.

Energy optimized data fusion approach for scalable wireless sensor network using deep learning-based scheme