Abstract:The growing demand for intelligent applications beyond the network edge, coupled with the need for sustainable operation, are driving the seamless integration of deep learning (DL) algorithms into energy-limited, and even energy-harvesting end-devices. However, the stochastic nature of ambient energy sources often results in insufficient harvesting rates, failing to meet the energy requirements for inference and causing significant performance degradation in energy-agnostic systems. To address this problem, we consider an on-device adaptive inference system equipped with an energy-harvester and finite-capacity energy storage. We then allow the device to reduce the run-time execution cost on-demand, by either switching between differently-sized neural networks, referred to as multi-model selection (MMS), or by enabling earlier predictions at intermediate layers, called early exiting (EE). The model to be employed, or the exit point is then dynamically chosen based on the energy storage and harvesting process states. We also study the efficacy of integrating the prediction confidence into the decision-making process. We derive a principled policy with theoretical guarantees for confidence-aware and -agnostic controllers. Moreover, in multi-exit networks, we study the advantages of taking decisions incrementally, exit-by-exit, by designing a lightweight reinforcement learning-based controller. Experimental results show that, as the rate of the ambient energy increases, energy- and confidence-aware control schemes show approximately 5% improvement in accuracy compared to their energy-aware confidence-agnostic counterparts. Incremental approaches achieve even higher accuracy, particularly when the energy storage capacity is limited relative to the energy consumption of the inference model.

Region-based Energy Neural Network for Approximate Inference

Amortized Bethe Free Energy Minimization for Learning MRFs

Learning Approximate Inference Networks for Structured Prediction

Unification of MAP Estimation and Marginal Inference in Recurrent Neural Networks

Relational Neural Markov Random Fields

A Neural Network Implementation for Free Energy Principle

Neural Network Implementation of Inference on Binary Markov Random Fields with Probability Coding

Neural Sampling in Hierarchical Exponential-family Energy-based Models

NeuroLifting: Neural Inference on Markov Random Fields at Scale

Energy-Aware Dynamic Neural Inference

Adversarial Localized Energy Network for Structured Prediction.

Rényi Neural Processes

Learning Proposals for Practical Energy-Based Regression

Energy Scaling Laws for Distributed Inference in Random Fusion Networks

Full Conditional Free Energy Based Inference

Multilevel Belief Propagation for Fast Inference on Markov Random Fields

ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation

Energy-efficient Neural Networks Using Approximate Computation Reuse.

Neural Autoregressive Distribution Estimation

Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation

Energy efficient building envelope using novel RBF neural network integrated affinity propagation