Abstract:With the rapid development of Deep Learning, more and more applications on the cloud and edge tend to utilize large DNN (Deep Neural Network) models for improved task execution efficiency as well as decision-making quality. Due to memory constraints, models are commonly optimized using compression, pruning, and partitioning algorithms to become deployable onto resource-constrained devices. As the conditions in the computational platform change dynamically, the deployed optimization algorithms should accordingly adapt their solutions. To perform frequent evaluations of these solutions in a timely fashion, RMs (Regression Models) are commonly trained to predict the relevant solution quality metrics, such as the resulted DNN module inference latency, which is the focus of this paper. Existing prediction frameworks specify different RM training workflows, but none of them allow flexible configurations of the input parameters (e.g., batch size, device utilization rate) and of the selected RMs for different modules. In this paper, a deep learning module inference latency prediction framework is proposed, which i) hosts a set of customizable input parameters to train multiple different RMs per DNN module (e.g., convolutional layer) with self-generated datasets, and ii) automatically selects a set of trained RMs leading to the highest possible overall prediction accuracy, while keeping the prediction time / space consumption as low as possible. Furthermore, a new RM, namely MEDN (Multi-task Encoder-Decoder Network), is proposed as an alternative solution. Comprehensive experiment results show that MEDN is fast and lightweight, and capable of achieving the highest overall prediction accuracy and R-squared value. The Time/Space-efficient Auto-selection algorithm also manages to improve the overall accuracy by 2.5% and R-squared by 0.39%, compared to the MEDN single-selection scheme.

Toward Low-Latency Cross-Modal Communication: A Flexible Prediction Scheme

Edge Collaborative Learning Acceleration Based on Latency Prediction

SocialGrid: A TCN-enhanced Method for Online Discussion Forecasting

Automatic Depression Prediction Via Cross-Modal Attention-Based Multi-Modal Fusion in Social Networks

Time-Varying Channel Prediction for RIS-Assisted MU-MISO Networks Via Deep Learning

Driver intention prediction based on multi-dimensional cross-modality information interaction

Cross-Modal Semantic Communications

Cross-Modal Collaborative Communications

Stock Movement Prediction with Multimodal Stable Fusion via Gated Cross-Attention Mechanism

Heterogeneous Stream Scheduling for Cross-Modal Transmission

Real-Time Multimodal Turn-taking Prediction to Enhance Cooperative Dialogue during Human-Agent Interaction

GACL: Graph Attention Collaborative Learning for Temporal QoS Prediction

Dynamical User Intention Prediction Via Multi-modal Learning

Rate-Adaptive Coding Mechanism for Semantic Communications With Multi-Modal Data

A Bidirectional Context-Aware and Multi-Scale Fusion Hybrid Network for Short-Term Traffic Flow Prediction

MGCP: A Multi-Grained Correlation based Prediction Network for Multivariate Time Series

From Abstract to Details

Exploring the Benefits of Cross-Modal Coding

Motion and Context-Aware Audio-Visual Conditioned Video Prediction

Towards A Flexible Accuracy-Oriented Deep Learning Module Inference Latency Prediction Framework for Adaptive Optimization Algorithms

Adaptive contrastive learning based network latency prediction in 5G URLLC scenarios