Abstract:The massive parallelism provided by Graphics Processing Units (GPUs) to accelerate compute-intensive tasks makes it preferable for Real-Time Systems such as autonomous vehicles. Such systems require the execution of heavy Machine Learning (ML) and Computer Vision applications because of the computing power of GPUs. However, such systems need a guarantee of timing predictability. It means the Worst-Case Execution Time (WCET) of the application is estimated tightly and safely to schedule each application before its deadline to avoid catastrophic consequences. As more applications use GPUs, running many applications simultaneously on the same GPU becomes necessary. To provide predictable performance while the application is running in parallel, it must be WCET-aware, which GPUs do not fully support in a multitasking environment. Nvidia recently added a feature called the Multi-Process Service. It allows the different applications to run simultaneously in the same CUDA context by partitioning the compute resources of the GPU. Using this feature, we can measure the interference from co-running GPU applications to estimate WCET. In this paper, we propose a novel technique to estimate the WCET of the GPU kernel using an ML approach. Our approach is based on the application's source, and the model is trained based on the large data set. The approach is flexible and can be applied to different GPU-sharing mechanisms. We allow the victim and enemy kernel of the GPU to execute in parallel to get the maximum interference from the enemy to estimate the WCET of the victim kernel. Enemy kernels are chosen to cause a higher slowdown by acquiring the resources of the victim kernel. We compare our implementation with state-of-the-art approaches to show its effectiveness. Our ML approach reduces the time by 99% in most cases because inferences take only seconds to predict WCET, and the resource consumption required to estimate WCET compared to traditional approaches is minimal because we don't need to execute the application on GPU for hours. Although our approach does not offer safety guarantees because of its empirical nature, we observed that predicted WCETs are always higher than any observed execution times for all benchmarks, and the maximum overestimation factor observed is 11x.

Worst-Case Convergence Time of ML Algorithms via Extreme Value Theory

Utilizing Machine Learning Techniques for Worst-Case Execution Time Estimation on GPU Architectures

Deep Neural Network Approach to Estimate Early Worst-Case Execution Time

Generalization Analysis of Machine Learning Algorithms via the Worst-Case Data-Generating Probability Measure

An extreme worst-case risk measure by expectile

Toward Scalable Risk Analysis for Stochastic Systems Using Extreme Value Theory

High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise

Extremely Fast Convergence Rates for Extremum Seeking Control with Polyak-Ruppert Averaging

Worst-Case Input Generation for Concurrent Programs under Non-Monotone Resource Metrics

High-Probability Bounds for Stochastic Optimization and Variational Inequalities: the Case of Unbounded Variance

Modeling Extreme Events in Time Series Prediction

Multi-Group Fairness Evaluation via Conditional Value-at-Risk Testing

Accurately approximating extreme value statistics

Worst-case analysis of finite-time control policies

Exact Worst-Case Execution-Time Analysis for Implicit Model Predictive Control

High Probability Convergence Bounds for Non-convex Stochastic Gradient Descent with Sub-Weibull Noise

Artificial Intelligence Applied to Extreme Value Prediction of Non-Gaussian Processes with Bandwidth Effect and Non-monotonicity

How Much Data Is Sufficient to Learn High-Performing Algorithms?

Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise

High-Probability Complexity Bounds for Non-smooth Stochastic Convex Optimization with Heavy-Tailed Noise

Optimality in Mean Estimation: Beyond Worst-Case, Beyond Sub-Gaussian, and Beyond $1+α$ Moments